See also: IRC log
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514
MH: Eric is unable to make today's meeting so
Martiyn is filling in for him.
... please review editor's draft next week and the following week we'll
discuss. Next week is a public holiday in Europe so no meeting, so please
review draft
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/comments-20140130#comment77
MH: first today: comment #77
... regarding evaluating third party content
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#considerations
MH: is this re-write of that section okay?
... explained the re-write, please +1 if you agree
+1
<martijnhoutepen> Agree to rewrite?
<martijnhoutepen> +1
SA: don't understand how the re-write addresses the comments
MH: original text contained ambiguous terms such as 'however'. Re-write to make it clearer.
SA: the use is in the use of the word 'third party'
SA: differential view doesn't really show this
... problem is using the word 'third party' of conformance and then content
which refers to WCAG
... in this we don't mean third party as defined by WCAG
MH: we need to find out which one this is supposed to be - maybe 'external'
SA: disambiguate from third-party content
VC: comment 77 in the disposition of comments is evaluating third-party content - which is WCAG definition
MH: should put in the WCAG definition
ME: in reading the re-written paragraph addresses the issue of extrapolating from looking at a number of pages to the website as a whole in respect to evaluating conformance. Seems to have disappeared from the re-write.
SA: re-reading the comment - it's about content, not about the conformance statement
Sa: we don't talk about evaluating - maybe add a
sentence or 2 to the current section to say something like "for third party
content you need to check..."
... could just rephrase "for third party content that is not under the control
of the website the evaluator needs to determine whether this continuously
monitored and repaired, meets WCAG 2 and that its clearly identified for any
non-conforming sections"
... check if errors in the third-party content can affect the other content
... this clarification needs to be added
MH: add to the definition so that it is more clear to the evaluator
<shadi> "Editors to add some guidance like "ask website owner if monintoring is implemented" and "check if errors in third party content can affect other content" etc (see minutes for more discussion)"
<shadi> (from Comment 77)
<shadi> +1
<martijnhoutepen> +1
MH: is everbody in agrement to adding the sentence?
+1
<Mike_Elledge> +1
<Sarah_Swierenga> +1
MH: seems to be agreement
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/comments-20140130#comment82
next agenda comment - #82
MH: EOWG thnks too much link to definitions which makes the document confusing. Maybe we can check the links and make sure that we don't unnecessarily link to definitions that are not essential in that context.
<shadi> +1
+1
<martijnhoutepen> +1
<Sarah_Swierenga> +1
SA: editors need to do that - Shadi & Eric
MH: this is important for the more difficult parts of the content
<Mike_Elledge> +1 certainly not link to EOWG :^)
SA: could be an older comment as there were more links in an earlier version
next comment: ID 90
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/comments-20140130#comment90
MH: put links at the end of sentences - not in
the middle of the sentence
... just relocating the links to the end of sentences. Editors can do this.
SA: this is a bit more substantial, but agree with comment
MH: has already improved since previous version
<shadi> [[Some of the content might already be reflected in the web pages and web page states selected in _Step 3.a: Include Common Web Pages of the Website_ and _Step 3.c: Include Other Relevant Web Pages_.]]
SA: agree with the comment, but not too sure what to do. Discuss with Eric.
MH: Have we decided what random is? Agendum 4
<martijnhoutepen> http://www.w3.org/2014/05/15-eval-minutes.html
MH: last week we ended on a discussion about randomness and random sampling
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/comments-20140130#comment23
MH: should we skew the sample toward frequently visited pages or frequently used pages. The consensus was that the random must be random.
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#step3e
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#step2e
MH: relates to step 3e. We need to keep the
random, random. It is to validate our structured sampling. This was decided
previously. However we can put in top search results and maybe changing step 2e
where we say at 'identify other relevant web pages'
... top visited pages are relevant to people with and without disability. This
could be added there.
AG: last week we discussed how people are meant to be selecting random pages. If this is left to the evaluator, they may not be looking totally randomly across the site. The evaluator needs to identify how they select the random sample and making sure it is really random - without or without a tool
MH: we provide some suggestions to the evaluators - such as make a script that will generate a list of all pages, use search engines. We do provide some guidance. Do we need more guidance?
AG: it is not perfectly random if someone is
involved in the selection process
... if there are 10,000 how random is the selection going to be? Mostly it
won't be random, will be skewed.
... we have emphasized that we need random pages
MH: we do state that even though the random is not strictly scientifically random, it can fill the importance for the applicability and reliability of the results. It adds to the reliability of the evlauation.
<shadi> [[While the random sample need not be selected according to strictly scientific criteria, the scope of the selection needs to span the entire scope of the website (any web page and web page state on the website may be selected), and the selection of individual web page and web page states does not follow a predictable pattern.]]
SA: I agree. We decided on the above sentence
... it will depend on the website. If you have a tool that will generate
random-like pages, that would be fine. Otherwise your method of random samplng
should span the entire score of 10,000 pages.
... we decided not to overdo this bit as the effort will outweigh the
benefits. It is to cross-check the results.
... this differs from the structured sample. We might want to skew the
structured section to look at certain types of pages.
... the guidaance here is sufficient. You as an evaluator need to decide what
random is in the context - a tool can be helpful if you have something at hand.
You need to ensure that your own structured sample is a sufficient example of
the website
MH: we don't want to stipulate an algorithm. We need to be clear in that the random sample may not be really random
AG: understand its a good idea to pick another selection of pages to cross-check yours. But if we say random, it becomes a task to discover what random really is. You might want to say to select a few other pages to validate your pages
SA: maybe another term? randomly picked. Unstructured doesn't describe it. It shouldn't be just 'select a couple of pages'. We should say something about what kind of pages and how you should select them.
Kathy: as far as the random pages, its hard to select random, especially for applications. What if you say that you need to select additional pages and give guidance on how to select those pages. When we do an audit to make sure that people have made fixes across the website we want to choose pages that weren't chosen the same way as the first sample. Just give additional direction. They are
still somewhat randomly picked by the evaluator.
SA: seems to be going counter to Alistair. Which is to not to pick something intentially. That would have the tendency to have some kind of bias.
Kathy: how are you going to be able to know what the pages are in an application?
SA: using an application would be different - pages belong to the complete process requirement. If you select a page from the application that happens to be on the website, would be different. It will be too difficult to define what 'random' is.
Kathy: it isn't possible to create a random sample from an application - due to dynamic creation of pages
MH: maybe the blind-folded colleague in that
application
... what do most people mean as 'random' is not scientifically 'random'
AG: you could say 'pick some pages at random' means your actions are random, rather than the pages are random
I like that Alistair
Kathy: good suggestion
SA: a good suggestion
... 'picking at random a further selection'
MH: shadi can you prepare something along that line
+1
<martijnhoutepen> +1
<shadi> +1
<Kathy> +1
MH: that's all of the agenda items. No meeting next week because of European holiday. Next meetingJune 5th
next time pick up on discussion 4C on the list
MH: please work on this items on the list
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#step4c
<martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#step4c
SA: there were several issues relating to this in
the discussion. AG raised the issue that there is a notion where you check a
page and find it is non-conforming and then you find an alternate version. This
isn't the same as WCAG. Section 4C and 4A need to be rolled together.
... the definition of 'conforming alternate version' has certain requirements
for both the page and the aternate version in order for it to qualify as an
alternate version. A brief clarifying statement is needed - 'a web page has an
alternate version that meets the requirement of a conforming alternate version'
to encourage people to look at the WCAG requirements. This is more editorial
change.
Mh: in WCAG is it more of a process - together they may conform. Although the web page needs to have certain accessibility requirements - you need to be able to get to the alternate version. If the initial page is too broken, it may not qualify.
AG: I agree. You only need to check any web page and its conforming alternate version that it is done properly. You have to check the page 100% against all the SC. I expected you to be able to check that the link is provided and then to check that alternate version.
SA: on a case by case basis. It may not always be apparent that there are alternate verisons. Sometimes the page is designed to work with certain technologies or user groups. One page willnot have something, but the other page could have that. EG higher contrast and lower contrast. They would need to identify this.
AG: if they present a page with a conforming
version and you include it in your sample, its going to fail and throw off the
statistics for that website. YOu have been told that this page will fail,
that's why you give an alternate version
... when you provide a link to a conforming alternative version. Together they
conform. You know the page with the link to the other page will fail.
MH: maybe the page does not fail because they have provided a passing version
AG: why would you check the original page if you know there is a conforming alternate version
SA: if you put them together you will get the results
AG: I provided some wording
SA: we might actually pull that together within 4A. We'll look at the wording suggestion and propose something for people to review.
Review May 14 Editor's draft for next week
<Sarah_Swierenga> good discussion. have a good holiday weekend (US) and/or a good holiday next week.
MH: everyone please read the emails and if you can carry on the discussion.
AG: a lot of the misunderstandings that get into the list are from people who don't read the emails correctly
Mh: please be patient with each other, remember
not everyone has English as their first language. If it isn't clear, then ask
on the call.
... see you all in 2 weeks. Please read the discussions on the list and the
editor draft.