13:48:20 <RRSAgent> RRSAgent has joined #eval
13:48:20 <RRSAgent> logging to http://www.w3.org/2014/05/22-eval-irc
13:48:22 <trackbot> RRSAgent, make logs world
13:48:22 <Zakim> Zakim has joined #eval
13:48:24 <trackbot> Zakim, this will be 3825
13:48:24 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 12 minutes
13:48:25 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
13:48:25 <trackbot> Date: 22 May 2014
13:51:31 <Mike_Elledge> Mike_Elledge has joined #eval
13:51:59 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
13:52:06 <Zakim> +Mike_Elledge
13:56:01 <Vivienne> Vivienne has joined #eval
13:57:58 <Zakim> +[IPcaller]
13:58:04 <Vivienne> zakim, IPcaller is me
13:58:04 <Zakim> +Vivienne; got it
13:58:41 <Zakim> +MartijnHoutepen
13:58:43 <shadi> zakim, call shadi-617
13:58:43 <Zakim> ok, shadi; the call is being made
13:58:44 <Zakim> +Shadi
13:59:12 <shadi> zakim, who is on the phone?
13:59:12 <Zakim> On the phone I see Mike_Elledge, Vivienne, MartijnHoutepen, Shadi
14:00:13 <shadi> regrets: Detlev, Liz, Tim, Eric
14:00:36 <agarrison> agarrison has joined #eval
14:02:12 <Sarah_Swierenga> Sarah_Swierenga has joined #eval
14:02:30 <Zakim> +[IPcaller]
14:02:31 <agarrison> zakim, ipcaller is alistair
14:02:32 <Zakim> +alistair; got it
14:03:12 <shadi> scribe: Vivienne
14:03:31 <Vivienne> zakim, mute me
14:03:31 <Zakim> Vivienne should now be muted
14:03:32 <Zakim> +Sarah_Swierenga
14:03:54 <shadi> agenda: http://lists.w3.org/Archives/Public/public-wai-evaltf/2014May/0033.html
14:03:56 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514
14:04:01 <shadi> chair: Martijn
14:04:09 <Vivienne> MH: Eric is unable to make today's meeting so Martiyn is filling in for him.
14:04:26 <shadi> zakim, mute me
14:04:26 <Zakim> Shadi should now be muted
14:04:46 <Vivienne> MH: please review editor's draft next week and the following week we'll discuss.  Next week is a public holiday in Europe so no meeting, so please review draft
14:04:48 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/comments-20140130#comment77
14:04:52 <Vivienne> MH: first today: comment #77
14:05:18 <Vivienne> MH: regarding evaluating third party content
14:05:27 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#considerations
14:05:50 <martijnhoutepen> q?
14:05:53 <Vivienne> MH: is this re-write of that section okay?
14:06:56 <Vivienne> MH: explained the re-write, please +1 if you agree
14:07:01 <Vivienne> +1
14:07:03 <martijnhoutepen> Agree to rewrite?
14:07:05 <martijnhoutepen> +1
14:07:08 <shadi> q+
14:07:16 <martijnhoutepen> ack shadi
14:07:18 <shadi> ack me
14:08:26 <Vivienne> SA: don't understand how the re-write addresses the comments
14:08:49 <Vivienne> MH: original text contained ambiguous terms such as 'however'.  Re-write to make it clearer.
14:09:05 <Vivienne> q+
14:09:24 <Kathy> Kathy has joined #eval
14:09:36 <Vivienne> SA: the use is in the use of the word 'third party'
14:09:41 <shadi> http://services.w3.org/htmldiff?doc1=http%3A%2F%2Fwww.w3.org%2FTR%2FWCAG-EM%2F&doc2=http%3A%2F%2Fwww.w3.org%2FWAI%2FER%2Fconformance%2FED-methodology-20140514#considerations
14:09:57 <Vivienne> SA: differential view doesn't really show this
14:10:23 <Zakim> +Kathy_Wahlbin
14:10:33 <Vivienne> Sa: problem is using the word 'third party' of conformance and then content which refers to WCAG
14:10:47 <Vivienne> SA: in this we don't mean third party as defined by WCAG
14:11:04 <Vivienne> MH: we need to find out which one this is supposed to be - maybe 'external'
14:11:09 <martijnhoutepen> q?
14:11:21 <Mike_Elledge> q+
14:11:38 <Vivienne> SA: disambiguate from third-party content
14:11:48 <Mike_Elledge_> Mike_Elledge_ has joined #eval
14:11:50 <Vivienne> ack me
14:13:13 <Vivienne> VC: comment 77 in the disposition of comments is evaluating third-party content - which is WCAG definition
14:13:18 <Vivienne> zakim, mute me
14:13:18 <Zakim> Vivienne should now be muted
14:13:20 <Mike_Elledge> q+
14:13:31 <Vivienne> MH: should put in the WCAG definition
14:14:12 <Vivienne> ME: in reading the re-written paragraph addresses the issue of extrapolating from looking at a number of pages to the website as a whole in respect to evaluating conformance.  Seems to have disappeared from the re-write.
14:15:03 <Vivienne> SA: re-reading the comment - it's about content, not about the conformance statement
14:15:36 <shadi> http://services.w3.org/htmldiff?doc1=http%3A%2F%2Fwww.w3.org%2FTR%2FWCAG-EM%2F&doc2=http%3A%2F%2Fwww.w3.org%2FWAI%2FER%2Fconformance%2FED-methodology-20140514#thirdparty
14:15:56 <Vivienne> Sa: we don't talk about evaluating - maybe add a sentence or 2 to the current section to say something like "for third party content you need to check..."
14:16:26 <martijnhoutepen> q?
14:16:29 <martijnhoutepen> q- mi
14:16:40 <Vivienne> Sa: could just rephrase "for third party content that is not under the control of the website the evaluator needs to determine whether this continuously monitored and repaired, meets WCAG 2 and that its clearly identified for any non-conforming sections"
14:17:02 <Vivienne> SA: check if errors in the third-party content can affect the other content
14:17:19 <Vivienne> SA: this clarification needs to be added
14:17:30 <Vivienne> MH: add to the definition so that it is more clear to the evaluator
14:17:31 <shadi> "Editors to add some guidance like "ask website owner if monintoring is implemented" and "check if errors in third party content can affect other content" etc (see minutes for more discussion)"
14:17:42 <shadi> (from Comment 77)
14:18:04 <shadi> +1
14:18:08 <martijnhoutepen> +1
14:18:12 <Vivienne> MH: is everbody in agrement to adding the sentence?
14:18:14 <Vivienne> +1
14:18:26 <Mike_Elledge> +1
14:18:28 <Sarah_Swierenga> +1
14:18:37 <Vivienne> MH: seems to be agreement
14:18:43 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/comments-20140130#comment82
14:18:43 <Vivienne> next agenda comment - #82
14:19:39 <Vivienne> MH: EOWG thnks too much link to definitions which makes the document confusing.  Maybe we can check the links and make sure that we don't unnecessarily link to definitions that are not essential in that context.
14:19:42 <shadi> +1
14:19:51 <Vivienne> +1
14:20:01 <martijnhoutepen> +1
14:20:04 <Sarah_Swierenga> +1
14:20:10 <Vivienne> SA: editors need to do that - Shadi & Eric
14:20:47 <Vivienne> MH: this is important for the more difficult parts of the content
14:20:49 <Mike_Elledge> +1 certainly not link to EOWG  :^)
14:21:05 <Vivienne> SA: could be an older comment as there were more links in an earlier version
14:21:12 <Vivienne> next comment: ID 90
14:21:15 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/comments-20140130#comment90
14:21:43 <Vivienne> MH: put links at the end of sentences - not in the middle of the sentence
14:22:03 <Vivienne> MH: just relocating the links to the end of sentences.  Editors can do this.
14:22:25 <Vivienne> SA: this is a bit more substantial, but agree with comment
14:22:40 <Vivienne> MH: has already improved since previous version
14:22:51 <shadi> [[Some of the content might already be reflected in the web pages and web page states selected in _Step 3.a: Include Common Web Pages of the Website_ and _Step 3.c: Include Other Relevant Web Pages_.]]
14:23:34 <Vivienne> SA: agree with the comment, but not too sure what to do. Discuss with Eric.
14:23:54 <martijnhoutepen> q?
14:24:14 <Vivienne> MH: Have we decided what random is? Agendum 4
14:24:18 <martijnhoutepen>  http://www.w3.org/2014/05/15-eval-minutes.html
14:24:30 <Vivienne> MH: last week we ended on a discussion about randomness and random sampling
14:24:32 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/comments-20140130#comment23
14:25:07 <Vivienne> MH: should we skew the sample toward frequently visited pages or frequently used pages.  The consensus was that the random must be random.
14:25:11 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#step3e
14:26:01 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#step2e
14:26:16 <Vivienne> MH: relates to step 3e.  We need to keep the random, random.  It is to validate our structured sampling.  This was decided previously.  However we can put in top search results and maybe changing step 2e where we say at 'identify other relevant web pages'
14:26:30 <agarrison> q+
14:26:31 <shadi> q+
14:26:40 <Vivienne> MH: top visited pages are relevant to people with and without disability.  This could be added there.
14:27:52 <Vivienne> AG: last week we discussed how people are meant to be selecting random pages.  If this is left to the evaluator, they may not be looking totally randomly across the site.  The evaluator needs to identify how they select the random sample and making sure it is really random - without or without a tool
14:28:32 <Vivienne> MH: we provide some suggestions to the evaluators - such as make a script that will generate a list of all pages, use search engines.  We do provide some guidance.  Do we need more guidance?
14:28:49 <Vivienne> AG: it is not perfectly random if someone is involved in the selection process
14:29:20 <Vivienne> AG: if there are 10,000 how random is the selection going to be?  Mostly it won't be random, will be skewed.
14:29:36 <Vivienne> AG: we have emphasized that we need random pages
14:30:04 <agarrison> q-
14:30:07 <shadi> ack me
14:30:10 <Vivienne> MH: we do state that even though the random is not strictly scientifically random, it can fill the importance for the applicability and reliability of the results.  It adds to the reliability of the evlauation.
14:30:12 <shadi> [[While the random sample need not be selected according to strictly scientific criteria, the scope of the selection needs to span the entire scope of the website (any web page and web page state on the website may be selected), and the selection of individual web page and web page states does not follow a predictable pattern.]]
14:30:23 <Vivienne> SA: I agree. We decided on the above sentence
14:31:29 <Vivienne> SA: it will depend on the website.  If you have a tool that will generate random-like pages, that would be fine.  Otherwise your method of random samplng should span the entire score of 10,000 pages.
14:31:55 <Vivienne> SA: we decided not to overdo this bit as the effort will outweigh the benefits.  It is to cross-check the results.
14:32:32 <Vivienne> SA: this differs from the structured sample.  We might want to skew the structured section to look at certain types of pages.
14:32:32 <martijnhoutepen> q?
14:33:25 <Vivienne> Sa: the guidaance here is sufficient.  You as an evaluator need to decide what random is in the context - a tool can be helpful if you have something at hand.  You need to ensure that your own structured sample is a sufficient example of the website
14:33:57 <Vivienne> MH: we don't want to stipulate an algorithm.  We need to be clear in that the random sample may not be really random
14:34:50 <Vivienne> AG: understand its a good idea to pick another selection of pages to cross-check yours. But if we say random, it becomes a task to discover what random really is.  You might want to say to select a few other pages to validate your pages
14:35:13 <Kathy> q+
14:35:28 <Vivienne> SA: maybe another term?  randomly picked.  Unstructured doesn't describe it.  It shouldn't be just 'select a couple of pages'.  We should say something about what kind of pages and how you should select them.
14:36:08 <shadi> q+
14:36:13 <shadi> ack kathy
14:36:48 <Vivienne> Kathy: as far as the random pages, its hard to select random, especially for applications.  What if you say that you need to select additional pages and give guidance on how to select those pages.  When we do an audit to make sure that people have made fixes across the website we want to choose pages that weren't chosen the same way as the first sample.  Just give additional direction.  They are
14:36:48 <Vivienne> still somewhat randomly picked by the evaluator.
14:36:58 <shadi> ack me
14:37:55 <Vivienne> SA: seems to be going counter to Alistair.  Which is to not to pick something intentially.  That would have the tendency to have some kind of bias.
14:38:05 <agarrison> q+
14:38:18 <Vivienne> Kathy: how are you going to be able to know what the pages are in an application?
14:38:31 <agarrison> q-
14:39:21 <Vivienne> SA: using an application would be different - pages belong to the complete process requirement.  If you select a page from the application that happens to be on the website, would be different.  It will be too difficult to define what 'random' is.
14:39:39 <Vivienne> Kathy: it isn't possible to create a random sample from an application - due to dynamic creation of pages
14:39:50 <Vivienne> MH: maybe the blind-folded colleague in that application
14:39:56 <agarrison> q+
14:40:08 <Vivienne> MH: what do most people mean as 'random' is not scientifically 'random'
14:40:12 <shadi> q+
14:40:30 <Vivienne> AG: you could say 'pick some pages at random'  means your actions are random, rather than the pages are random
14:40:40 <Vivienne> I like that Alistair
14:40:42 <shadi> q-
14:41:08 <Vivienne> Kathy: good suggestion
14:41:17 <Vivienne> SA: a good suggestion
14:41:41 <Vivienne> SA: 'picking at random a further selection'
14:41:54 <agarrison> q-
14:42:03 <Vivienne> MH: shadi can you prepare something along that line
14:42:07 <Vivienne> +1
14:42:07 <martijnhoutepen> +1
14:42:12 <shadi> +1
14:42:14 <Kathy> +1
14:42:56 <Vivienne> MH: that's all of the agenda items.  No meeting next week because of European holiday.  Next meetingJune 5th
14:43:05 <Vivienne> next time pick up on discussion 4C on the list
14:43:20 <Vivienne> MH: please work on this items on the list
14:44:01 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#step4c
14:44:43 <martijnhoutepen> http://www.w3.org/WAI/ER/conformance/ED-methodology-20140514#step4c
14:46:07 <Vivienne> SA: there were several issues relating to this in the discussion.  AG raised the issue that there is a notion where you check a page and find it is non-conforming and then you find an alternate version.  This isn't the same as WCAG.  Section 4C and 4A need to be rolled together.
14:47:04 <agarrison> q+
14:47:24 <Vivienne> SA: the definition of 'conforming alternate version' has certain requirements for both the page and the aternate version in order for it to qualify as an alternate version.  A brief clarifying statement is needed - 'a web page has an alternate version that meets the requirement of a conforming alternate version' to encourage people to look at the WCAG requirements. This is more editorial change.
14:48:54 <Vivienne> Mh: in WCAG is it more of a process - together they may conform.  Although the web page needs to have certain accessibility requirements - you need to be able to get to the alternate version.  If the initial page is too broken, it may not qualify.
14:49:24 <martijnhoutepen> ack agar
14:50:17 <Vivienne> AG: I agree.  You only need to check any web page and its conforming alternate version that it is done properly.  You have to check the page 100% against all the SC.  I expected you to be able to check that the link is provided and then to check that alternate version.
14:51:30 <Vivienne> SA: on a case by case basis.  It may not always be apparent that there are alternate verisons.  Sometimes the page is designed to work with certain technologies or user groups.  One page willnot have something, but the other page could have that.  EG higher contrast and lower contrast.    They would need to identify this.
14:52:26 <Vivienne> AG: if they present a page with a conforming version and you include it in your sample, its going to fail and throw off the statistics for that website.  YOu have been told that this page will fail, that's why you give an alternate version
14:53:00 <Vivienne> AG: when you provide a link to a conforming alternative version.  Together they conform.  You know the page with the link to the other page will fail.
14:53:19 <Vivienne> MH:  maybe the page does not fail because they have provided a passing version
14:54:29 <Vivienne> AG: why would you check the original page if you know there is a conforming alternate version
14:54:36 <Vivienne> SA: if you put them together you will get the results
14:54:46 <Vivienne> AG: I provided some wording
14:55:06 <Vivienne> SA: we might actually pull that together within 4A.  We'll look at the wording suggestion and propose something for people to review.
14:55:28 <martijnhoutepen> q?
14:55:44 <agarrison> q+
14:55:58 <Vivienne> Review May 14 Editor's draft for next week
14:56:12 <Sarah_Swierenga> good discussion. have a good holiday weekend (US) and/or a good holiday next week.
14:56:28 <Zakim> -Sarah_Swierenga
14:56:55 <Vivienne> MH:  everyone please read the emails and if you can carry on the discussion.
14:57:10 <Vivienne> AG: a lot of the misunderstandings that get into the list are from people who don't read the emails correctly
14:58:15 <Vivienne> Mh:  please be patient with each other, remember not everyone has English as their first language.  If it isn't clear, then ask on the call.
14:58:38 <Vivienne> MH:  see you all in 2 weeks.  Please read the discussions on the list and the editor draft.
14:58:39 <Vivienne> ack me
14:58:45 <Mike_Elledge> bye!
14:58:49 <agarrison> q-
14:58:54 <Zakim> -Kathy_Wahlbin
14:58:55 <Zakim> -alistair
14:58:58 <Zakim> -Mike_Elledge
14:59:01 <Zakim> -Shadi
14:59:02 <Zakim> -Vivienne
14:59:51 <martijnhoutepen> martijnhoutepen has left #eval
15:05:01 <Zakim> disconnecting the lone participant, MartijnHoutepen, in WAI_ERTWG(Eval TF)10:00AM
15:05:03 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
15:05:03 <Zakim> Attendees were Mike_Elledge, Vivienne, MartijnHoutepen, Shadi, alistair, Sarah_Swierenga, Kathy_Wahlbin
15:05:13 <shadi> trackbot, end meeting
15:05:13 <trackbot> Zakim, list attendees
15:05:13 <Zakim> sorry, trackbot, I don't know what conference this is
15:05:21 <trackbot> RRSAgent, please draft minutes
15:05:21 <RRSAgent> I have made the request to generate http://www.w3.org/2014/05/22-eval-minutes.html trackbot
15:05:22 <trackbot> RRSAgent, bye
15:05:22 <RRSAgent> I see no action items