14:41:50 <RRSAgent> RRSAgent has joined #eval
14:41:50 <RRSAgent> logging to http://www.w3.org/2012/02/09-eval-irc
14:41:52 <trackbot> RRSAgent, make logs world
14:41:52 <Zakim> Zakim has joined #eval
14:41:54 <trackbot> Zakim, this will be 3825
14:41:54 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 19 minutes
14:41:55 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:41:55 <trackbot> Date: 09 February 2012
14:51:37 <vivienne> sure, it will keep me awake
14:52:24 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
14:52:31 <Zakim> +[IPcaller]
14:52:43 <vivienne> zakim, IPCaller is me
14:52:43 <Zakim> +vivienne; got it
14:54:00 <vivienne> zakim, mute me
14:54:00 <Zakim> sorry, vivienne, muting is not permitted when only one person is present
14:54:22 <ericvelleman> ericvelleman has joined #eval
14:54:30 <Zakim> +Katie_Haritos-Shea
14:54:42 <Detlev> Detlev has joined #eval
14:54:55 <Zakim> +Shadi
14:55:00 <MartijnHoutepen> MartijnHoutepen has joined #eval
14:55:06 <vivienne> ack me
14:56:34 <Kathy> Kathy has joined #eval
14:56:57 <Liz> Liz has joined #eval
14:57:01 <Zakim> +MartijnHoutepen
14:57:16 <shadi> scribe: vivienne
14:57:26 <shadi> chair: Eric
14:57:37 <Ryladog> Ryladog has joined #eval
14:57:40 <Zakim> +[IPcaller]
14:57:41 <Zakim> + +1.978.760.aaaa
14:57:51 <Zakim> +Liz
14:58:06 <Kathy> zakim, aaaa is Kathy
14:58:06 <Zakim> +Kathy; got it
14:58:35 <kerstin> kerstin has joined #eval
14:58:51 <kerstin> zakim, mute me
14:58:51 <Zakim> sorry, kerstin, I do not know which phone connection belongs to you
14:59:16 <shadi> zakim, who is on the phone?
14:59:16 <Zakim> On the phone I see vivienne, Katie_Haritos-Shea, Shadi, MartijnHoutepen, [IPcaller], Kathy, Liz
14:59:30 <Kathy> zakim, mute me
14:59:30 <Zakim> Kathy should now be muted
14:59:41 <Zakim> +Eric
14:59:53 <Zakim> +Don
14:59:59 <kerstin> zakim, ipcaller is me
14:59:59 <Zakim> +kerstin; got it
15:00:06 <kerstin> zakim, mute me
15:00:06 <Zakim> kerstin should now be muted
15:00:27 <shadi> zakim, who is on the phone?
15:00:27 <Zakim> On the phone I see vivienne, Katie_Haritos-Shea, Shadi, MartijnHoutepen, kerstin (muted), Kathy (muted), Liz, Eric, Don
15:00:29 <Ryladog> Yes
15:00:32 <MartijnHoutepen> zakim, mute me
15:00:32 <Zakim> MartijnHoutepen should now be muted
15:00:53 <Zakim> + +49.404.318.aabb
15:01:03 <Zakim> +Mike
15:01:23 <vivienne> EV: welcome
15:01:38 <Detlev> Zakim, aabb is Detlev
15:01:38 <Zakim> +Detlev; got it
15:02:24 <Mike_Elledge> Mike_Elledge has joined #eval
15:02:36 <shadi> http://www.w3.org/WAI/ACT/
15:02:52 <vivienne> Shadi: WAI-ACT Open Meeting: project started Sept.1 open meeting will be in Brussels on Tuesday.  Shadi will provide home page about the project.
15:03:15 <kerstin> who else from this group will be in brussels?
15:03:16 <shadi> Topic: Welcome
15:03:18 <shadi> agenda+ Short introduction to changes in last version of Methodology
15:03:18 <shadi> agenda+ Definition of Website
15:03:18 <shadi> agenda+ Discussion on Sampling of pages, Clause 4
15:03:25 <vivienne> EV: methodology will be presented at the WAI-ACT meeting
15:03:29 <richard> richard has joined #eval
15:04:03 <Zakim> +[IPcaller]
15:04:50 <Detlev> Zakim, mute me
15:04:50 <Zakim> Detlev should now be muted
15:04:53 <vivienne> Shadi: CSUN, presentations shown on the WAI home page.
15:05:02 <Kathy> Yes, I am attending and presenting
15:05:26 <Ryladog> Great. Thank you!
15:06:00 <shadi> zakim, take up next
15:06:00 <Zakim> agendum 1. "Short introduction to changes in last version of Methodology" taken up [from shadi]
15:06:05 <shadi> zakim, mute me
15:06:05 <Zakim> Shadi should now be muted
15:06:16 <shadi> http://www.w3.org/WAI/ER/methodology/
15:06:56 <shadi> ack me
15:07:20 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Feb/0036.html
15:07:28 <shadi> zakim, mute me
15:07:28 <Zakim> Shadi should now be muted
15:07:41 <vivienne> EV: described the update to the methodology in a table
15:09:53 <shadi> zakim, who is on the phone?
15:09:53 <Zakim> On the phone I see vivienne, Katie_Haritos-Shea, Shadi (muted), MartijnHoutepen (muted), kerstin (muted), Kathy (muted), Liz, Eric, Don, Detlev (muted), Mike, [IPcaller]
15:10:14 <vivienne> EV: email relating to the new version methodology and how he has put the changes in that we discussed 
15:11:38 <vivienne> EV: main change was in Clause #6 and #9
15:12:16 <Mike_Elledge> q+
15:12:22 <vivienne> EV: item #9 is not set, but we should discuss
15:12:32 <vivienne> q+
15:12:50 <Ryladog> q+
15:13:05 <Zakim> -Don
15:13:39 <Zakim> +Don
15:14:06 <vivienne> ME: at the university their library has electronic documents that are provided by 3rd party vendors which are not accessible.  So they are discussing how they can make a claim about the system for providing the document is accessible, but the documents themselves are not.  Also, LMS eg Blackboard where there is 3rd party material, but the system is accessible.  Thoughts?
15:14:26 <Detlev> q+
15:14:33 <kerstin> q+
15:14:39 <richard> richard is on the phone
15:14:57 <ericvelleman> q?
15:16:28 <vivienne> VC: not comfortable with being too flexible, but should encourage 3rd party vendors accountable for their content
15:17:04 <vivienne> EV: Netherlands have encountered this also.  System accessible, but content is not.  Looks like it is more accessible than it really is.
15:18:18 <vivienne> Katie: strongly says you shouldn't be able to scope out content - makes it non-compliant with WCAG.  Organisations make a choice over which content they use and we want to push them to get accessible content.  Should be choosing 3rd party content that is accessible.  If they want to say they use this methodology that is one thing, but they can't make a claim.
15:18:23 <Detlev> ack me
15:18:24 <Mike_Elledge> q+
15:18:31 <ericvelleman> q?
15:18:52 <shadi> q- mike
15:18:56 <shadi> q- viv
15:18:59 <shadi> q- ryl
15:19:55 <vivienne> Detlev: doesn't think anyone agreed to taking out parts of complete processes.  Regarding taking out other things from the scope, but it shouldn't be picking and choosing which pages conform.  Should be self-proclaimed parts of sites, not just removing pages. It should be whole parts of websites, not bits and pieces.
15:20:22 <shadi> q+
15:20:42 <kerstin> zakim, unmute me
15:20:42 <Zakim> kerstin should no longer be muted
15:20:42 <vivienne> EV: you shouldn't be able to scope out the contents.  e.g. a CMS wants an evaluation and a conformance claim where there would be no content, only if it is not available.
15:20:51 <shadi> ack k
15:20:56 <Detlev> Zakim, mute me
15:20:56 <Zakim> Detlev should now be muted
15:21:07 <shadi> zakim, unmute me
15:21:07 <Zakim> Shadi should no longer be muted
15:21:30 <Zakim> -kerstin
15:21:32 <vivienne> Kerstin is not coming through
15:21:48 <shadi> ack me
15:22:31 <Zakim> +??P9
15:22:52 <kerstin> zakim, P9 is me
15:22:52 <Zakim> sorry, kerstin, I do not recognize a party named 'P9'
15:23:02 <vivienne> Shadi: don't underestimate the power of reporting.  Re the empty CMS the user can make a claim and doesn't need to exclude anything, because there is no content.  You may be able to use this methodology for the university example, but can say that the reason for the university website not being conformant is the university library website that is using non accessible third party providers.
15:23:23 <shadi> zakim, mute me
15:23:23 <Zakim> Shadi should now be muted
15:24:23 <Mike_Elledge> q+
15:24:27 <vivienne> EV: Examples for exclusion #9.  Statement of Partial Conformance e.g. Conformant with WCAG 2.0, but not for third party providers.
15:24:29 <kerstin> sorry, don't know what the problem is
15:24:34 <ericvelleman> q?
15:25:03 <kerstin> zakim, ??P9 is me
15:25:03 <Zakim> +kerstin; got it
15:25:42 <vivienne> Mike: If the university is making everything on line accessible, but it would be helpful to say that the system is compliant.  Allows the focus to be put on the part that is not compliant.
15:26:22 <shadi> ack me
15:26:28 <Mike_Elledge> +1
15:26:31 <shadi> q+
15:26:36 <shadi> q- mike
15:26:41 <shadi> ack me
15:28:22 <vivienne> Shadi: agreed with Eric's statement.  We discussed whether the target website conforms or not, and in addition to that a figure (not sure how to state e.g. 90%) to compare.  Not saying that something is 90% conformant.  We need to find a way to say which things are work and which do not.
15:28:24 <shadi> zakim, mute me
15:28:24 <Zakim> Shadi should now be muted
15:29:12 <vivienne> Discussion on Item #10 - deleted Error Margin
15:30:00 <Detlev> q+
15:30:07 <vivienne> EV: Clause 6 - the positive approach - describing items which meet more than the conformance claim.  However there are problems when the person who does the evlauation will have to list the items which met the higher level. This would add more cost and time to the evaluation.  Is that what people want?
15:30:19 <vivienne> q+
15:30:21 <Detlev> ack me
15:30:21 <ericvelleman> q?
15:30:37 <shadi> q+ det
15:30:46 <shadi> ack viv
15:30:57 <shadi> q- det
15:31:01 <shadi> q+ detlev
15:31:06 <Detlev> q+
15:31:12 <Detlev> ack me
15:31:31 <vivienne> Vivienne: developers won't want to pay for something they haven't contracted to be done.
15:32:38 <vivienne> Detlev: You shouldn't have to look at all of the standards for the level above.  If you are looking at an issue such as colour for A, if the contrast is almost at the AA standard, you can mention that.  You may be finding the website is better for some criteria than the standard you are looking for.
15:32:39 <Kathy> agree
15:32:43 <vivienne> agree
15:33:00 <MartijnHoutepen> agree
15:33:00 <Mike_Elledge> agree
15:33:04 <vivienne> I just don't want an assessor to have to look at all levels
15:33:12 <kerstin> I'm not sure, because unless you have tested, you can't say something
15:33:15 <Detlev> Zakim, mute me
15:33:15 <Zakim> Detlev should now be muted
15:33:21 <vivienne> Eric: will change document to reflect this
15:34:20 <vivienne> EV: definition of a website - do we need to change this in the scope?  
15:34:25 <vivienne> next agenda item
15:34:47 <vivienne> Thanks Shadi, didn't know how to do this
15:34:52 <shadi> ack me
15:35:14 <Detlev> I think Shadi's phrasing was quite good
15:35:27 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Feb/0017.html
15:35:32 <vivienne> Shadi: it is not actually changing the definition, but could be a note under the definition or a clarification in the scope.  We should re-use what we already have.
15:35:49 <vivienne> Shadi: it is an added thing to the definition.
15:36:00 <vivienne> EV: would rather put it in the scope as a clarification
15:37:19 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Feb/0019.html
15:37:22 <vivienne> EV: will put this on the discussion board
15:38:45 <vivienne> Shadi: this methodology is supposed to be for a set of pages, but anyone can claim conformance to WCAG for a single page if they want to under the existing conformance claim
15:38:59 <kerstin> Yes, I think there should be more clarification for the last sentence
15:39:06 <vivienne> EV: will put it in the document
15:39:12 <shadi> zakim, mute me
15:39:12 <Zakim> Shadi should now be muted
15:39:38 <vivienne> Agendum 5
15:39:38 <ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20120209.html
15:40:19 <vivienne> EV: Clause 4, Sampling of pages.  Eric proposed to take out the full sample section, but participants wanted it to stay in.
15:40:25 <Zakim> -Don
15:41:02 <Zakim> +Don
15:41:09 <vivienne> EV: the sample tells you what has been looked at, but not just the individual pages in the sample.  You would still sample the section of the website that you are looking at.  Not sure if I got that right Eric.
15:41:37 <vivienne> EV: you don't always need a 'contact us' page if you are looking at a specific part of a website.
15:42:22 <vivienne> EV: people can choose their own scope, but within this scope they need to use the 3 types of sampling if it is in there - e.g. if a help page is in the selection, it should be in the sample.
15:43:06 <vivienne> EV:  random resources also need to be selected.  Do we want to describe the exact methods people have to use to get a random set of resources or is this free for anyone to do it in his own way?
15:43:10 <shadi> q?
15:43:16 <Kathy> q+
15:43:20 <Kathy> ack me
15:44:09 <vivienne> Kathy: it would be good to have some suggestions.  Clients ask how you determine what the random sample is and there are number of different ways to create a random sample.  W3C can provide ways you can create a random sample and suggest these.
15:44:14 <Mike_Elledge> +1
15:44:20 <Kathy> zakim, mute me
15:44:20 <Zakim> Kathy should now be muted
15:44:21 <vivienne> +1
15:44:33 <richard> +1
15:44:35 <shadi> ack me
15:44:52 <Detlev> q+
15:45:21 <vivienne> Shadi: not aware of any proven method to do this and it could get difficult.
15:45:42 <vivienne> EV: we need to make a split between a manual selection and an automatic selection or a combination.
15:45:53 <ericvelleman> q?
15:46:03 <vivienne> Shadi: what is UWEM saying?
15:46:18 <shadi> [[Unified Web Evaluation Methodology]]
15:46:48 <vivienne> EV: UWEM made a special method to select a random sample - called Uniform Random Sampling Without Replacement
15:46:57 <vivienne> EV: under 4.1.3.
15:47:20 <vivienne> EV: technique developed by UWEM that is statistically verified and is fairly easy to describe
15:47:48 <vivienne> EV: will send a pointer on the mailing list
15:47:55 <Detlev> ack me
15:47:56 <ericvelleman> q?
15:49:19 <shadi> q+
15:49:48 <MartijnHoutepen> q+
15:50:37 <vivienne> Detlev: re: random sampling.  We agree that we need the core resource e.g. home page, contact, search results etc.  That will give you 5-10 pages for your sample.  If you add to that random sampling - may take frequency of visits into account.  YOu may have a lot of redundancy - pages that are the same as other pages in the sample.  Could be a waste of resources. Prefers a way to look for the 
15:50:37 <vivienne> differences in content type, and not specific on the way you do random sampling.  There can be things outside the sample that have not been covered.  The disclaimer shows that some content types could be missed.
15:51:06 <vivienne> EV: Netherlands have a minimum of 50 pages
15:51:16 <kerstin> what customers would pay depends on what they are used
15:51:39 <Detlev> Zakim, mute me
15:51:39 <Zakim> Detlev should now be muted
15:52:08 <shadi> regrets: Alistair, Denis, Emmanuelle, Samuel, Tim
15:52:22 <vivienne> EV: UWEM also has stop criteria.  4.3 shows this.  Instead of taking large numbers of pages because they are all the same and we can look at specific content types after that.
15:52:24 <shadi> ack me
15:54:40 <vivienne> Shadi: Agree with Detlev.  UWEM good for tool-based approaches  To have a good structure, good advice on how to select pages, thinking 50 pages.  Take the benefit of the human evaluator who can decide if a page belongs to a complete process.  Lean toward a much manual as possible.  You will have to have a tool that can scan the pages to select pages for you to evaluate manually.  Need a good 
15:54:40 <vivienne> structure to document how we are going to sample pages.
15:55:55 <vivienne> Shadi: depends on how you define core resource.  e.g. don't only take the first form you find.  Don' t just look for things that are linked from the home page.
15:56:14 <shadi> zakim, mute me
15:56:14 <Zakim> Shadi should now be muted
15:56:16 <vivienne> Shadi: we need to provide more details of how to select the pages
15:56:18 <MartijnHoutepen> ack me
15:56:22 <ericvelleman> q?
15:56:56 <shadi> q+ to propose approach for random sample
15:56:57 <Detlev> q+
15:57:06 <vivienne> Martijn: agrees the sample does not have to be very big, but need random sampling.  Random provides good verification.
15:57:10 <ericvelleman> ?
15:57:12 <ericvelleman> q?
15:57:15 <shadi> ack me
15:57:17 <Zakim> shadi, you wanted to propose approach for random sample
15:57:21 <MartijnHoutepen> zakim, mute me
15:57:21 <Zakim> MartijnHoutepen should now be muted
15:57:55 <vivienne> Shadi: go to search engine and type something randomly and put the delimiter site=" and you will come up with a page, from which you can follow say the 5th link.
15:58:11 <vivienne> Shadi: there may be easy ways for an evaluator to find the random page easily.
15:58:19 <shadi> zakim, mute me
15:58:19 <Zakim> Shadi should now be muted
15:58:32 <shadi> ack me
15:58:38 <Zakim> -Don
15:58:49 <vivienne> EV:  doesn't need to be scientific, it would be interesting to give a few examples (simply) on how you could do it.  Keep away from an academic discussion.
15:59:07 <vivienne> Shadi: can point to things such as UWEM
15:59:18 <shadi> zakim, mute me
15:59:20 <Zakim> Shadi should now be muted
15:59:32 <shadi> ack me
15:59:54 <Detlev> ack me
15:59:54 <shadi> zakim, mute me
16:00:08 <Zakim> Shadi should now be muted
16:00:20 <vivienne> Detlev: need to pick pages that are unpredictable so no one can know what pages you will pick
16:00:28 <vivienne> EV: end of call
16:00:48 <kerstin> BYE
16:00:49 <Detlev> bye
16:00:49 <MartijnHoutepen> zakim, unmute me
16:00:50 <Kathy> thanks, bye
16:00:54 <MartijnHoutepen> MartijnHoutepen has left #eval
16:00:57 <vivienne> -vivienne
16:01:00 <vivienne> vivienne has left #eval
16:01:15 <Zakim> MartijnHoutepen should no longer be muted
16:01:24 <Zakim> -[IPcaller]
16:01:28 <Zakim> -Shadi
16:01:31 <Zakim> -Mike
16:01:34 <Zakim> -Kathy
16:01:36 <Zakim> -Detlev
16:01:42 <Zakim> -kerstin
16:01:46 <Zakim> -Katie_Haritos-Shea
16:01:55 <Zakim> -vivienne
16:01:59 <Zakim> -Eric
16:02:00 <Zakim> -Liz
16:02:04 <Zakim> -MartijnHoutepen
16:02:06 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
16:02:08 <Zakim> Attendees were vivienne, Katie_Haritos-Shea, Shadi, MartijnHoutepen, +1.978.760.aaaa, Liz, Kathy, Eric, Don, kerstin, +49.404.318.aabb, Mike, Detlev, [IPcaller]
16:03:03 <shadi> trackbot, end meeting
16:03:03 <trackbot> Zakim, list attendees
16:03:09 <Mike_Elledge> bye!
16:03:11 <trackbot> RRSAgent, please draft minutes
16:03:11 <RRSAgent> I have made the request to generate http://www.w3.org/2012/02/09-eval-minutes.html trackbot
16:03:12 <trackbot> RRSAgent, bye
16:03:12 <RRSAgent> I see no action items