13:57:53 <RRSAgent> RRSAgent has joined #eval
13:57:53 <RRSAgent> logging to http://www.w3.org/2013/09/12-eval-irc
13:57:55 <trackbot> RRSAgent, make logs world
13:57:55 <Zakim> Zakim has joined #eval
13:57:57 <trackbot> Zakim, this will be 3825
13:57:57 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 3 minutes
13:57:58 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
13:57:58 <trackbot> Date: 12 September 2013
13:58:02 <Liz> Liz has joined #eval
13:58:23 <ericvelleman> ericvelleman has joined #eval
13:58:37 <Liz> Liz has joined #eval
14:00:51 <ericvelleman> test
14:00:56 <Kathy> hi
14:01:04 <MartijnHoutepen> Zakim, who is here?
14:01:04 <Zakim> WAI_ERTWG(Eval TF)10:00AM has not yet started, MartijnHoutepen
14:01:05 <Zakim> On IRC I see Liz, ericvelleman, Zakim, RRSAgent, Vivienne, Mike_Elledge, Kathy, Detlev, MartijnHoutepen, Bim, shadi, trackbot
14:01:39 <MartijnHoutepen> Zakim, this is 3825
14:01:39 <Zakim> ok, MartijnHoutepen; that matches WAI_ERTWG(Eval TF)10:00AM
14:01:45 <Detlev> Zakim, mute me
14:01:45 <Zakim> Detlev should now be muted
14:02:18 <Vivienne> zakim, mute me
14:02:18 <Zakim> sorry, Vivienne, I do not know which phone connection belongs to you
14:02:24 <Kathy> zakim, mute me
14:02:25 <Zakim> sorry, Kathy, I do not know which phone connection belongs to you
14:02:26 <MartijnHoutepen> Zakim, who is on the phone?
14:02:26 <Zakim> On the phone I see Mike, +1.978.443.aaaa, Liz, ??P11, [IPcaller], MartijnHoutepen, Detlev (muted)
14:02:36 <Vivienne> zakim, IPcaller is me
14:02:36 <Zakim> +Vivienne; got it
14:02:37 <Kathy> zakim, aaaa is me
14:02:38 <Zakim> +Kathy; got it
14:02:46 <Vivienne> zakim, mute me
14:02:46 <Zakim> Vivienne should now be muted
14:03:00 <ericvelleman> hi
14:03:14 <Zakim> +Shadi
14:03:25 <MartijnHoutepen> Zakim, who is on the phone?
14:03:26 <Zakim> On the phone I see Mike, Kathy, Liz, ??P11, Vivienne (muted), MartijnHoutepen, Detlev (muted), Shadi
14:03:26 <Liz> zakim; Liz is here
14:03:51 <ericvelleman> Zakim, ??BP is me
14:03:51 <Zakim> sorry, ericvelleman, I do not recognize a party named '??BP'
14:04:00 <MartijnHoutepen> Zakim, ??P11 is eric
14:04:00 <Zakim> +eric; got it
14:04:14 <ericvelleman> about to start
14:05:06 <Kathy> scribe: Kathy
14:05:27 <Kathy> Topic: Continue discussion about new section ..Context of WCAG-EM
14:05:29 <MartijnHoutepen> Zakim, mute me
14:05:29 <Zakim> MartijnHoutepen should now be muted
14:05:38 <ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#conformanceintro
14:05:51 <ericvelleman> list discussion: http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Sep/0004.html
14:06:51 <Kathy> Eric:  had discussion in the last meeting and the list.  Should we change the title of the section "Context of WCAG-EM" or should we move this section?
14:07:08 <Kathy> Eric:  We should see how far in the discussion during the telco
14:07:17 <MartijnHoutepen> q+
14:07:19 <Mike_Elledge> q+
14:07:26 <MartijnHoutepen> ack me
14:08:01 <Kathy> Martijn:  Agree with Shadi in putting it in the scope section, and Mike section on the edits for Section 5
14:08:12 <MartijnHoutepen> Zakim, mute me
14:08:12 <Zakim> MartijnHoutepen should now be muted
14:09:09 <shadi> http://www.w3.org/WAI/ER/methodology/#scope
14:09:16 <ericvelleman> q?
14:09:28 <Kathy> Mike: seems to me that part of the question is what is conformance.  We should define what has conform.  Listing what pages are included in the conformance.
14:09:33 <MartijnHoutepen> q- Mike
14:09:43 <shadi> q+
14:09:50 <shadi> ack me
14:11:02 <Detlev> Shadi, weren't you suggesting to leave out the term  'conformance' altogether?
14:11:12 <Vivienne> q+
14:11:27 <ericvelleman> q?
14:12:22 <Kathy> Shadi:  I like the word scope.  Agree with Peter that we need to clear about conformance claim.  My preference is to avoid the word conformance.  It is a reserved term.  We should try to avoid overlaps within the document
14:12:25 <MoeKraft> MoeKraft has joined #eval
14:12:25 <Vivienne> zakim, ack me
14:12:25 <Zakim> unmuting Vivienne
14:12:25 <Zakim> I see no one on the speaker queue
14:13:00 <Detlev> q+
14:13:44 <Kathy> Vivienne:  Confused about the whole thing since it has gone back and forth.   If we evaluate every page and it meets WCAG 2.0 then they could say that it conforms to WCAG 2.0 for that particular date and time.  We need to make it clear that conformance is for every page.
14:14:05 <Detlev> q-
14:14:15 <Kathy> Vivienne:  For a website that we did not review all pages then we could list the pages for conformance
14:14:36 <Vivienne> zakim, mute me
14:14:36 <Zakim> Vivienne should now be muted
14:14:44 <Kathy> Eric: Who is ok with including it in the scope?
14:14:49 <Liz> Put is scope.
14:14:51 <MartijnHoutepen> +1
14:15:02 <Vivienne> I need the two points explained more
14:15:16 <Kathy> Shadi:  We can't just put it in the scope to remove the overlap.
14:15:39 <Detlev> Might be better to take up again when Peter is present?
14:15:48 <Zakim> + +1.978.899.aabb
14:16:15 <Mike_Elledge> q+
14:16:19 <Kathy> Eric:  It is in multiple sections.  We can see if we want to take the word conformance out or move it to 5b
14:16:24 <MoeKraft> aabb is MoeKraft
14:16:54 <Kathy> Mike:  it could be in a separate section then we should change the name to be specfic to WCAG 2.0 conformance claim
14:16:56 <Detlev> q+
14:17:00 <Kathy> q+
14:18:23 <Kathy> Shadi:  We should try to avoid the word conformance but we can use conformance.  We cannot redefine WCAG 2.0 conformance within this document
14:18:28 <shadi> q- mike
14:18:48 <Kathy> Miike:  Agree with Shadi
14:18:50 <Detlev> ack me
14:20:08 <Kathy> Detlev:  Accessibility statement is good.  Conformance claims can only be made for single pages.
14:20:48 <Kathy> Eric:  We imply in the document that if you look at a sample, then you have looked at the entire website.  We don't have conformance claim unless we look at all pages
14:21:15 <Kathy> Detlev:  Good to separate the evaluation statement from conformance claim
14:21:56 <Detlev> Zakim, mute me
14:21:56 <Zakim> Detlev should now be muted
14:22:03 <Kathy> Eric:  We could claim conformance for the sample and explain what we mean for the rest of the site
14:22:32 <shadi> Kathy: throughout the document we will have people wondering about how this relates to conformance claims
14:22:33 <Tim> Tim has joined #eval
14:22:47 <shadi> ...so agree with Peter that need to tackle head-on
14:22:48 <shadi> q+
14:22:54 <Kathy> q-
14:22:54 <shadi> ack kath
14:23:32 <MartijnHoutepen> good title
14:23:39 <Kathy> Shadi:  How about the title "how this output relates to WCAG 2.0 conformance claims"?  That is clear and will explain how they relate
14:23:40 <shadi> ack me
14:24:17 <Kathy> Eric:  we will take a look at this
14:24:29 <shadi> "How WCAG-EM Accessibility Statements relate to WCAG 2.0 Conformance Claims"
14:24:42 <Kathy> Topic:   Discussion of listitems in Step 3: Select a Representative Sample
14:24:45 <ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#step3
14:25:08 <Kathy> Eric:  There is a list that has been added.
14:25:46 <Kathy> Eric:   Added factors that are related to the likelyhood of a sample being representative for a website. We should include in this section also text about the notion of "reasonable confidence". This notion is also used in section 5.b called likelyhood of a conformance claim being valid for a complete website (within the scope of the evaluation as described in section 1.a).
14:26:24 <Kathy> Eric: What do you think of the list?
14:26:34 <Detlev> q+
14:26:48 <Vivienne> q+
14:26:49 <Kathy> q+
14:26:54 <Detlev> ack me
14:27:58 <Kathy> Detlev:  List is true but from a practical level it may not be clear as to what to do.  Because there is so many different factors it will be hard to give more information
14:28:23 <Kathy> Eric:  If you start describing it, they you have a long explanation about all of them
14:28:31 <Vivienne> zakim, ack me
14:28:31 <Zakim> unmuting Vivienne
14:28:32 <Zakim> I see Kathy on the speaker queue
14:29:03 <Detlev> Zakim, mute me
14:29:03 <Zakim> Detlev should now be muted
14:29:08 <Kathy> Vivienne:  Ideas are good and things that people can think about.  If I was novice person but I would not know how big the sample needs to be
14:29:48 <Detlev> q+
14:30:08 <Kathy> Vivienne:  We need to go a bit further then here are some examples.  An illustration or typical scenarios based on common situations
14:30:28 <Vivienne> zakim, mute me
14:30:28 <Zakim> Vivienne should now be muted
14:30:55 <shadi> Kathy: when recommending a sample size to the clients, we work with a questionnaire
14:31:15 <shadi> ...maybe different scenarios of how these factors come into play
14:32:03 <ericvelleman> q?
14:32:07 <shadi> ...for example to spell out parameters like website categories
14:32:16 <shadi> ...because not all factors will always come into play
14:32:19 <Detlev> ack me
14:33:05 <Kathy> Detlev - one thing is a bit odd; "Experience level of the developers".  If you increase the sample size you may not get better results
14:33:09 <Kathy> q+
14:34:21 <Kathy> Detlev - importance of the page should be included in the list.  Practical advice on how to pick pages to reduce the number of pages needed for the review.
14:34:41 <Kathy> Eric:  This is in some places but not here.
14:35:31 <Detlev> Kathy - I see - I misread that!
14:35:37 <shadi> Kathy: point refered to level of experience with accessibility of a developer
14:35:51 <shadi> ...so might catch something in one page but not another
14:36:01 <Detlev> I thought it was talking about experience level of evaluators!
14:36:05 <Vivienne> I think it should stay there
14:36:18 <Detlev> ack me
14:36:41 <MoeKraft> q+
14:36:52 <Detlev> zakim, mute me
14:36:52 <Zakim> Detlev should now be muted
14:36:53 <Kathy> Eric:  Should we remove the list of factors?
14:36:54 <Vivienne> I want it to stay Eric
14:36:56 <Kathy> q-
14:36:58 <MartijnHoutepen> q- kath
14:37:14 <Kathy> Eric:  We need a more pratical approach.  People will need to know what this means
14:37:38 <Vivienne> I like the idea of some examples
14:37:39 <Kathy> Eric:  How could we get to a more practical approach?
14:37:41 <Detlev> Someone should draft something whichwe can then discuss...
14:37:43 <Kathy> q+
14:38:42 <Kathy> Moe:  Raise one concern; type of technology and templates used.  The evaluator may not know this information.  Concrete example will be helpful
14:38:56 <MoeKraft> q-
14:39:53 <shadi> Kathy: if evaluators have less experience in selecting pages then may need more pages
14:40:16 <shadi> ...might need to work with other peers to get the right pages
14:41:16 <Vivienne> q+
14:41:20 <Kathy> q-
14:41:25 <Vivienne> zakim, ack me
14:41:25 <Zakim> unmuting Vivienne
14:41:26 <Zakim> I see no one on the speaker queue
14:41:32 <shadi> ...multiple people's experience often helps in the start
14:42:12 <shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#expertise
14:42:19 <Kathy> Vivienne:  in the beginning we have a section about who should use this but that there is a base level of knowledge required.
14:42:20 <shadi> "Required Expertise"
14:42:44 <shadi> also "Review Teams (Optional)" http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#teams
14:42:52 <Kathy> Vivienne:  we have stated that expertise is required to use the methodology so perhaps we can refer to that
14:42:54 <Vivienne> zakim, mute me
14:42:54 <Zakim> Vivienne should now be muted
14:43:53 <Vivienne> q+
14:43:55 <Kathy> Shadi:  we have a later section on review teams.  Maybe we can have this clarified earlier.  We say encourage review teams but we could add something about being new or less experience then review teams are useful
14:45:01 <Zakim> +Tim_Boland
14:45:11 <Kathy> Eric:  we need to work on this but we will need help.  Examples will be clear.  But if we say it will require a larger sample
14:45:12 <Vivienne> zakim, ack me
14:45:12 <Zakim> unmuting Vivienne
14:45:12 <Kathy> q+
14:45:13 <Zakim> I see no one on the speaker queue
14:46:42 <Kathy> Vivienne:  The evaluator we state is responsible for the page selection.  This removes the bias.  Our concerns may be misplaced
14:47:18 <Kathy> Vivienne:  to answer Eric's question, we could put together 2 examples for a typcial website
14:47:26 <Vivienne> zakim, mute me
14:47:26 <Zakim> Vivienne should now be muted
14:47:29 <ericvelleman> q?
14:48:15 <shadi> Kathy: yes, evaluator is ultimately responsible but can get the commissioner/owner involved in the selection process
14:48:36 <shadi> ...to figure out the site and what to consider
14:48:50 <shadi> ...think could provide examples
14:48:58 <shadi> ...but also wondering about a chart
14:49:13 <shadi> ...not sure what that would look like, just as an idea
14:49:31 <shadi> ...might help people better understand the complexities
14:49:33 <Vivienne> I really like the idea of a chart
14:50:00 <Kathy> Eric:  I like that.  We could try that.
14:50:26 <Vivienne> I was going to say that I could help
14:50:28 <shadi> regrets: Sarah
14:50:38 <Kathy> Eric:  Who can help write the examples?
14:50:54 <Kathy> Eric: If you have time, please work on examples
14:51:06 <Kathy> Topic:  Prescriptiveness
14:51:38 <Kathy> Eric:  We should do the discussion on the list.  Should we be more prescriptive in the document?
14:51:54 <Kathy> Eric:  and where should we?
14:52:09 <Kathy> Topic:  Discussion start about Step 3.e: Include a Randomly Selected Sample
14:52:22 <Kathy> Eric:  How much more should we add there?
14:52:53 <Vivienne> q+
14:52:58 <ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#step3e
14:53:09 <Mike_Elledge> q=
14:53:10 <Vivienne> zakim, ack me
14:53:12 <Zakim> unmuting Vivienne
14:53:12 <Zakim> I see Kathy on the speaker queue
14:53:14 <Kathy> Eric:  We have received many articles about this.  Do we want to say how to select a random sample?  Is this necessary?  I think we should leave it out
14:53:29 <Mike_Elledge> q+
14:53:33 <Kathy> Vivienne:  We have a random sampler application that students are working on
14:53:44 <shadi> q- kath
14:54:08 <Zakim> -Mike
14:54:10 <Kathy> Eric:  all of the papers are describing a certain method and many different ways depending on the site
14:54:23 <Kathy> q+
14:54:54 <Zakim> +Mike
14:55:13 <shadi> Kathy: if we stay away from it, then stay away all together
14:55:13 <shadi> ...or leave it open for people to decide
14:55:34 <shadi> ...if we add example it should not be exclusive
14:55:46 <shadi> q- kath
14:56:07 <shadi> ack mike
14:56:34 <Kathy> Mike:  agreed with Kathy.  What do you mean by leave out of the document? In the appendix, we could include this information as it will be useful.  We should recommend an approach
14:56:44 <Kathy> q-
14:57:12 <Kathy> Eric:  How could we get this done?  It will be hard to say go to this website and refer to it
14:57:14 <Kathy> q+
14:57:51 <Kathy> Shadi:  possible but the content needs to be develop.
14:58:12 <Kathy> Eric:  investigating not including it in this document and linking to another document
14:59:09 <Kathy> Shadi: the other document would need to exists.  If it causes a depency then we will need it
14:59:30 <shadi> ack kath
14:59:53 <shadi> Kathy: is there something that we can summarize in a chart or something?
15:00:37 <shadi> ...might support an overview
15:00:47 <Kathy> Eric:  we will open a list item
15:01:20 <Kathy> Eric:  Participate in the discussion on the list.
15:01:31 <MartijnHoutepen> ack me
15:01:32 <Mike_Elledge> Bye!
15:01:32 <Detlev> bye
15:01:34 <Vivienne> okay, good night all
15:01:35 <Liz> bye
15:01:38 <Zakim> -Detlev
15:01:39 <Zakim> -Mike
15:01:41 <Zakim> - +1.978.899.aabb
15:01:44 <Zakim> -Kathy
15:01:46 <Zakim> -Vivienne
15:01:46 <Zakim> -Shadi
15:01:48 <Zakim> -eric
15:01:50 <Zakim> -MartijnHoutepen
15:01:51 <Zakim> -Liz
15:01:54 <MartijnHoutepen> MartijnHoutepen has left #eval
15:03:12 <Zakim> -Tim_Boland
15:03:13 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
15:03:13 <Zakim> Attendees were Mike, +1.978.443.aaaa, Liz, MartijnHoutepen, Detlev, Vivienne, Kathy, Shadi, eric, +1.978.899.aabb, Tim_Boland
15:46:28 <shadi> trackbot, end meeting
15:46:28 <trackbot> Zakim, list attendees
15:46:28 <Zakim> sorry, trackbot, I don't know what conference this is
15:46:36 <trackbot> RRSAgent, please draft minutes
15:46:36 <RRSAgent> I have made the request to generate http://www.w3.org/2013/09/12-eval-minutes.html trackbot
15:46:37 <trackbot> RRSAgent, bye
15:46:37 <RRSAgent> I see no action items