IRC log of eval on 2013-07-18

Timestamps are in UTC.

13:52:51 [RRSAgent]
RRSAgent has joined #eval
13:52:51 [RRSAgent]
logging to http://www.w3.org/2013/07/18-eval-irc
13:52:53 [trackbot]
RRSAgent, make logs world
13:52:53 [Zakim]
Zakim has joined #eval
13:52:55 [trackbot]
Zakim, this will be 3825
13:52:55 [Zakim]
ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 8 minutes
13:52:56 [trackbot]
Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
13:52:56 [trackbot]
Date: 18 July 2013
13:57:55 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has now started
13:58:02 [Zakim]
+Shadi
13:58:38 [ericvelleman]
ericvelleman has joined #eval
13:59:09 [shadi]
agenda+ New Editor Draft, DoC and survey
13:59:27 [shadi]
agenda+ Rewrite of section 1.b
13:59:28 [shadi]
agenda+ Parameters of the sample size
13:59:39 [shadi]
agenda+ Planning of next meetings
13:59:46 [shadi]
chair: Eric
13:59:47 [Zakim]
+??P8
14:00:04 [shadi]
Regrets: Detlev, Kathy, Tim
14:00:08 [MartijnHoutepen]
Zakim, ??P8 is me
14:00:08 [Zakim]
+MartijnHoutepen; got it
14:00:40 [Liz]
Liz has joined #eval
14:00:55 [Zakim]
+ +31.30.239.aaaa
14:01:06 [ericvelleman]
zakim, aaaa is me
14:01:06 [Zakim]
+ericvelleman; got it
14:01:12 [Zakim]
+ +1.301.975.aabb
14:01:37 [Liz]
zakim, aabb is Liz
14:01:37 [Zakim]
+Liz; got it
14:02:12 [Vivienne]
Vivienne has joined #eval
14:03:10 [korn]
korn has joined #eval
14:03:20 [Zakim]
+ +1.517.432.aacc
14:03:21 [Zakim]
+[IPcaller]
14:03:26 [Vivienne]
Zakim, IPcaller is me
14:03:26 [Zakim]
+Vivienne; got it
14:03:47 [shadi]
scribe: shadi
14:03:58 [Sarah_Swierenga]
Sarah_Swierenga has joined #eval
14:04:08 [MartijnHoutepen]
zakim, please mute me
14:04:08 [Zakim]
MartijnHoutepen should now be muted
14:04:29 [shadi]
http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Jul/0037.html
14:04:41 [shadi]
Survey: https://www.w3.org/2002/09/wbs/48225/evaltfq10/
14:04:41 [shadi]
Editor Draft: http://www.w3.org/WAI/ER/methodology/
14:04:41 [shadi]
DoC: http://www.w3.org/WAI/ER/conformance/comments-20130226
14:05:02 [Zakim]
+ +1.703.861.aadd
14:05:27 [shadi]
EV: circulated documents, including new Editor Draft and survey
14:05:36 [shadi]
zakim, aacc is Sarah_Swierenga
14:05:36 [Zakim]
+Sarah_Swierenga; got it
14:05:43 [shadi]
zakim, aadd is korn
14:05:43 [Zakim]
+korn; got it
14:05:51 [Ryladog]
Ryladog has joined #eval
14:06:09 [shadi]
...comments addressed in batches
14:06:09 [Zakim]
+Peter_Korn
14:06:30 [shadi]
...please complete survey by next Tuesday
14:06:32 [korn]
Zakim, aadd is NOT korn...
14:06:32 [Zakim]
I don't understand 'aadd is NOT korn', korn
14:06:45 [shadi]
zakim, korn is Ryladog
14:06:45 [Zakim]
+Ryladog; got it
14:06:53 [shadi]
zakim, where is 703?
14:06:53 [Zakim]
North American dialing code 1.703 is Virginia
14:06:56 [MoeKraft]
MoeKraft has joined #eval
14:07:13 [Zakim]
+MoeKraft
14:07:42 [ericvelleman]
q?
14:07:47 [shadi]
...still have a number of large issues that need further discussion
14:08:08 [Vivienne]
I haven't had time to look at the edited draft or other documents yet
14:08:23 [Sarah_Swierenga]
Sorry, I haven't either.... too many fires for me, too.
14:08:39 [Vivienne]
I'll set the time aside and make sure that I do it
14:10:13 [Sarah_Swierenga]
i'll give it a try for next Tuesday, too
14:11:31 [shadi]
EV: survey links to the corresponding sections in the Editor Draft
14:11:56 [shadi]
zakim, take up next
14:11:56 [Zakim]
agendum 1. "New Editor Draft, DoC and survey" taken up [from shadi]
14:12:07 [shadi]
zakim, close agendum 1
14:12:07 [Zakim]
agendum 1, New Editor Draft, DoC and survey, closed
14:12:08 [Zakim]
I see 3 items remaining on the agenda; the next one is
14:12:08 [Zakim]
2. Rewrite of section 1.b [from shadi]
14:12:09 [shadi]
zakim, take up next
14:12:10 [Zakim]
agendum 2. "Rewrite of section 1.b" taken up [from shadi]
14:12:30 [ericvelleman]
http://www.w3.org/WAI/ER/conformance/ED-methodology-20130712#step1b
14:12:33 [shadi]
http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Jul/0035.html
14:14:07 [shadi]
EV: tried to explain in the email
14:14:27 [shadi]
...propose to change the title of the section
14:14:32 [shadi]
...the goal of the evaluation is set by the document
14:14:46 [ericvelleman]
q?
14:14:52 [shadi]
...not really the goal being determined here
14:15:04 [shadi]
PK: maybe "purpose" or "focus" instead?
14:15:32 [ericvelleman]
http://www.w3.org/WAI/ER/conformance/ED-methodology-20130712#audience
14:15:34 [shadi]
...assumption is that there are multiple usecases
14:15:47 [shadi]
...don't want to discourage use in other settings
14:16:03 [shadi]
...for example product team during development
14:16:07 [shadi]
q+
14:16:27 [shadi]
...to say the only goal is full conformance evaluation would be a mistakre
14:19:40 [shadi]
SAZ: think the document describes a process for conformance evaluation
14:20:05 [shadi]
...like the terms "purpose" or "focus"
14:20:34 [MartijnHoutepen]
q+
14:20:40 [shadi]
PK: seems we are not sufficiently expressing the dichtomy between absolute conformance
14:20:49 [shadi]
...and levels of conformance or performance
14:21:27 [shadi]
EV: have purposes already listed in the introduction section
14:21:57 [shadi]
...would be nice to get to a minimum requirement that is flexible
14:22:06 [ericvelleman]
q?
14:22:12 [shadi]
ack me
14:23:52 [MartijnHoutepen]
ack me
14:24:12 [shadi]
SAZ: agree on dichotomy issue, but this is separate
14:24:35 [shadi]
...discussion not tying back into this section
14:24:45 [shadi]
MH: agree with Shadi
14:24:53 [shadi]
...not best title for this section
14:25:19 [MartijnHoutepen]
zakim, mute me
14:25:19 [Zakim]
MartijnHoutepen should now be muted
14:25:27 [shadi]
...purpose and goal should be in abstract or introduction
14:25:32 [shadi]
PK: not commenting on title per se
14:25:46 [Vivienne]
q+
14:25:55 [shadi]
EV: agree, risk of what is sometimes understood by "conformance"
14:26:27 [shadi]
PK: maybe section explaining what we mean by "conformance"
14:26:28 [shadi]
...as the extent to which, or something like that
14:26:32 [Ryladog]
+1
14:26:32 [Liz]
+1
14:26:35 [shadi]
q+
14:26:44 [shadi]
ack viv
14:26:47 [ericvelleman]
+1
14:26:56 [shadi]
VC: like the way Peter put this
14:27:27 [shadi]
...asked often by clients if they conform and what they need to fix it
14:27:27 [korn]
"Representative Conformance"?
14:27:52 [shadi]
...then need to explain that we only evaluate a sample
14:28:11 [Sarah_Swierenga]
I agree with Peter and Vivienne on making sure we clearly define 'conformance' - 'representative conformance' would be a term, but maybe appropriate for our document/approach
14:28:27 [shadi]
...different people have different understanding of what conformance is
14:28:32 [ericvelleman]
q?
14:28:52 [shadi]
...several other terms that also need explanation
14:29:27 [shadi]
...to avoid legal issues
14:29:34 [shadi]
...we put disclaimer on our evaluations
14:29:40 [korn]
q?
14:29:41 [shadi]
...to explain this
14:29:59 [korn]
q+ to say that this reminds me of home purchase pre-inspections (e.g. for termites).
14:31:27 [shadi]
SAZ: agree with Peter's suggestion
14:31:35 [shadi]
...believe we even have the wording in the document
14:31:50 [shadi]
...but just not pulled out and clearly highlighted
14:32:11 [korn]
+1 to the new section.
14:32:31 [Vivienne]
+1 from me too
14:32:31 [MartijnHoutepen]
+1
14:32:48 [Liz]
+1 to add a new section on conformance.
14:32:52 [korn]
q+
14:32:58 [shadi]
q-
14:33:10 [shadi]
EV: strong support, will add to document
14:33:27 [shadi]
...but different from our initial discussion
14:33:55 [shadi]
PK: on Step 1.b, agree that title does not match the text
14:34:04 [shadi]
...text is about report results
14:34:13 [shadi]
...all about how much detail you want
14:34:27 [shadi]
...think need to be synched
14:35:03 [ericvelleman]
So instead of branching out the current section 1.b. we narrow it down to scoping additional wishes by evaluation commissioners that would normally not fall within the minimum requirements for WCAG-EM. This will make the methodology more flexible.
14:35:10 [shadi]
EV: thinking of it as "extra things that a commissioner asks for beyond the minimum of what you would do to evaluate the website"
14:35:17 [Vivienne]
q+
14:35:23 [korn]
q-
14:35:29 [shadi]
ack viv
14:37:04 [shadi]
EV: think goal is to do a conformance evaluation
14:37:30 [shadi]
...this document shold define the minimum requirements to do so
14:37:55 [shadi]
...evaluation commissioner may want more information than that minimum
14:38:27 [shadi]
...these additions need to be in the scope at first
14:38:28 [MartijnHoutepen]
q+
14:38:41 [MartijnHoutepen]
ack me
14:39:27 [shadi]
MH: maybe add this information in the "definition of scope" section?
14:39:28 [shadi]
q+
14:39:53 [korn]
q+
14:39:59 [shadi]
...could work with the basic report
14:40:11 [shadi]
...and extended based on the request
14:40:15 [MartijnHoutepen]
zakim, mute me
14:40:15 [Zakim]
MartijnHoutepen should now be muted
14:40:27 [shadi]
ack me
14:42:50 [shadi]
SAZ: think relates to the depth rather than the breadth
14:43:11 [shadi]
...it is about detail rather than area scope
14:43:34 [shadi]
PK: think may need a new 1.x section
14:43:46 [shadi]
...one that describes scope and detail
14:44:01 [shadi]
...and one about goal
14:44:02 [shadi]
q+
14:44:05 [shadi]
q- k
14:45:06 [korn]
q+
14:45:30 [shadi]
ack me
14:45:32 [Vivienne]
q+
14:45:48 [MartijnHoutepen]
q- k
14:45:56 [shadi]
SAZ: how does this impact the remaining steps as opposed to the detail?
14:46:27 [shadi]
PK: in a regression test, I will look at previous sample and add new pages
14:46:40 [shadi]
...that purpose impacts the sampling technique
14:47:05 [shadi]
...for development evaluation, I will prioritize areas of development
14:47:13 [shadi]
...so it influences the scope
14:49:27 [shadi]
SAZ: think have something on "re-testing" in sampling
14:49:57 [shadi]
...wonder if current "Baic Report", "Detailed Report", and "In-Depth Analysis" works?
14:50:27 [shadi]
PK: "Basic Report" to me is what I can give to a senior executive
14:50:38 [shadi]
..."Detailed" contains more about the issues
14:51:27 [shadi]
...and "In-Depth" is very useful for website owners and developers who are new to accessibility
14:51:51 [korn]
q-
14:52:02 [shadi]
...depends on if you are looking for consultancy on "what is wrong", or on "what is wrong and how to fix it"
14:52:06 [shadi]
ack viv
14:52:35 [shadi]
VC: think need separate sections for goal and reporting
14:52:48 [shadi]
...also have other extras that we sometimes offer
14:53:06 [shadi]
...for example testing with users if the client wants
14:53:35 [shadi]
...don't offer three levels of reporting but mostly do "In-Depth"
14:53:43 [shadi]
...that is the service we offer
14:54:02 [shadi]
...but like the suggestion of "additions requested by the commissioner"
14:55:03 [ericvelleman]
q?
14:55:07 [Vivienne]
I think we need to remember the difference between the purpose of the document and the purpose of the evaluation
14:56:49 [shadi]
SAZ: hearing agreement that title and text are a mismatch
14:57:00 [ericvelleman]
http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Jul/0028.html
14:57:24 [shadi]
...also that "level of detail" seems to be one axis
14:57:35 [shadi]
...then also "additional services" like testing with users or providing repair suggestions
14:57:47 [shadi]
zakim, take up next
14:57:47 [Zakim]
agendum 3. "Parameters of the sample size" taken up [from shadi]
14:57:59 [korn]
Eric - I would add to the parameter list any additional requests from the eval commissioner
14:58:26 [shadi]
EV: looking for parameters that influence the sample size
14:58:32 [shadi]
...please send feedback on this
14:59:32 [shadi]
zakim, take up next
14:59:32 [Zakim]
agendum 4. "Planning of next meetings" taken up [from shadi]
14:59:42 [shadi]
EV: holiday season
15:00:11 [shadi]
...intending to meet next two weeks
15:00:27 [Zakim]
-Peter_Korn
15:00:31 [korn]
korn has left #eval
15:00:38 [shadi]
[25.7 and 1.8]
15:00:39 [Liz]
bye
15:00:41 [MartijnHoutepen]
ack me
15:00:43 [Vivienne]
bye
15:00:44 [shadi]
...then two weeks break
15:00:45 [Sarah_Swierenga]
I will be presenting at HCII next week and then will be on vacation the following week. Good luck in the next couple of meetings!
15:00:46 [Sarah_Swierenga]
bye
15:00:46 [Zakim]
-MartijnHoutepen
15:00:48 [Zakim]
-Ryladog
15:00:49 [Zakim]
-Liz
15:00:50 [Zakim]
-MoeKraft
15:00:51 [Zakim]
-Vivienne
15:00:56 [shadi]
[8.8 and 15.8]
15:01:00 [Zakim]
-Sarah_Swierenga
15:01:01 [ericvelleman]
ericvelleman has left #eval
15:01:02 [Zakim]
-ericvelleman
15:01:07 [shadi]
trackbot, end meeting
15:01:07 [trackbot]
Zakim, list attendees
15:01:07 [Zakim]
As of this point the attendees have been Shadi, MartijnHoutepen, +31.30.239.aaaa, ericvelleman, +1.301.975.aabb, Liz, +1.517.432.aacc, Vivienne, +1.703.861.aadd, Sarah_Swierenga,
15:01:10 [Zakim]
... Peter_Korn, Ryladog, MoeKraft
15:01:14 [Zakim]
-Shadi
15:01:15 [trackbot]
RRSAgent, please draft minutes
15:01:15 [RRSAgent]
I have made the request to generate http://www.w3.org/2013/07/18-eval-minutes.html trackbot
15:01:15 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has ended
15:01:15 [Zakim]
Attendees were Shadi, MartijnHoutepen, +31.30.239.aaaa, ericvelleman, +1.301.975.aabb, Liz, +1.517.432.aacc, Vivienne, +1.703.861.aadd, Sarah_Swierenga, Peter_Korn, Ryladog,
15:01:15 [Zakim]
... MoeKraft
15:01:16 [trackbot]
RRSAgent, bye
15:01:16 [RRSAgent]
I see no action items