IRC log of eval on 2012-01-05

Timestamps are in UTC.

14:52:29 [RRSAgent]
RRSAgent has joined #eval
14:52:29 [RRSAgent]
logging to http://www.w3.org/2012/01/05-eval-irc
14:52:31 [trackbot]
RRSAgent, make logs world
14:52:31 [Zakim]
Zakim has joined #eval
14:52:33 [trackbot]
Zakim, this will be 3825
14:52:33 [Zakim]
ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 8 minutes
14:52:34 [trackbot]
Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:52:34 [trackbot]
Date: 05 January 2012
14:52:44 [shadi]
chair: Eric
14:53:01 [ssirois]
zakim, who is on phone?
14:53:01 [Zakim]
I don't understand your question, ssirois.
14:53:17 [shadi]
agenda: http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Jan/0004.html
14:53:53 [shadi]
agenda+ Welcome
14:54:11 [shadi]
agenda+ Methodology overall reactions
14:54:16 [shadi]
agenda+ Section 4
14:54:23 [shadi]
agenda+ Section 3.7
14:54:36 [shadi]
agenda+ Face-to-face meeting
14:55:18 [ssirois]
zakim, who is on the phone?
14:55:18 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has not yet started, ssirois
14:55:20 [Zakim]
On IRC I see RRSAgent, ssirois, vivienne, shadi, trackbot
14:56:02 [houtepen]
houtepen has joined #eval
14:56:43 [houtepen]
zakim, mute me
14:56:44 [Zakim]
sorry, houtepen, I don't know what conference this is
14:57:34 [Kathy]
Kathy has joined #eval
14:58:17 [shadi]
zakim, who is on the phone?
14:58:17 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has not yet started, shadi
14:58:18 [Zakim]
On IRC I see Kathy, houtepen, Zakim, RRSAgent, ssirois, vivienne, shadi, trackbot
14:58:33 [shadi]
zakim, this is evaltf
14:58:33 [Zakim]
sorry, shadi, I do not see a conference named 'evaltf' in progress or scheduled at this time
14:58:36 [ericvelleman]
ericvelleman has joined #eval
14:58:44 [shadi]
zakim, list conferences
14:58:44 [Zakim]
I see T&S_GLDWG()10:00AM, WAI_ERTWG(Eval TF)10:00AM, TAG_f2f()7:30AM active
14:58:47 [Zakim]
also scheduled at this time are IA_WebApps(Widgets)9:00AM, XML_PMWG()10:00AM, WF_TF()9:00AM, UW_UWA()9:00AM, Math_IG()10:00AM, MM_MMI(EMMA)10:00AM, Team_Global(review)8:00AM,
14:58:50 [Zakim]
... INC_(DecisionXG)10:00AM, SW_RDFa()10:00AM
14:58:55 [shadi]
zakim, this is wai_ertwg
14:58:55 [Zakim]
ok, shadi; that matches WAI_ERTWG(Eval TF)10:00AM
14:58:58 [Zakim]
+ +1.301.975.aacc
14:59:02 [shadi]
zakim, who is on the phone?
14:59:02 [Zakim]
On the phone I see ??P3, +31.30.239.aaaa, Shadi, +1.978.443.aabb, +1.301.975.aacc
14:59:17 [ssirois]
zakim, ??P3 is me
14:59:17 [Zakim]
+ssirois; got it
14:59:21 [dboudreau]
dboudreau has joined #eval
14:59:27 [Zakim]
+ +31.30.239.aadd
14:59:36 [shadi]
zakim, aaaa is houtepen
14:59:36 [Zakim]
+houtepen; got it
14:59:46 [shadi]
zakim, aabb is Kathy
14:59:46 [Zakim]
+Kathy; got it
14:59:50 [houtepen]
zakim, mute me
14:59:50 [Zakim]
houtepen should now be muted
14:59:55 [shadi]
zakim, aacc is Liz
14:59:55 [Zakim]
+Liz; got it
15:00:04 [shadi]
zakim, aadd is ericvelleman
15:00:04 [Zakim]
+ericvelleman; got it
15:00:09 [Detlev]
Detlev has joined #eval
15:00:20 [Zakim]
+ +1.514.312.aaee
15:00:34 [efong]
efong has joined #eval
15:00:46 [shadi]
zakim, aaee is dboudreau
15:00:46 [Zakim]
+dboudreau; got it
15:00:58 [Zakim]
+ +49.404.318.aaff
15:01:20 [ssirois]
zakim, mute me
15:01:25 [Zakim]
+ +1.517.353.aagg
15:01:26 [Detlev]
Zakim, aaff is me
15:01:33 [Zakim]
ssirois should now be muted
15:01:42 [Mike_Elledge]
Mike_Elledge has joined #eval
15:01:43 [Zakim]
+Detlev; got it
15:01:58 [vivienne]
zakim, mute me
15:02:03 [Zakim]
+??P30
15:02:07 [SarahSwierenga]
SarahSwierenga has joined #eval
15:02:10 [shadi]
zakim, aagg is Mike
15:02:16 [Detlev]
zakim, mute me
15:02:30 [shadi]
zakim, ??p30 is Vivienne
15:02:31 [Zakim]
sorry, vivienne, I do not know which phone connection belongs to you
15:02:41 [Zakim]
+Mike; got it
15:02:43 [Zakim]
Detlev should now be muted
15:02:43 [Ryladog]
Ryladog has joined #eval
15:03:05 [Zakim]
+Vivienne; got it
15:03:19 [Zakim]
+ +1.517.432.aahh
15:03:23 [houtepen]
zakim, unmute me
15:03:25 [shadi]
scribe: Shadi
15:03:31 [houtepen]
zakim. mute me
15:03:37 [vivienne]
zakim, unmute me
15:03:39 [houtepen]
zakim, mute me
15:03:45 [Zakim]
houtepen should no longer be muted
15:03:48 [shadi]
zakim, aahh is Sarah
15:03:56 [vivienne]
zakim, mute me
15:03:59 [Zakim]
Vivienne was not muted, vivienne
15:04:01 [Zakim]
houtepen should now be muted
15:04:04 [Zakim]
+Katie_Haritos-Shea
15:04:05 [Zakim]
+Sarah; got it
15:04:05 [vivienne]
zakim, mute me
15:04:11 [Zakim]
Vivienne should now be muted
15:04:11 [shadi]
zakim, who is on the phone?
15:04:13 [Zakim]
Vivienne was already muted, vivienne
15:04:19 [Zakim]
On the phone I see ssirois (muted), houtepen (muted), Shadi, Kathy, Liz, ericvelleman, dboudreau, Detlev (muted), Mike, Vivienne (muted), Sarah, Katie_Haritos-Shea
15:04:21 [shadi]
zakim, mute me
15:04:22 [Zakim]
Shadi should now be muted
15:04:24 [shadi]
agenda?
15:04:32 [Mike_Elledge]
q+
15:04:39 [shadi]
zakim, tahe up agendum 1
15:04:42 [shadi]
zakim, take up agendum 1
15:04:45 [Zakim]
I don't understand 'tahe up agendum 1', shadi
15:04:51 [Zakim]
agendum 1. "Welcome" taken up [from shadi]
15:04:56 [shadi]
EV: happy new year!
15:05:01 [Kathy]
zakim, mute me
15:05:02 [Zakim]
Kathy should now be muted
15:05:07 [shadi]
...new version of the methodology online
15:05:14 [shadi]
...changes to section 3 and 4
15:05:29 [shadi]
...addressed comments from the WCAG group
15:05:40 [shadi]
...getting more flesh but still needs more work
15:06:25 [shadi]
q?
15:07:03 [shadi]
...want to specifically look at scope and size of sample today
15:07:33 [shadi]
...counting the minimum pages is 25 but could lead up to 5000
15:07:41 [dboudreau]
q+
15:07:44 [shadi]
...5000 is unrealistic but 5 may be too little
15:08:02 [shadi]
ack mike
15:08:21 [shadi]
q- dboudreau
15:08:26 [shadi]
zakim, take up next
15:08:28 [Zakim]
agendum 2. "Methodology overall reactions" taken up [from shadi]
15:08:30 [shadi]
q+ dboudreau
15:08:51 [shadi]
ME: confused by the use of the term "resource"
15:09:11 [shadi]
...also wondering about getting hung on trying to define a specific number of pages
15:09:19 [shadi]
...maybe better to define ranges
15:09:30 [shadi]
...because could vary depending on context
15:09:40 [Detlev]
agree
15:09:41 [shadi]
...for instance goal and cost etc
15:10:20 [shadi]
EV: used "resource" but did not introduce it
15:10:27 [ericvelleman]
q?
15:10:30 [shadi]
...could discuss other options too
15:10:32 [vivienne]
q+
15:10:44 [shadi]
ack dboudreau
15:11:25 [shadi]
DB: we use the term "template", usually ~4 pages
15:11:35 [shadi]
..."template" because they are representative
15:11:52 [shadi]
...like pages with forms, multiple heading levels, etc
15:12:27 [Detlev]
absolutely agree
15:12:51 [shadi]
...we try to find the most important issues for the developers to fix
15:13:00 [shadi]
...usually in an iterative process
15:13:15 [vivienne]
I absolutely agree with Denis' approach, and I do much the same
15:13:30 [Tim]
Tim has joined #eval
15:13:31 [shadi]
...smaller evaluation first but a more complete one later on
15:14:20 [shadi]
...initial sample is small that can be done by manual evaluation too
15:14:46 [shadi]
...not dependent on large-scale automatic tool
15:15:05 [vivienne]
q-
15:15:14 [Mike_Elledge]
+1 to Denis' comment
15:15:17 [shadi]
EV: have description of "template"
15:15:31 [shadi]
...but just counting those we end up with several pages already
15:15:37 [Detlev]
q+
15:15:54 [ssirois]
I cannot disagree with dboudreau on that one since i do work the same way (at the same place! :P)
15:16:08 [efong]
15:16:29 [shadi]
...we also do not clearly describe the role of tools
15:16:37 [shadi]
...for example to help select pages
15:16:43 [ericvelleman]
q?
15:17:10 [Zakim]
+Tim_Boland
15:17:30 [shadi]
DB: many of our clients may not be large websites
15:17:55 [shadi]
...thorough review is up to 2 hours per page
15:18:56 [shadi]
...for small websites can be only 4 pages but sometimes up to 8 pages
15:19:09 [shadi]
...because sets can be overlapping
15:19:26 [shadi]
...like data tables may be together with forms or such
15:19:57 [shadi]
EV: also have preliminary review
15:20:09 [shadi]
...could that cover first phase?
15:20:25 [shadi]
q+
15:20:29 [Detlev]
zakim, unmute me
15:20:29 [Zakim]
Detlev should no longer be muted
15:20:35 [shadi]
ack det
15:20:35 [dboudreau]
zakim, mute me
15:20:36 [Zakim]
dboudreau should now be muted
15:21:00 [shadi]
DF: also in a similar situation with types of websites
15:21:16 [shadi]
...vivienne had also mentioned the pages with overlapping content
15:21:43 [shadi]
...also add specific "elements" to a sample
15:22:13 [shadi]
...for example only evaluating the tables in a page because the other aspects have been already reviewed
15:22:50 [Detlev]
zakim, mute me
15:22:50 [Zakim]
Detlev should now be muted
15:22:56 [ericvelleman]
q?
15:22:57 [shadi]
ack me
15:24:12 [shadi]
zakim, mute me
15:24:12 [Zakim]
Shadi should now be muted
15:24:18 [dboudreau]
@shadi - tho is exactly what we do when we have larger clients that can actually afford to start with a preliminary
15:24:22 [Kathy]
q+
15:24:31 [shadi]
SAZ: agree with what was said, leads to a dynamic approach
15:24:35 [dboudreau]
s/tho/this
15:24:51 [shadi]
...but need to keep preliminary review separare from conformance evaluation
15:24:57 [Detlev]
sure - states and elements have t be exactly determined
15:25:01 [ericvelleman]
q?
15:25:04 [Kathy]
ack me
15:25:08 [Mike_Elledge]
q+
15:25:16 [shadi]
ack Kathy
15:25:30 [shadi]
KW: agree with Denis and Detlev
15:25:59 [Ryladog]
q+
15:26:08 [shadi]
...but with more complex sites, particularly more application type, we find that more task-based approaches are more useful
15:26:32 [shadi]
...maybe need different scenarios on the different page sizes rather than a single limit
15:26:45 [shadi]
EV: example of a scenario?
15:27:16 [shadi]
KW: for example a retail store, you could have scenarios like "user registering" or "registered users checking out" etc
15:27:40 [shadi]
...still have repetitvness of pages but often the content is different
15:27:53 [ericvelleman]
q?
15:27:59 [shadi]
ack Mike_Elledge
15:28:00 [Detlev]
q+
15:28:02 [Kathy]
zakim, mute me
15:28:02 [Zakim]
Kathy should now be muted
15:28:09 [shadi]
ME: agree with everyone
15:28:33 [shadi]
...scenarios and tasks rather than a hard-limit on pages
15:29:04 [shadi]
zakim, who is making noise?
15:29:11 [ericvelleman]
q?
15:29:15 [Zakim]
shadi, listening for 10 seconds I heard sound from the following: ericvelleman (59%), Katie_Haritos-Shea (59%)
15:29:22 [shadi]
ack Ryladog
15:29:41 [shadi]
KHS: need to have something measurable and repeatable
15:29:59 [shadi]
...maybe need a range to give people a goal to shoot for
15:30:48 [shadi]
EV: range seems to make sense
15:30:58 [shadi]
zakim, mute katie
15:30:58 [Zakim]
Katie_Haritos-Shea should now be muted
15:30:58 [Detlev]
zakim, unmute me
15:30:59 [Zakim]
Detlev should no longer be muted
15:31:00 [dboudreau]
adding one page for every x page makes no sense if we had 2000 pages of smile content form the same template
15:31:01 [ericvelleman]
q?
15:31:10 [shadi]
ack Detlev
15:31:15 [dboudreau]
s/smile/simple
15:31:39 [shadi]
DF: with small websites it usually only needs a small number of pages
15:32:06 [shadi]
...also if you look enough you will surely find an issue somewhere
15:32:30 [shadi]
...need to focus on things that are vitally important
15:33:06 [Detlev]
zakim, mute me
15:33:06 [Zakim]
Detlev should now be muted
15:33:16 [shadi]
EV: looking at error margins later in the document
15:33:30 [shadi]
...will come back to it later on
15:33:38 [vivienne]
q+
15:33:40 [ssirois]
zakim, drop ssirois
15:33:40 [Zakim]
ssirois is being disconnected
15:33:41 [Zakim]
-ssirois
15:33:47 [shadi]
q+
15:34:05 [shadi]
EV: seems there is a tendency for less pages
15:34:08 [ericvelleman]
q?
15:34:11 [vivienne]
ack me
15:34:14 [shadi]
ack vivienne
15:34:39 [Zakim]
+??P3
15:34:49 [ssirois]
zakim, ??P3 is me
15:34:49 [Zakim]
+ssirois; got it
15:34:53 [ssirois]
zakim, mute me
15:34:53 [Zakim]
ssirois should now be muted
15:35:17 [Detlev]
blackboxing (local copies?) will often not work with dynamic sites
15:35:26 [shadi]
VC: wondering if need to consider proof of the state of the website at the time of evaluation
15:35:34 [Ryladog]
date and hour of each page tested - not the site
15:35:34 [shadi]
...do people mirror the website?
15:35:50 [vivienne]
zakim, mute me
15:35:50 [Zakim]
Vivienne should now be muted
15:36:00 [Mike_Elledge]
Definitely need to say when the site/which version/etc. is tested
15:36:32 [shadi]
EV: pages can change very quickly
15:36:35 [Detlev]
q+
15:36:47 [shadi]
...may need to cover in the evaluation report
15:37:03 [vivienne]
ack me
15:37:56 [shadi]
VC: wondering if we are going to do some form of proof of what was tested, maybe as part of the scope
15:38:04 [ericvelleman]
q?
15:38:08 [vivienne]
zakim, mute me
15:38:08 [Zakim]
Vivienne should now be muted
15:38:09 [shadi]
ack me
15:39:15 [shadi]
zakim, mute me
15:39:15 [Zakim]
Shadi should now be muted
15:39:26 [ericvelleman]
q?
15:39:27 [Detlev]
zakim, unmute me
15:39:27 [Zakim]
Detlev should no longer be muted
15:39:35 [shadi]
SAZ: sample size should not only grow by size but also by complexity
15:39:58 [shadi]
...Detlev had mentioned adding "elements" like tables or forms etc to the sampel
15:40:45 [shadi]
DF: many pages could also impact the score, especially if the score is based on the number of pages tested
15:40:51 [Detlev]
zakim, mute me
15:40:51 [Zakim]
Detlev should now be muted
15:41:05 [shadi]
...for example by diluting a single instance of a significant barrier
15:41:13 [dboudreau]
q+
15:41:16 [ericvelleman]
q?
15:41:19 [shadi]
EV: what about proof?
15:41:22 [Detlev]
zakim, unmute me
15:41:22 [Zakim]
Detlev should no longer be muted
15:41:24 [Mike_Elledge]
q+
15:41:37 [shadi]
...should we damand that people mirror the pages they test?
15:41:42 [shadi]
ack detlev
15:41:58 [shadi]
DF: did not work very well for dynamic pages
15:42:06 [shadi]
...is of limited value
15:42:22 [dboudreau]
+1 to detlev
15:42:28 [shadi]
...but may also use screenshots
15:42:52 [shadi]
...but essential is the date and time of the testing
15:42:57 [dboudreau]
ack me
15:43:05 [Detlev]
zakim, mute me
15:43:05 [Zakim]
Detlev should now be muted
15:43:09 [shadi]
...arguments over the result is not very frequent
15:43:16 [shadi]
ack dboudreau
15:43:34 [shadi]
DB: similar experience as Detlev
15:43:59 [shadi]
...do not always record the pages, hardly ever used
15:44:11 [shadi]
...but still good idea to suggest
15:44:38 [shadi]
EV: may be possible to suggest it but may not be easy to require it as part of the methodology
15:45:08 [shadi]
ack Mike_Elledge
15:45:19 [shadi]
ME: we do not save the pages for the reasons mentioned
15:45:40 [dboudreau_]
dboudreau_ has joined #eval
15:45:41 [shadi]
...but we do take screenshots to keep a sense of what the page looked like
15:45:46 [Kathy]
q+
15:45:56 [shadi]
EV: we save the code locally
15:46:11 [shadi]
...for occassional querries
15:46:31 [dboudreau_]
zakim, mute me
15:46:31 [Zakim]
sorry, dboudreau_, I do not know which phone connection belongs to you
15:46:40 [ericvelleman]
q?
15:46:42 [Kathy]
ack me
15:46:53 [shadi]
zakim, dboudreau_ is really dboudreau
15:46:53 [Zakim]
sorry, shadi, I do not recognize a party named 'dboudreau_'
15:47:10 [shadi]
ack Kathy
15:47:22 [Kathy]
mute me
15:47:29 [shadi]
KW: in addition to the screenshot also helpful to have the original code
15:47:39 [shadi]
agenda?
15:47:55 [shadi]
zakim, take up agendum 5
15:47:55 [Zakim]
agendum 5. "Face-to-face meeting" taken up [from shadi]
15:48:00 [shadi]
ack me
15:49:15 [shadi]
http://www.w3.org/2012/10/TPAC/
15:51:13 [shadi]
SAZ: no critical mass yet
15:51:23 [shadi]
...but really important to consider TPAC
15:51:30 [vivienne]
It's all far from Australia!
15:51:42 [shadi]
...could co-locate meeting with WCAG WG and/or other relevant groups
15:51:47 [shadi]
zakim, mute me
15:51:47 [Zakim]
Shadi should now be muted
15:51:53 [shadi]
zakim, take up next
15:51:53 [Zakim]
agendum 2. "Methodology overall reactions" taken up [from shadi]
15:52:01 [shadi]
zakim, take up next
15:52:01 [Zakim]
agendum 2 was just opened, shadi
15:52:08 [shadi]
zakim, drop agendum 2
15:52:08 [Zakim]
agendum 2, Methodology overall reactions, dropped
15:52:10 [shadi]
zakim, take up next
15:52:10 [Zakim]
agendum 3. "Section 4" taken up [from shadi]
15:52:16 [Detlev]
q+
15:52:41 [ericvelleman]
q?
15:52:43 [Detlev]
zakim, unmute me
15:52:43 [Zakim]
Detlev should no longer be muted
15:52:51 [shadi]
ack det
15:53:06 [shadi]
EV: need to test alternative versions?
15:53:19 [shadi]
DF: at least some aspects of the alternative versions
15:53:39 [Tim]
q+
15:53:40 [shadi]
...for example that you can get to the alternative version, or switch it on an off etc
15:53:53 [shadi]
...also need to ensure that the content is equivalent
15:53:58 [ericvelleman]
q?
15:54:08 [shadi]
...but not need to test the content twice
15:54:10 [vivienne]
I think that Detlev's point of equivalency of content is the important aspect
15:54:25 [shadi]
ack tim
15:54:31 [Detlev]
zakim, mute me
15:54:31 [Zakim]
Detlev should now be muted
15:54:42 [shadi]
TB: WCAG has a definition of "accessibility supported"
15:54:52 [shadi]
...how does that come into play?
15:55:27 [shadi]
q+
15:55:46 [shadi]
ack me
15:56:20 [Mike_Elledge]
Not sure I understand, Tim. Can you explain further?
15:57:20 [ericvelleman]
q?
15:59:34 [Kathy]
zakim, mute me
15:59:35 [Zakim]
Kathy should now be muted
16:00:53 [ssirois]
zakim, unmute me
16:00:53 [Zakim]
ssirois should no longer be muted
16:01:00 [Detlev]
bye
16:01:00 [dboudreau]
thanks all, take care
16:01:01 [vivienne]
thanks, bye for now
16:01:01 [Zakim]
-Tim_Boland
16:01:02 [Kathy]
bye
16:01:03 [Mike_Elledge]
Thanks, bye!
16:01:04 [Zakim]
-Sarah
16:01:05 [Zakim]
-Katie_Haritos-Shea
16:01:05 [vivienne]
vivienne has left #eval
16:01:06 [houtepen]
bye
16:01:06 [Zakim]
-Detlev
16:01:07 [Zakim]
-Kathy
16:01:07 [ericvelleman]
ericvelleman has left #eval
16:01:08 [Zakim]
-Shadi
16:01:09 [Zakim]
-Mike
16:01:09 [Zakim]
-dboudreau
16:01:12 [Zakim]
-Liz
16:01:16 [houtepen]
houtepen has left #eval
16:01:24 [Zakim]
-ssirois
16:01:28 [Zakim]
-Vivienne
16:02:26 [Zakim]
-houtepen
16:03:22 [dboudreau]
dboudreau has left #eval
16:04:32 [shadi]
regrets: Kerstin, Alistair
16:04:40 [shadi]
zakim, who is on the phone?
16:04:40 [Zakim]
On the phone I see ericvelleman
16:04:45 [shadi]
zakim, drop eric
16:04:45 [Zakim]
ericvelleman is being disconnected
16:04:47 [Zakim]
WAI_ERTWG(Eval TF)10:00AM has ended
16:04:48 [Zakim]
Attendees were +31.30.239.aaaa, Shadi, +1.978.443.aabb, +1.301.975.aacc, ssirois, +31.30.239.aadd, houtepen, Kathy, Liz, ericvelleman, +1.514.312.aaee, dboudreau, +49.404.318.aaff,
16:04:50 [Zakim]
... +1.517.353.aagg, Detlev, Mike, Vivienne, +1.517.432.aahh, Katie_Haritos-Shea, Sarah, Tim_Boland
16:04:57 [shadi]
trackbot, end meeting
16:04:57 [trackbot]
Zakim, list attendees
16:04:58 [trackbot]
RRSAgent, please draft minutes
16:04:58 [RRSAgent]
I have made the request to generate http://www.w3.org/2012/01/05-eval-minutes.html trackbot
16:04:59 [trackbot]
RRSAgent, bye
16:04:59 [RRSAgent]
I see no action items