WCAG 2.0 Evaluation Methodology Task Force Teleconference

26 Sep 2013

See also: IRC log


Vivienne, Peter, Martijn, Liz, Eric, Mary, Detlev, Kathy, Tim
Shadi, Sarah, Moe


ev: welcome
... new member: Mary Jo Mueller

MJM: introduces herself

EV: Working on new ED

Reporting - Appendix C

EV: survey 11 is still open


Appendix C http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#reports

<ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#reports

EV: not a lot work done yet on this
... question is: What do we want in the reports, is it optional

DF: appendix c says "should have'', does that mean "must have"?

PK: The sample report should be a sufficient evaluation statement
... should remain optional
... some of the bullets in the list are insufficiently specified

<Detlev> So it would be "Date of the "accessibility evaluation statement" rather than "Date of the conformance claim"

PK: f.e. terminology
... Example report should also be an accessibility evaluation statement

VC: good suggestion of PK, we need more explaining on the items of the bullet-list
... problem with validity in time
... uri's of claim is not reasonable, more reasonable to include uri's of sample

PK: 3rd bullet, conformance claim does not belong here

<Vivienne> maybe "tested WCAG level"

PK: see also "conformance level" etc. all occurrances of "conformance"

EV: copied some info from WCAG, we could include some (with reference)

DF: URL: sufficient to use just URL if it includes all pages on that site, agree with VC to include list of sample pages
... remover occurrances of "claim"
... only date to include can be the evaluation date

EV: we will look at the use of conformance and claim, etc

DF: Good idea to list all pages in sample

PK: the list of pages makes sence for traditional website, for applications it doesn't
... we need to include a way to describe states in webapps
... keep in mind development patterns, could be more data behind statement than can be included in list
... add release number , version number when known

EV: instead of date? extra option?

PK: varies, which is most applicable or most significant

EV: or combination

KW: list of pages does not always apply, sometimes specific use cases fit better
... sometimes use cases are very long (more than 50 pages)

MJM: when you use webcrawlers on enterprise sites, the list could be very long. Settings of crawler can also of use

EV: what do we want to do with output from automated testing

MJM: We have to have some generic example of the use of automated tool

EV: so many tools, has to be very generic

<Tim> maybe use several different tools (at least 2) - do we want to specify?

VC: we use automated tools as assistance, but not rely on them. We agreed to list tools used, but not get into the use of automated tools too much
... we wanted to steer away from reliance on automated tools

KW: in reporting, there is added value in including results of automated tools, though we would not want to rely on these results alone
... we should include this in reporting

EV: in the reporting step 5a we refer to an optional step 4e where we refer to the use of tools
... we don't have anything on crawlers
... at the moment just tools that support evaluation
... should we make a division optional/non-optional, should we define a minimum?

<ericvelleman> http://www.w3.org/WAI/ER/conformance/ED-methodology-20130903#reports

EV: in step 5a we already ask to report almost everything

KW: we do need to make reporting too much work if it is not necessary
... do not

DF: some items on the list should be mandatory, others can be optional

<Vivienne> +1

DF: maybe put the items in a survey and discuss this

<Liz> +1

Ev: I could split in two in a new ED

<Tim> +1


EV: will split into mandatory and optional

<Vivienne> Yes, Eric that's fine

PK: the most important mandatory parts are those that make it an evaluation statement

EV: what should be mandatory

PK: level A, AA or AAA

EV: DF mentioned date, pages in sample

TB: name, name of org, some attribution

the result

VC: the scope of what you're using, the guideline, description of the website, description of the sample

PK: scope of evaluation, critical, core features

EV: number of things already in step 5
... we need to include introduction text on evaluation commissionar etc

DF: scope of evaluation includes what is core, etc

EV: will include this in new ED

other issues

EV: survey will be discussed next time, things everyone agrees on will be in next ED
... no telco next week

PK: concerned on time dilation

<Detlev> fine

PK: lot of unanimity on lot of items, maybe you can put these things in the new ED
... as long as it is transparent

<Liz> +1

<Vivienne> +1

<MaryJo> +1

EV: does everyone agree?


<Tim> +1

<Detlev> you will be missed!

<Mike_Elledge> Yes--good luck to you Peter! I hope you'll be able to continue as an Invited Expert.

<Tim> hope you can continue..

<Vivienne> You would be sorely missed Peter,let's hope you can stay on

PK: will be leaving group, hope to focus on evaluation statement and reporting on the short term

<Mike_Elledge> +1

EV: great work PK

<korn> Thanks Martijn, Vivienne, TIm, Detlev, Mike, all.

EV: no telco next week

Thanks Peter

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.138 (CVS log)
$Date: 2013/09/26 15:49:53 $