WCAG 2.0 Evaluation Methodology Task Force Teleconference

16 Aug 2012

See also: IRC log


Detlev, Martijn, Shadi, Vivienne, Liz, Mike, Richard, Peter, Aurelien, Kathy, Tim, Eric
Alistair, Sarah, Emanuelle


Comment Resolution

The new comments. Reminder sent out survey last week and updated draft.

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20120730

Dated July 30. Complete document. See editor notes for changes.

<shadi> https://www.w3.org/2002/09/wbs/48225/WCAG-EM20120730/results

Will show approval or not. May get more comments from WCAG working group. Hopefully will also approve publication.

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730

At stage where most public comments addressed in doc, and gotten approval from this group to go ahead and publish.

Revised doc based on survey, pls review by next Monday, if not sooner. Will incorporate comments into working draft. Need 3-4 days to look over them.

<vivienne> huge apologies all for not doing my review this week - I wasn't able to get the survey done and wish I'd been able to do my share. Sorry

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c2

Will not go over comments now. Mostly simple changes and fixes. A couple will need more thought and discussion. One is #2: Link document from Wane Guage? Does not have official standing, but want discussions to elicit more comments.

Want agreement to open issue on referencing related resources. Will work on issue after publication.

<shadi> s/Wane Gauge/WAI Engage

Connection is not as clear as usual. But we will persevere!

Peter: LIke the idea of what WAI is trying to do, so want to explore linkages. Maybe in appendix some foreshadowing. Different roles in website dev, some roles have different impacts on WCAG criteria.

<richard> appendix for these type of links +1

Peter: "May find this info helpful." To provide reference, for information purposes.

Shadi: Before publication?

P: Doesn't need to be before draft publication. Yes to final.

V: In some WCAG criteria, have related resources. Provided but not endorsed statement. Some additional help, not required. Keep for next pub working draft. Not needed for this one.

Sh: Okay, great. No objections. We can revisit. When we pub next working draft, ed outreach working group will be involved. Asked them to complete review. Then joint mtg at Tech Plenary. Spend full day talking about it.
... Editorial refinements when have full doc standing.

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c5

<shadi> http://www.w3.org/WAI/ER/conformance/comments#c001

Sh: #5 from Detlev. More guidance on defining minimum stds for web browsers and AT. Baseline. Before A. Chen commented, asked to completely remove it. Proposed resolution: open issue on how to provide tool support.

Will be fairly tricky. What guidance should we provide. Issue has been present since beginning.

P: Will we note in draft what issues are, so public can know that there is known issue?

Sh: Yes, usually have pointers asking for feedback. Linking to issue will also be helpful.
... In status section of document. What input looking for.

P: Makes sense.

Sh: We will entertain motion to accept resolution of this issue to address both comments.

<Detlev> fine with me...

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c6

<Liz> +1

<vivienne> +1

<MartijnHoutepen> +1

<Kathy> +1

Sh: #6 from Detlev. Ambiguity between "tool" for eval tools and AT. When reading doc usually not mixed. Not referenced at same time. Need longer revision, nonetheless. Propose we work on with Ed Outreach.

<vivienne> +1

D: Maybe introduce abbreviations of UA and AT. Whatever way, not a big issue if clear from context. But prefer clear-cut definitions throughout.

Sh: Don't disagree. Usually at beginning of section is defined. Not so unclear from context, not major issue, but still good to go over this. Acronyms or something else, should address with editorial staff.

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c17

#17 from Kathy Waldin and Detlev. Tricky one. Two distinct examples from webpage (?). Could be more depending on situation. Not straightforward to get this right. Evaluator'

's call on how many pages are appropriate. Suggestion to keep working on this to get it right, including public comment.

<korn> +1

Sh: Does this need to be fixed for next draft?

D: Fine to open to discussion then. Don't hold back publishing draft.

K: I agree. Warrnets further discussion.

Sh: Worth discussing, won't be easy.

K: Can clarify it though. Get input.

D: Adding page states into draft. If page has several different states should be indentified, just like pages. If no objection, would like to add to draft.

Sh: You raised that as comment. Proposed resolution to include state of webpage. May apply to more than one section.
... Should be addressed in this draft.

P: Treatment of web apps and advice to evaluators on how to look at web applications is large and significant. Adding "page states" won't be sufficient. Putting reminder in before publication is fine but won't have finished by doing that.

<Tim> is "state of webpage" or "page state" well defined? Do we need definition?

Sh: Anticipate comments on web apps. See #13B. Not only "states of web page" but also section of document. Address more comprehensively than in previous drafts. Keep comments coming.

P: Thinking of opening issue about deeper review of web apps. We should remember.

Sh: This group also welcome to put comments on public draft. Ideally get it before, but if not can add them there. For everyone to review and comment.

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c32

Sh: #32: Providing a11y statement in document. Owner commits to remove any problems within ten days. Purpose response within ten days. Will get lots of comments.

TB: Why ten?

Sh: Anticipate quite some comments. Open an issue in any case. Look into more detail. What are conditions of a11y statement based on EVAL, we should put conditions and requirements.

<aurelien_levy> +1 for me

<MartijnHoutepen> +1

P: Uncomfortable with suggesting that a11y statement has to include resposnej/compliance within a atime period.

Sh: Peter please look at it and respond--by Monday?

P: How best done?

Sh: Just respond to latest email for group to respond to comments.

P: Sent an hour ago?

Sh: Yes.

P: Will do that now.

Sh: Maybe put in some more about your discomfort and proposed revision.

P: Yep.

Sh: Also, do you oppose opening issue to discuss?

P: Don't mind at all.

<Detlev> no opposition

<richard> I agree with Peter - it is not in our remit to specify how and when corrections should be made

Sh: Keep as is and discuss, or not publish. Two choices. Can take out and come back later.

<vivienne> no objection

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c34

Sh: Seems some agreement. Will be good discussion.

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c35

Sh: #34 and #35. Objectivity and repeatiblity of methodology.

<shadi> http://www.w3.org/WAI/ER/conformance/comments#c004

Sh: Evaluator has responsibility for objective assessment. ? Also says it's important for methodology to be objective. Also needs to be "Goodness" criteria.

Will be a lengthier discussion. Our plan is to have first final version by December.

Open issue to come back to this later. Won't be easy to address now.

<vivienne> no, I'm okay with doing it later

<Liz> +1

Sh: Needs to be resolved/discussed before publication?

D: Not sure it makes sense to open. Since not sure how we define goodness and objectivity. How can we define it? Don't see need to discuss.

Sh: Part of reason to define understanding of what it is. If evalutors follow methodology should have same outcome. But method not only factor. Many variables.

<Detlev> I'm fine with that discussion

<Detlev> q*

Sh: How do we reduce variability in methodology. Worth discussion. May not be able to address in next draft. May depend on practice after publication in Dec.

<aurelien_levy> can we have a recap on the face 2 face meeting in lyon ?

Erik: In Ministry of Interior--not a good place to get Internet.

Sh: Completes all the comments we agreed to discuss later. Since no objections except #32, may be able to close comments and approve publication of this draft next week.

<Detlev> I support Aurelien's suggestion

Sh: Look at two comments: #29 and #30. For those we will add to proposal hopefully tomorrow.
... Will go over Lyons meeting at end of call.

T: Comments we will have after draft, will we note still under discussion. So people will know they haven't disappeared.

Sh: Will ask for specific comments on those.

P: What is review period for public review.

Sh: Typically 3 weeks with W3C. Will extend to 4 weeks to be sufficient.

P: Just realizing what is consuming time WCAG to ICT and 376, somewhat longer review period will be wise. Espec. how much text came in at end of editor's draft.

Sh: Let's talk about it next week. Need to make Dec deadline as well.

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c24

Sh: #29 & #30 will send out proposals. Another comment needs more time for discussion: Use of templates #24.

<shadi> http://www.w3.org/WAI/ER/conformance/comments-20120730#c13a

Sh: Several comments mentioned our emphasis on templates. Related comment #13a to reduce it. Will be implemented in next draft, unless objections.
... Encourages evaluators to review templates themselves so can identify issues that might not otherwise be found. Some webpages are using templates but hard to see on web page. For instance, identify templates adn where they are.
... Can then find them on pages.
... Detlev said makes sense/doesn't make sense to evaluate tetmplates. What do you all thik.

V: I particularly look at templates, so don't have to mention issue on each page separately. Call Site-wide issues. Will provide visual map on template. Will refer to template area when find on pages.
... Will clutter otherwise. Still put Pass/Fail, but refer so don't have to discuss on every page.

Sh: Template, repeated content, and

D: Can you explain how you evaluate template? Use page with template, or extract template?

V: Ask commissioner to point me to common templates that are used. Then evaluate common elements on that template. Quite often have to evaluate specific areas, but will review common areas, like navigation, footer, etc. separate from content.
... Make sure content is still looked at.

D: Still wonder if look at template do you look at page that contains that template, or look at template itself.

V: Separate template.

D: Look at skip links on page to make sure they go to right place.

V: Yes, but can look at common areas.

ME: Like looking at templates within Dreamweaver.

Sh: Depends on type of template. Many types, including empty templates like Detlev said.

Upcoming face-to-face meeting

Sh: Need to discuss and decide on how much emphasis to put on templates. Everyone encouraged to comment.
... Lyons meeting: Monday and Tuesday. Potentially meet Tuesday with Educ Outreach Working Group, they publish the WAI resources on the home page. Excellent communication and education skills.
... Monday will be working on closing comments still open. Assuming publishing beg of December, 4 week comment period beg October, will use time to discuss thornier topics.
... Will be a subgroup, so can't make final decisions. Will have to bring back solutions. Need people to register so know who will be there.

P: Registration. If know can't come, can we register in negative?

Sh: No, unfortunately.

ME: Mike Elledge can't come either.

P: Any possibility for teleconferencing to discussions?

<Detlev> good suggestion (phone participation)!

<Tim> that would be great if possible

Sh: Depends on agenda. Will request phone participation. Sometimes difficult to coordinate, vote. Will think about more.


Sh: Also will be early in day for US or late for Australia. Don't want to exclude, but sometimes better to just have a follow-up.
... Maybe discuss particular topics on phone.

Sh: Please review comments. Hope you'll be in agreement with proposed solutions. Wrap up via email. Then publish.

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.133 (CVS log)
$Date: 2012/08/20 14:24:52 $