See also: IRC log
<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20111102.html
EV: Shadi put on revised doc on line for me. Still needs filling out.
Different sections. Some discussion so far. Divide the sections, start with #1. Send in proposed text. Choose one section to start with, not most difficult, then have discussion.
<vivienne> how about starting with #6?
EV: Looked at sections, not really one without discussion. Before continuing w/ TOC, let's start w/ Methodology.
EV: Agreed on WCAG EM. Hold discussion, go back later. Any disagreement?
<vivienne> no, I'm fine
<Kathy> I am fine with that
<AmyChen> +!
<Detlev> OK
<AmyChen> +1
<agarrison> +1
<ssirois> fine with using WCAG EM for now.
+1
<Vincent> +1
<houtepen> ok
<vivienne> +1
EV: Lots of +1! Use WCAG EM as temporary short
time. Bring back to agenda later. Better name. Maybe we'll like and keep.
... Table of Contents.
<shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Nov/0014.html
D: Made contribution proposing reorganizing in
line w/ suggestion last week. From practical pov, general context, then general
procedure. Useful for guiding document. Some sections should be reorganized:
Procedure shld go thru SC or guidelines, then recogntion for checkpoints. Now
more a description of technique and not sequential when stepping through
checkpoints.
... Propose a different order.
EV: More or less in document. Not yet step by
step. Changing order would have to reflect how walk thru evaluation. You
covered it in your proposal.
... Discussion?
D: Don't know if people find approach useful.
EV: TOC looks theoretical. D approach practical.
<ssirois> Detlev proposal for table of content can be found here: http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Nov/0014.html
EV: Can add to TOC or other section. Everyone looked over?
V: Looked over, really like it. If someone
looking into audit, could understand what's req in part A. If someone already
knows could go to Part B. w/out hveing to go through whole section.
... Not get bogged down in theoretical details.
M: Makes sense to me.
S: Not sure can hear. <drops out> Unsure about B4: level of detail and overlap w/ existing techniques. Can figure out details later.
<vivienne> is section 4 - i.e. b4.1 etc. just an example?
EV: Agree w/ Shadi. Proposal for template, what it would look like. DK if want to go to that degree. Wld be lots of discussion.
K: Level of detail have to be careful not duplicate WCAG. Use ful to have links to WCAG, though.
<Kathy> zakim mute me
Sam: B6. DK how people will go into best practices, may see as part c. Could be middle of part c reporting.
EV: Have appendices for templates. Reports are out of document. Part c a good idea.
AC: Wondering if combine Detlev doc and EV outline. Splitting context and x so using same terms.
EV: Split of TOC and parts of method B, some things in TOC that should be in B. How can we make easier and create more overview. Practical overview missing.
AC: or split TOC in two parts, see what is missing, see how they fit together.
EV: will discuss with Detlev.
D: Just a draft, did not check to see if everythign there. Will need to see if runs down 12 checkpoints. Hve to include different SC depending on level. Why didn't go there. Techniques should not be replciated in methodology? However SC can be met by Check if headings are met, real value that you can bring techniques into somoe sort or order so don't get lost.
<shadi> +1 to Eric
EV: Should stick to describing when to use. Not like telling them per SC or guideline.
D: What's point of methodology then? Not telling peoole how to do it.
<shadi> +1 to Vivienne
VC: I kind of like what Amy was saying...taking original 1 and splitting it up. Like Detlev testing as B, C as perfoarmnce claims and reporting. Easy to use. Understand what D is saying, but we'd be repeating things in WCAG. Don't have problem putting links to SC and guideliens. Currenting difficlt to do in WCAG. Appendix that links things to make it easier. Methodology not a poinint by point, more how you do it.,
<Zakim> shadi, you wanted to say out of scope
EV: Makes sense. Documents are there. We're guiding people in methodology. Not reusing WCAG a second time. Danger if go into too much tech detail.
<shadi> www.w3.org/WAI/WCAG20/quickref/
Sh: How to use WCAG 2 guidance, how to follow it
precisely, quick reference guide more geared to that. Is being updated. Should
not be in that level of detai in this document. Amount of detial should look
into, cross links may make sense...
... Overall approach...methodology. Not about step by step testing but overall
assessment procedure.
<agarrison> People, including me, still have an unclear view of what the methodology will be and do. I suggest we get some content together first, and then re-evaluate the structure at a later date to make sure it reflects what the methodology has become.
<shadi> +1 to Alistair -- start drafting sections then we will need to reorganize anyway
Al: People still have questions about what method will do. Get some context into document. Evalaute structure at later date. DK what methodology will actually do.
EV: When we write sections will see some will be covered in different places.
Det: Different perceition than I. How will it be used? If won't be useful for evaluator, who will use it?
<ssirois> i agree with shadi about the methodology as an overall approach. i believe that clarification about the "understanding" wcag is responded by Education & Outreach. the methodology is the how to evaluate. not how to meet accessibility. in my humble opinion.
EV: Methodology for anyone doing evaluation,.
Difference betw writing details and using document in proper way. Help people
to set the scope of their eval. You're proposigin we go into more detail. We
should try to stay at higher level.
... WCAG is there. We point to it. If unclear, go to WCAG working group.
Should be so clear we only need to poitn to working group.
<AmyChen> http://accessibility.gtri.gatech.edu/aem/AEM1.html
AC: Thinkig that shouldn't repeat anything. Maybe
when there is unclear point give an example? Someone sent link to GA Tech
methodology. That one is good stayed at higher level. Thinking approach of how
to test product.
... Would point to WCAG for techniques if unclear.
Sh: Yeah think most has been said. Value add of method: 1, How to evaluate pages doesn't exist currently. 2. Applying WCAG 2 on selected pages. 3. Scoring, aggregating of results. Probably most difficult part. These three parts would be major contribution.
EV: Are they in current TOC?
<AmyChen> Shadi: Would you mind repeating or typing the 3 points you are making? we can't hear you well.
<vivienne> I agree with Shadi, as an evaluator they are areas that I struggle with - scoring and aggregating, etc
<shadi> http://www.w3.org/WAI/ER/2011/eval/eval-ws
Sh: The scorring is not. Agree with what saying
before. Start drafting and then things will pan out. Will reorganize any
way.
... ?
<shadi> [[Selecting representative samples of web pages from entire websites; this includes defining approaches for dynamically generated websites and web applications, large-scale surveys, and other contexts.
<shadi> Carrying out evaluation of individual web pages using WCAG 2.0 Techniques; this includes defining approaches for selection appropriate WCAG 2.0 Techniques and assessing Accessibility-Support assumptions.
<shadi> Aggregating individual results into an overall conformance statement; this includes defining approaches for assessing the relative impact on failures, potentially through incorporating tolerance metrics.]]
EV: Three pieces. Split into A and B. Start drafting and see where we arrive. Expect TOC will change. Important parts: selection, applying and aggregating and scoring will be there. Perhaps not as clear in WCAG 2.0 but should be.
Det: Add that no longer nec to have split betw A and B. Would help guide people through SC. So that kind of logic is still missing. No idea relation between SC. But won't need split if don't provide that context.
EV: Doesn't mean there isn't need for clarification. If not clear when drafting, we'll clarify. Could mean need more text of how to use it, without going into guidelines, perhaps pointing or linking. Will come up in drafting.
Vc: Few areas in WCAG where hard to know which
criteria accoutn for which things. Labels for example. Overlap. We can address
to make it clearer how to decide which area you score something in. A value
add. One item can violate a whole bunch of critera. But have to decide which
level it violates.
... People struggle which is violating.
EV: Perhaps coming more detail. Will have to linclude techniques if necessary in methodlology.
Det: If we dive in will need to be at guidelines and aspects VC just mentioned. Hve to be at level of SC, otherwise too general. Give at sC or don't. Dipping toe into water won't help.
<shadi> +1 to Alistair -- need to get started drafting
Al: Need to get on and push for first draft. Test it out. Will move on to second draft. Just get it down there.
EV: Sounds so easy! But I agree.
... Just start drafting. Send me text for different sections, place and
discuss. In mailing list, name and title. Put proposed text there. Or choose a
certain section and start on that.
Sh: Everyone is ready to contribute, become
co-editor, though we need to have a lead editor.
... Eric you are the lead editor, and do heavy lifting. But if you have a
certain section want to do, commit to it, no guarantee is what will be
final.
<shadi> +1 to Eric!
<AmyChen> +1 to Eric too!
<agarrison> Go Eric Go!!!
EV: Make a lot of text that is for sections not
in document. You'll see lots coming. Difficult thing to do methodology. Section
of website sounds so easy. Easy to miss things. Will need input.
... Will add things. Feel free to shoot at them.
AC: Question. Going through these in order, or just put comment in mailing list. Or focus on a few sections.
EV: Will send around link to section and we can
all comment on it.
... Let's keep it for this moment. Go to point 4.
EV: Got many scenaros and methodologies, but need
more. Any scenarios keep sending them to Shadi.
... Any other business?
Det: Want to raise questin Aaron Leventhal raised. Advanced dynamic web apps operating only on modern web browsers, some might not apply. For to different technology scenarios. Can ti be implemented?
EV: We could make it a discussion item for after this meeting. It is interesting and not yet covered.
Sh: So I think...had a series of discussions with Aaron...our approach for ARIA...but ARIA is a draft...part relates to ally support, all pieces that are needed for complete evaluation, but be carefl about scope creep.
<shadi> http://www.w3.org/WAI/ER/2011/eval/
<Detlev> yes
Sh: Refer you to...different activities on evaluation and testing that are taking...need all peices together to hav complete support. Are working on it, but can't do everything at once. Ours is just part of this.
EV: Answer question, Detlev?
Det: Yes.
<vivienne> thanks Eric. Looking forward to adding text into the document.
EV: Will have to start adding, will send link when do so.