See also: IRC log
EV mentions discussion on sampling, wanted to send newer version of methodology
EV: lots of discussions on scope, will be worked
... sampling: 1 core resources, 2 Complete processes, 3 Random sampling
... Question: what are the minimum requirements for sampling?
<Mike_Elledge> looked good to me
Kathy: liked task-oriented sampling approach, identify core tasks by differnet types of users. Random sampling also OK, mor as validation
EV: Barriers and amoount of pages in sample an issue
Kathy: Regarding processes: if you run automated scan, that indicates pages with more technical issues
DF: raises issue of dropping issues in evaluation when they have been identified once, they may be played down too much
Sarah: asks clients about the different templates
used to assist in page sampling; start with core pages then extend
... concern about the random sample is the number of pages ending up in the sample
EV: agree to add templates as important element
Vivienne: Important to go through WCAG
checkpoints to select pages to whcih as many checkpoints as possible apply
... also uses random sample, but mainly uses combination of pages manually selected. Complete processes must not be missed
ME: Frequency of issue appearing - raises question of consistency. Templates help to check consistency of coding in website. Problem with limited sample is that developers may claim a structured approach but that may not be true and should be checked, ideally.
MH: In partial evaluations, can we use the same sampling rules?
EV: Needs discussion. Focus is full evaluations, but there may be claims for partial conformance.
MH: Maybe - full sample but parial conformance claim
EV: confirms focus on full evaluation
... any comment on the issue of trust between tester and site owner?
Vivienne: Does not see this tension as a big problem - parties need to develop trust, tester wants to be helpful for project owner, gets more out of it if communication is open and trustful
EV: Input from developer regarding setting the page sample OK?
Vivienne: owner should not dictate sample, but communicate to assist tester in sampling, explain scenarios of use, identify cirtical pages and processes
RW: When identifying a barrier, evaluator cannot tell site owner exactly how to fix a barrier, the developer must address the issue - a number of occurences might not be needed
Sarah: You cannot dictate trust but llook at
pages and paths to check whether design and a11y is used consistently, check
several parts from top to bottom
... not so much complete process but user task-based approach, looking for paths and features that tell what WCAG items are relevant
... Scenarios and tasks included
Detlev: difference between developer focus where issues are identified once, and user perspective where quantify of problems also counts
EV: raises question of automatic, semi-auutom.
and manual sampling
... can you have a full evaluation with an automatic sample?
<Kathy> manual sample is important
<SarahSwierenga> Manual sample is very important!
<Tim> automatic plus manual
Vivienne: Seems clear that relying on automated tools is not sufficient, issue is choice of methods to select pages that are then checked manually
EV: Raises issue of number of pages to sample
Vivienne: Australian Government has decided for national transition strategy that 10% of the page total should be checked - on large sites this is probably not feasible...
Sarah: Above 20 or 25 pages will not make sense, takes a significant amount of time, below 20 should be OK - customers will not want larger samples, costs get prohibitive
<vivienne> I agree with Sarah. Even 20 pages would take a significant amount of time
Sarah: Discount usability procedure is an example of a weay of getting a pretty good idea of the state, there are diminishing returns with more users / pages
RW: With a site like Amazon you got 1000s of pages but templates mean tzhey are mostly similar pages; on other more diverse sites you need a larger sample. A ruigit perecntage won't work
<SarahSwierenga> Responding to Richard - depends on what you call the 'site.' You seemed to be describing a portal-type design.
DF: is it fair to selct pages with more issues?
<kerstin> sorry, I have to leave
Kathy: If client has tight budget, should areas pages identified that have key a11y issues
EV: will be difficult to make part of the methodology text...
Kathy: Value of taking selectively pages
AG: Run a tool over pages to identify pages that where relevant for each checkpoint, makimng up the sample
EV: Good approach, Alistair?
<vivienne> What kind of tool would you use to identify this Alistair?
<agarrison> What we used to do was run a tool over the site to identify pages which would be most relevant to each checkpoint - maybe identifying 5 - 10 per checkpoint. All identified pages went into the sample.
<agarrison> This was a good approach for WCAG 1.0 - as pages were in effect delivered to the evaluator - however, WCAG 2.0 is a bigger beast.
TB: Site analytics could suggest most visited pages to enter the sample. There is no one sampling method working across the board; therefore the evaluator should document the sampling method used
MH: Choose some pages with multimedia if there are many in the site
EV: should pages be added manually (q to Aliastair)
AG: good point to select pages that are
frequently used - link that to pages to which many checkpoints apply
... Helps finding the few pages that have special elements like Video
<agarrison> http://code.google.com/p/googlesitemapgenerator/ have a look at this for selecting pages which are frequently viewed
EV: Proposes discussing number of pages in mailing list, continue discussion with AG about automatic selection of pages
EV: Is this version fit to be sent to WCAG WG as draft?
Shadi: no meetings next weeks, use draft and
study at the fireplace ;-)
... wonders whether current draft will give sufficient impression of the shape of methodology - may be not yet...
... use time to make it ready for publication
<vivienne> It might make a difference how much Christmas cheer we have consumed before that reading!
EV: will work in comments and hopes to publish update hopefully before Xmas
<ericvelleman_> Happily we have Sinterklaas in the Netherlands, so we already had our presents
Shadi: Availability for face-to-face meetings (survey) - some will be at CSUN, other opportunities in Europe
<agarrison> Should try England for meeting - seems to be in the atlantic between Europe and America ;-)
TPAC is a W3C meeting with other groups
<vivienne> Alistair, a little far from Western Australia.
Shadi: face-to-face good for detailed work, but
... more ideas for meetings at conferences are welcome, but may be difficult to get to
EV: Shadi might send list of meetings where he will be
Shadi: will attend most of them (in the survey)
EV: any other issues? No. Closes meeting.
<vivienne> okay, bye for now. A very blessed Chrismtas and New Year to you all.
<Kathy> Thanks! Merry Christmas to you too... bye
<Mike_Elledge> Merry Xmas and Happy Holidays, all!
EV: Merry Xmas to all, read Methodology, meet again on 5. January