URL: http://www.w3.org/WAI/ER/conformance/comments-20140130
SA: We will be going over the comments
SA: Tension with Techniques - prep disc with Micheal Cooper, hopefully we can draft resolutions that the entire group can adopt later on'
SA: MaryJo I responded to IBM's comments - thank you verymuch
SA: We are missing a few comments on the page
MJM: I had sent raw text but it was made into a table. So I cansend it to yo inits entireity intext
SA: I need to see if there is something I missed. Otherwise we should have all of the comments in
TB: We are going to prioritize then?
SA: Yes, we will go through for half a minute and say this is for add discussion or - we have exceeded 100 comments
SA: I am very glad that most of the comments are editorial - so we will NOT discuss those
SA: As we go along I want to draw up the high level issues...these are the things we need to tackle - and other comments mat be related
SA: This is not really how WCAG works
TB and KW: This about testing
MJM: Not the intent of this document to go into all that
KW: We do not want to go into specific tools
SA: How to test we have specifically scoped-out this, but it is something we need to speak with WCAG about. People want guidance on point by point checking
<shadi> ISSUE: how much guidance on point-for-point testing can we provide in WCAG-EM
<trackbot> Created ISSUE-14 - How much guidance on point-for-point testing can we provide in wcag-em. Please complete additional details at <http://www.w3.org/WAI/ER/2011/eval/track/issues/14/edit>.
SA: That is the same issue - how we do point by point checking
EV: I think we addressed this in the document
SA: 2 things - 1. how to test and 2. what to test with - maybe we need to provide more guidance of these two issues
KW: Can the W3C do a survey to get statistically valid data on what is being used
<shadi> ISSUE: how much guidance on selecting appropriate "baselines" can we provide in WCAG-EM
<trackbot> Created ISSUE-15 - How much guidance on selecting appropriate "baselines" can we provide in wcag-em. Please complete additional details at <http://www.w3.org/WAI/ER/2011/eval/track/issues/15/edit>.
TB: it would be good document what works with what
SA: I would be happy to walk folks through the Accessibility Support Database that we have at the W3C
TB: Just start a repository
SA: But statistically valid is harder to accomplish - no Working Group currently is chartered to do that
KW: Judy and I talked about doing a survey for mobile yesterday
SA: It was called Have Text Only
TB: Do they mean that the site shouldbe continuous monitoring in time
SA: Come back to this later
SA: Instead of 1.0 it could be misinterpreted.
SA: The point we want to make is to make it even LESS formal
SA: We may/will publish an updated version
SA: The tentetive point to bring out here is that we want to - Drop the version number all together
EV: Maybe we could say that there is a date attached to this version
SA: That is what we feel in this room right now
KW: yes we can use the date
SA: And the permanent URI
SA: Provide a PASS/FAIL rating
SA: I think that is what we do hvae
SA: Mandate 376 the result template there is PASS FAIL and PARTIAL. But we said that this does not apply to web
SA: Applies to scoring, but for individual SC
SA: Let us assume that she is talking about SC
SA: Tolerance and then for each SC do we want PASS FAIL and PARTIAL?
SA: Intro section
EV: It does say here that they are SATISFIED
EV: We do point to WCAG 2 to be satisfied
DM: Every page that you test HAS to pass
SA: One is for individual SC (cam be pass/fail) the other is "I evaluated this website and there is one minor issue" what do I do?
SA: We can change the whole thing around. This one can go back to the EDITORS
TB: Is there some other place to have a list - as technologies change
SA: Accept
SA: This relates to the advice we give on Techniques
<shadi> issue: need to further discuss the role of techniques (with WCAG WG)
<trackbot> Created ISSUE-16 - Need to further discuss the role of techniques (with wcag wg). Please complete additional details at <http://www.w3.org/WAI/ER/2011/eval/track/issues/16/edit>.
SA: Common procedure is not identified
SA: Common procedure, is common approach
DM: Maybe she thinks this is jargon
SA: Editors will make it less jargony maybe approach
KW: They are suggesting we add this?
SA: I do not agree with this wording
SA: The specific situation doesn't matter ,
MJM: But it is not a bad population to include
MJM: Editorial- link to the definition
SA: This one is from the EO Working Group and this relates to 'web page states'
SA: It is too jargony and too broad
SA: One suggestion we can reuse the 'change of context' that WCAG uses
SA: We think the name is good - we just need to improve the definition
DM: This is if you are taking somewhere new without letting the user know
SA: Either define better or change it to ANALYZE in Step 4
KW: Put the definition for AUDIT in our document
SA: If we agree with how we use it now. We either define AUDIT or RETITLE that section
MJM: Or we thank the commentor and say we discussed that we think this word is well used in this case
TB: Rationale for why we didnt make the change
SA: We need a resolution
Shadi: comment is to make clear that this is
thesis
... discussion here is that review team can cover a broader skill than an
individual - that is what we indicate
Katie: usually look at combined years of experience
David: on one team you have 4-5 people; worried
about perception of team
... expert looking at the site and combine feedback from users
Shadi: 69 is related
... by review team we are talking about people who is familar with auditing
the site
David: concerns about combined expertise
Shadi: want to be clear that it can be done by one person
David: should be careful that we don't say that one is better than the other
Shadi: should we have more discussion on this
... using combined expertise resource does say it is better - from 2005 so we
may want to look at this
David: has not seem team approach working better
Katie: some groups do combined testing
David: canadian government testers are just one person doing the testing
<shadi> issue: need to further discuss if team of reviewers is (still) preferable over single reviewer
<trackbot> Created ISSUE-17 - Need to further discuss if team of reviewers is (still) preferable over single reviewer. Please complete additional details at <http://www.w3.org/WAI/ER/2011/eval/track/issues/17/edit>.
Katie: other organizations have people doing testing have multiple testers testing
David: even in QA departments there is often one person
Shadi: we may come to the conclusion that the doc is outdated
Shadi: clarify that there are two separate
teams
... review teams and involving users... two separate sections
... important point is the division of responsibilities
David: I don't think it is reality
Katie: bug tracking systems collect tester information, then it goes back to the lead and then the infromation goes to the developer
Shadi: we need further discussion
Shadi: Principle of Website Enclosure
... could put that as part of the other section; does not need to be a
separate section
... can only expand scope
... heading now for this section
... is it good to make it stand out
David: it is a sub-heading
Kathy: good, leave it as is
Shadi: people could then reference this section in the doc
David: should define website enclosure
Shadi: all these sections overlap but responsive design needs to be called out separately
Kathy: will write section
Shadi: third-party evaluators
... more difficult to do by third-party evaluator
David: require insider knowledge - interview the developers
Shadi: the methodology is not get to a conformance claim
David: not realistic use case
Kathy: don't think that you would get into that situation
Shadi: not realistic for black-box testing
Kathy: call it black box testing not third-party testing
Katie: we should add a definition
Shadi: editorial comment
Mary Jo: what are the special considerations
Shadi: user generated content is considered
third-party content
... we need to be clear that we are evaluating
Eric: isn't this covered under WCAG
... pass or fail
Shadi: do you regularly monitor this content
Eric: people have 2 days to repair the content if complaint
Shadi: need to find out if there needs to be monitoring
Eric: in Netherlands, this occurs with video
and captions
MaryJo: blog comments
Shadi: if no monitoring then those pages could
not be compliant
... partial conformance except for the comment area
Shadi: is a link thrid party content?
David: each http address has it own site
... exception is step of process; e.g. paypal
Shadi: another example is help site
David: if you have control of the content then
you are responsible
... could not define a website
... so not in WCAG
Shadi: what should we do here
David: this is a hard question
... do you own it, have control of it
Shadi: evaluation of scope is what is included
David: we need third-party content
definition
Shadi: two things - linked vs. embedded - does
not matter
... third-party or not third-party
Katie: also part of a process
David: embedded is scope
... link then it depends
on whether defined in scope
Shadi: need to defined what is third-party
David: the difference is http address
Shadi: we are talking about dependencies - for
example blog with comments
... is it required to get the full information
... Google map may be redundant information
... in scope- must use this for the function of the website
Shadi - More than complete process, it is dependencies
David: this could have a huge impact
Shadi: look back and make sure it is clear - talking about dependencies
Shadi: evaluating during development
David: authoring and developing are the same thing
Kathy: author/editors different from developers
Shadi: part of the website, content that is being inserted
Eric: proposed no change
MC: I came from the element by element background... but no really realistic anymore
Cathy: Most people these days.... check all alt text, all forms, section by section, element by element
Tim: People want to know "what do I need to do"
Shadi: We don't want people to use this methodology to compare web sites
MC: need to say scores fro differe t evaluators cannot be compared
Kathy: Would have to be weighted... can't just can't provide accurate score... too subjective
Shadi: Is there a way forward on this or should we delete it?
MC: Need to discuss with WCAg tomorrow
Kathy: We had a number of options presented...
could not consense
... Incremental improvement can be indentified other ways from scoring
... Getting a level of score without reference to previous scores is not very
easy
Judy: Inter rater reliability is one of the
biggest issues
... It would be a great help to solve
Cathy: We could look at what grading designs have done...
Judy: Is there a way that we can have a score?
David: Team issue
Judy: We thought that getting a handful of teams, have people rate the site from different locations.... we weren't able to get enough teams... we wanted to train up a proliferation of peer reviewed organization to show that the results that would be in a range and accepted in the field
Shawn Henry: Sounds like we may want to archive using combined expertise
Shadi: We will discuss this tomorrow with WCAG Section on review teams