See also: IRC log
<ericvelleman> <http://www.w3.org/WAI/ER/conformance/comments-20120920>
<Detlev> looked at it but not its entirety
eric: Please look in document and look for Please
Review in column.
... Second column is status
... Id 8 Please review telco discussion, commenter, location, terms and
conditions. Proposed resolution. Rationale.
<Detlev> Eric, can you post link to the questionnaire?
Eric: Please review document let me know if you do not agree with any of them. We have a few more days to look at.
<ericvelleman> <http://www.w3.org/WAI/ER/conformance/comments-20120920>
Eric: This link is the updated disposition of comments
Peter: How do we determine the actual comment
<Detlev> This is the link to the W3C survey: https://www.w3.org/2002/09/wbs/48225/evaltfq6/
Eric: First column id, 3rd column commenter, 4th column link to their comment
Peter: This is a bit cumbersome while their link may go to several comments.
Eric: Places text of comment in the document and proposed changed.
Peter: Change column regarding Suggested Change and Rationale to Commenter's Suggested Change and Commenter's Rationale
Eric: Look for "Please review outcome" in column 2
<Vivienne> can we have a couple of days longer?
<Sarah_Swierenga> +1 monday deadline
Eric: Is Monday, Dec. 10 a good day for deadline to review?
+1
Detlev: Please let us know what to do before Monday?
Eric: Just look for "Please review" in column 2. Review Proposed resolution and confirm agreement or disagreement.
<MartijnHoutepen> +1
<Vivienne> that sounds better
Detlev: Would like a later deadline.
Eric: Deadline, Wednesday, 12/12
... Submit feedback to mailing list.
<Detlev> fine
Eric: Let's return to agenda
... Let's finish survey by Wednesday 12/12/12 as well and we will discuss next
Telco
... Agenda item 3: #3. Extending the Scope to include websites during
development Please join into the discussion we started about this on the list:
<http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Dec/0001.html>
Peter: While there a 2 separate concepts, one
leads to the other.
... Maybe the case that we should talk about exclusion in two or more cases.
Not only in shipping the product. Maybe more comfortable with exclusion in one
or other.
Vivienne: Both extend scope. Still like to keep separate. Some people will agree with one and not the other.
<Detlev> agree
Eric: #1 Extending scope to include websites during development
Peter: I think this is very important. We want to encourage use of our methodolgy during development as much as possible. Need to make a distinction about making a final report but see no reason why we should not invite developers to use methodology.
<Detlev> fully agree (as you know I would)
Peter: Only need to address situations where one
would need to make a public statement.
... We have no control over how developers use public entities but would be
useful to encourage it.
... We could add text in the introduction that would invite people to not wait
until application or site is done before using this methodology. Would not be
appropriate to publish final report during development while not ready for
that.
Vivienne: Difference is intent. Development, test
as you go. Cannot make a statement of compliance. Can be used during
development to test as you go however a full test is needed at the end.
... Gradual inclusion at the beginning. WCAG supports incremental testing,
life cycle from the beginning.
... Try to not include something that is not satisfactory. Knowing your
website is not compliant. Intent is not to get "out of something" but rather to
perform testing on an incremental basis.
Eric: I agree. More or less what was said by Detlev during last Telco. Made difference between the exclusions. A full compliance statement is not possible during development.
Shadi: I feel we are repeating quite a bit that
we discussed at face to face. Fully agree with Vivienne and Peter. The keyword
is purpose. Looking at work statement and abstract of the document, the main
purpose is to evaluate a website that exists.
... Not only make a statement but also deliver a report.
... Could be a website that is in development or a website that is completed.
Think of purpose rather than target audience.
... Encourage evaluation throughout development otherwise one will not achieve
accessibility. Primary purpose is still to evaluate existing website but does
not exclude one from testing during development.
<korn> +1
<Sarah_Swierenga> +1 agree with "purpose" emphasis, rather than target audience
<Vivienne> +1
+1
<Detlev> fine
<MartijnHoutepen> +1 to shadi
<Kathy> +1
<Liz> +1
Shadi: The outcome of the f2f discussion, please review the minutes. We did not have quorum.
Detlev: Can use the same process for both phases to be ensure that you have a proper outcome.
Correction: Detlev made previous comment.
Shadi: If pages are known to be inaccessible,
this is a method for sampling pages of an existing website. If you know website
in development, you may know that the outcome will ... Don't want to exclude
someone who wants to use this in development but do not want to expand scope of
document.
... Not sure that would need to be a change in the body.
Eric: Let's propose a redraft.
... Want to give a bit more time for folks to review the comments.
Eric: #4. Extending the scope to make exclusions possible Please join into the discussion started about this on the list: <http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Dec/0002.html>
Peter: Makes sense if someone performing the
evaluation, especially during development, to make a statement that they will
limit the scope of their evaluation.
... If we know a certain class of failures will happen, it may make sense at
the outset to exclude things while it is known that they will fail. It is
recognized that people will do it.
... Real quesiton: If you do this, are you able to make a conformance claim?
... Doing exclusions is a matter of practicality. Very useful and normal thing
to do.
Detlev: Agree with Peter. The real issue involves
conformace claim.
... Minimum: on a page level, everything has to comply or conform.
... Are cases in limiting certain things, like PDF document. Would need to
document that the inaccessible PDF is available. Clearly flagged as
inaccessible.
... Need to clearly draw the line.
... Should not raise the impression that everything is fine and conforms.
Shadi: May come to completely different results
and this raises confusion. We are providing a standardized procedure for doing
that.
... Don't think we should claim that we follow the procedure unless it is
truly followed. Reminder: This document is informative and not normative. Can
use the guidance but to make a statement about conformance need to use all
parts.
Vivienne: I agree with Shadi. Uncomfortable to
making this guidance messy as to what it can be used for.
... The purpose of this methodology is to evaluate entire websites. While one
can use this methodolgy in any way they choose, it is designed to audit a
complete website once established.
... Don't want to encourage folks to exclude parts of the websites if they do
not comply. Ok to add a statement in the intro that you could use methodology
as a guide but cannot make a statement of conformance or compliance.
Peter: I think real crux of the matter is
compliance statement. The world is messy already. When look at single url web
applications that extremely complex which has accessibility bugs that are not
hit during review pass.
... Take slight exception of purpose is to audit. The purpose is to evaluate.
<Vivienne> sorry Peter, poor choice of words with 'audit' meant 'evaluation'
Peter: There might be other statements that could
be made just short of a compliance statement. Is there other output that one
might make other than 100% compliance statement?
... Don't have a problem if you are making a compliance claim that you should
not exclude anything
... Purpose of report should have more than just a compliance claim.
Kathy: Majority of accessibility reviews are not
on complete products. We review as we go through. Would be good to have
methodology apply to development as well. And important to indicate that this
cannot be a compliance claim.
... Majority of accessibility evaluations that happen are not on full products
or full websites, especially when evaluating applications.
Eric: Agree if one wants to make a compliance statement there should not be a possibility of excluding things.
Shadi: I am interested by Kathy's statement.
Maybe there is something that we should consider. Could use methodology in
different ways but the question is around conformance statement.
... When evaluating products during development, how important is it to follow
a methodology such as Eval TF? As compared to after website dev is completed?
Kathy: The methodology is a good way of saying
this is how you go about testing a website or application. Here is how to
select sample. This procedure is valid whether looking at subset or entire
product.
... Challenge is that you have multiple development teams working on different
parts and finish at different times. Ususally you have a contained set of
functions per review.
... But still follow the same methodology.
<Detlev> Agree
Kathy: Report on all reviews the same way as if doing a full review. We combine results into a much larger review for conformance claim.
Eric: Biggest item is conformance claim and
whether you can or cannot exclude things. Let's take discussion to the list.
When would we want exclusions? What would the compliance statement depend upon?
... Focus on this during the coming week.
Detlev: Eric you state, "only point to make is that it is clear for final conformance statement we should not have exclusions". Feel this is still an open question. Still will have inaccessible PDFs etc.
Eric: I understand the legacy part also. Maybe we can point to WCAG statement on legacy parts.
Eric: #5. Other issues
... If not, wish everyone a good week and encourage discussion around these
topics.
Eric: Plan to launch an edited draft next Wednesday.