WCAG 2.0 Evaluation Methodology Task Force Teleconference

shadi: There will be no meeting next week due to US holidays and Shadi and Eric's travel plans

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131119

Shadi: Putting together next editor draft. Working to complete the draft and go into survey.
... Working discussion into document.
... Steps 1 & 2 and 4 have had lots of changes. Step 3 needs a lot of work.
... Comments received from Vivienne and waiting on comments from Moe and Kathy regarding Editorial review.
... Launch survey next week with the WCAG working group with full editor draft.
... Are there any particular comments or questions?

Detlev: Would like to add an important point to our business, Definition of States. This is very important for the entire Methodology.

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131119#procedure

Shadi: Let's look at Evaluation Procedure section.

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131119#step1

Shadi: Some copy/edit at early part of section. In Step 1, there is a little more cleanup but no big changes.

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131119#step1c

Shadi: The one thing we need to come back to is Section 1c defining techniques and failures to be used.

<shadi> http://www.w3.org/TR/UNDERSTANDING-WCAG20/understanding-techniques

Shadi: We describe a fair amount of what techniques and failures are in WCAG because we decided WCAG 2.0 had very little information however now there is more guidance in WCAG 2.0.
... There are other bits and pieces that they have added so what we have is redundant with what WCAG WG has in their working document, however, our flow is not too bad as written. Is it good to have this here to reiterate?

<agarrison> I would keep 1c as is.

Vivienne: I think it is important to keep at least a mention in here regarding the techniques. Remind people what the role of techniques and failures are and that you should test according to Success Criteria and not techniques.

<Mike_Elledge> +1

Detlev: Definte the way WCAG techniques and failures are used and how to go beyond techniques. Make our statement more pronounced. May be techniques on page to conform to WCAG but may not be documented in WCAG techniques but is sufficient.

Shadi: Two sections talk about Techniques and failures 1.c and 4.c.
... 1.c talks about techniques and failures before evaluation.

Detlev: Define in what context these techniques would be used.

Shadi: I plan to evaluate the website with these techniques because these are the techniques used to develop the website...
... Section 1.c is optional.
... We don't explain the situation in this section. Do we want to?

Martijn: I agree with Detlev. Defining the failures to use may not be a good idea. Move into section 1.d as something you aree upon.

Shadi: What do you mean by not using failures?

<Detlev> Anz failure that applies should be used!

Martijn: Just seems strange. Could say that they report specific failures.
... Could define techniques used in 1.d

Shadi: In upcoming WCAG documents Understanding Success Criteria also includes failures. We could drop failures as well. Just a conceptual type thing. An evaluator does use a failure technique

Alistair: I think this section is important. Would make things a lot clearer if agreed by the parties involved as to what techniques will be used.
... You can use the techniques defined in WCAG but it is important to document other techniques.

Mike: Would it be good to give folks a reminder that if they use non-WCAG 2.0 techniques, it would be good to promote them to WCAG 2.0 to have them included?

Shadi: I like to promote this wherever possible but is that too much?
... Will take this as an editor consideration.
... Might be useful to reference section in WCAG

Vivienne: Rereading 1.c which is optional. I am okay if you want to say define the techniques to be used. Is this WCAG techniques or how you will evaluate the page which may be personal techniques.
... Need to be clear about the intent of what techniques are being referred to here.

Shadi: These are all WCAG techniques. Techniques for Authoring and Techniques for Testing. Advisory and Failure Techniques. These are WCAG techniques.
... Wonder if we need to clarify in this section what we are referring to when we say techniques.

Vivienne: Yes. We should clarify.

Shadi: We do say throughout this does not have to be w3C technique. But this section is exactly for specifying techniques used for example "corporate guidelines"
... I want to evaluate with "German BITC checks" Does not have to be W3C techniques but must satisfy WCAG criteria.

Detlev: Put purpose right up front in this section. Too far down.
... Need to mention all techniques to be used in order to evaluate the website and not just custom technique. If one technique fails some other technique may meet the success criteria.
... Just an additional item for evaluators to keep in mind.

Shadi: But we also have to consider failure techniques. "This is the way we do testing at our company. These are the checks that we will be running" Also define particular set of techniques and failures that will be used. This set may need to be expanded during the evaluation.

<Detlev> I think the focus here is really "custom techniques"

Shadi: Which situations this is used in is an important note.
... Ok, this was enough good discussion. Have enough feedback to bring forward to Eric and to keep this section in the draft to reiterate use of techniques and failures.
... Cleaned up Step 1.d Additional evaluation requirements. Step 1a and Step 1b are the essential parts of defining webiste and conformance targets, additional requirements come from evaluation commissioner.

<Detlev> I still have a comment to Step 1.a: Define the Scope of the Website.. but can add that to mailing list.

Shadi: Also worked on Section 2. Explorin the target website. Removed repititive steps regarding you may not find all functionality or all pages at first. This get repeated alot throughout sections.
... Pulled this up into intro section of Step 2 so we only say this once.
... Step 2b is a little confusing.

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131119#step2b

Let's take a look at Step 2b http://www.w3.org/WAI/ER/conformance/ED-methodology-20131119#step2b

Shadi: Need to at least identify core functions of website and generate use cases.

<shadi> [[Web pages from the different website areas (home page, web shop, departments, etc.) including any applications;]]

Shadi: People feel concerned that items will get prioritized. There is an addition to Step 2c
... For example, website of large bank, Net Banking Application is main app plus a currency calculator. The currency calculator is not core, however, still want to get a snapshot of the calculator application and ensure it is accessible.
... Test calculator application more coarsely than overall Net banking application.
... Do we need to identify a different type of emphasis on the testing of the calculator app?

Vivienne: Are you saying when we are evaluating the web pages, that we look at some differently? Or when we are choosing pages, we need to ensure to keep a variety of pages?

Shadi: This is the exploration phase. Explore core functionality. Identify variety of web page types from different areas.
... Later we do not yet reflect in our current document, but this has an impact on Selection, 4.a Checking
... Core functionality will receive more emphasis
... Need to do some level of prioritization
... Would not exclude other functionality like the calculator app but it may not have the same emphasis.

Vivienne: Outcome - list of type of web pages. Is this a required section? Are we going to say we are required to document the type of pages before making the page selection?
... If so we may want some more in Step 2c to document to properly reflect the type of pages found on website.
... The reason we identify the variety of pages is to ensure we include the right balance of pages. Ensure page selection reflects the types of pages on the website.
... Just make it clear why we want to identify the variety.

Shadi: The idea being "Here is what I found. Here is what I selected for web pages. Here are the results."
... Section 2e Identify other relevant web pages, "Identify accessible features", "Contact information"

<Detlev> Shadi, briefly cover Step 1.a.?

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131119#step1a

Detlev: Section 1a does not really spell out 3rd party content that you find. Need to identify whether or not I am allowed to remove 3rd party content from evaluation like Social Media buttons included on the page.
... Do we need to add a statement here regarding how to define what type of 3rd party content can be removed from scope.

Shadi: Scope of applicability should answer question. It does have a section on 3rd party content.

<shadi> http://www.w3.org/WAI/ER/conformance/ED-methodology-20131119#thirdparty

Shadi: Maybe provide a link to the sub-section in Scope of applicability.

<Vivienne> other examples are external database for library website, or YouTube linked video

Vivienne: Some things we see for example on a library website we link to external databases that we have no control.
... Although included in the audit need to be clear that these are out of their control.
... Evaluator needs to know how to deal with that.
... Another example is a link to a You Tube video without captions. Out of website owner's control.

Shadi: What would you do with a library site that has an external database?

Vivienne: I would identify that the database is not accessible and add a statement to such in their records. For Section 508, things are tied to procurement in Australia. The only place you can get certain functionality is through 3rd parties
... Need to identify this 3rd party accessibility in accessibility statement.

Shadi: What type of statement is written comes after.
... Right now we are defining the scope which includes 3rd party content.

<Detlev> OK, fine - didn't see that provision

Alistair: Author of content. We are expecting that the author is conforming to WCAG 2.0 There is a possibility to update document and make accessible. When content comes from other places and there is no possible way to change,
... then this is out of the power of changing. But if content author, expect to make accessible.

Shadi: Question right now. Website has information from 3rd party, e.g. social media buttons, you tube videos, etc. We are asked to evaluate web site. We know it accesses 3rd party content. Are we not required to evaluate all this content? Results are somethin separate

<Vivienne> Yes, Shadi that is what I would see. If it is on the website, it should be evaluated.

Shadi: Or does the 3rd party content change the scope of what is evaluated?

<MartijnHoutepen> agree with vivienne

<Detlev> Sorry I have to leave.

Alistair: We would tell people what they have the ability to change.
... It's their choice of whether or not to change it.

<Vivienne> The website owner needs to know that the content they are hosting is not accessible and then this may change who they obtain products from. Maybe they will then tie it into procurement.

Shadi: Full editor draft out next week with a survey will go out early next week. Survey will be open to WCAG WG, Tools WG and Education WG. We would like to publish a public working draft by year's end unless too many open issues.
... How does everyone feel about that?

<Vivienne> That all sounds good to me. Please go ahead and work on the editor's draft and put it in a survey

Shadi: Editorial changes are okay. We have not really looked at Steps 3 & 4. Are we okay sending it out? If we don't we risk not sending a public working draft this year but that's okay too.

<Mike_Elledge> +1

<MartijnHoutepen> +1

Shadi: Will send out Monday morning for a 24 hour review by Eval Methodology WG first and if no objections we will sent out.

<Vivienne> +1

+1

<Sarah_Swierenga> +1

<Tim> +1

Shadi: Thanks everyone for your hard work on this!