14:49:44 <RRSAgent> RRSAgent has joined #eval
14:49:44 <RRSAgent> logging to http://www.w3.org/2012/12/06-eval-irc
14:49:46 <trackbot> RRSAgent, make logs world
14:49:46 <Zakim> Zakim has joined #eval
14:49:48 <trackbot> Zakim, this will be 3825
14:49:48 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 11 minutes
14:49:49 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
14:49:49 <trackbot> Date: 06 December 2012
14:56:06 <ericvelleman> ericvelleman has joined #eval
14:56:48 <MartijnHoutepen> MartijnHoutepen has joined #eval
14:58:30 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
14:58:37 <Zakim> + +1.248.342.aaaa
14:59:09 <Vivienne> Vivienne has joined #eval
14:59:27 <Zakim> +[IPcaller]
14:59:33 <shadi> zakim, ipcaller is me
14:59:33 <Zakim> +shadi; got it
14:59:37 <Zakim> + +31.30.239.aabb
14:59:37 <Zakim> +Eric_Velleman
15:00:06 <Zakim> +[IPcaller]
15:00:11 <MartijnHoutepen> Zakim, aabb is me
15:00:11 <Zakim> +MartijnHoutepen; got it
15:00:12 <Vivienne> zakim, IPcallelr is me
15:00:13 <Zakim> sorry, Vivienne, I do not recognize a party named 'IPcallelr'
15:00:25 <Bim> zakim, drop me
15:00:25 <Zakim> sorry, Bim, I do not see a party named 'Bim'
15:00:27 <Vivienne> zakim, IPcaller is me
15:00:28 <Zakim> +Vivienne; got it
15:00:40 <Bim> Bim has left #eval
15:00:46 <MartijnHoutepen> Zakim, please mute me
15:00:47 <Zakim> MartijnHoutepen should now be muted
15:00:54 <Liz> Liz has joined #eval
15:00:54 <Kathy> Kathy has joined #eval
15:01:04 <Vivienne> zakim, mute me
15:01:06 <Zakim> Vivienne should now be muted
15:01:13 <shadi> zakim, aaaa is Mike
15:01:13 <Zakim> +Mike; got it
15:01:40 <Zakim> + +1.301.975.aacc
15:01:42 <Sarah_Swierenga> Sarah_Swierenga has joined #eval
15:01:53 <Zakim> + +1.978.443.aadd
15:02:01 <MoeKraft> MoeKraft has joined #eval
15:02:13 <Liz> Zakim, aacc is Liz
15:02:13 <Zakim> +Liz; got it
15:02:24 <Zakim> +MoeKraft
15:02:26 <Kathy> zakim, aadd is me
15:02:26 <Zakim> +Kathy; got it
15:02:31 <Zakim> + +1.517.432.aaee
15:02:58 <MartijnHoutepen> Peter scribed last time
15:03:41 <shadi> zakim, who is on the call?
15:03:41 <Zakim> On the phone I see Mike, shadi, MartijnHoutepen (muted), Eric_Velleman, Vivienne (muted), Liz, Kathy, MoeKraft, +1.517.432.aaee
15:04:17 <Zakim> +Katie_Haritos-Shea
15:04:23 <Detlev> Detlev has joined #eval
15:04:53 <korn> korn has joined #eval
15:04:58 <Zakim> + +49.404.318.aaff
15:05:18 <Sarah_Swierenga> zakim, aaee is me
15:05:18 <Zakim> +Sarah_Swierenga; got it
15:05:33 <Detlev> Zakim aaff is Detlev
15:05:43 <Detlev> Zakim, aaff is Detlev
15:05:44 <Zakim> +Detlev; got it
15:05:58 <MartijnHoutepen> ack me
15:06:02 <ericvelleman> Scribe: MoeKraft
15:06:05 <MartijnHoutepen> zakim, mute me
15:06:05 <Zakim> MartijnHoutepen should now be muted
15:06:37 <ericvelleman> Topic: Updated Disposition of Comments
15:07:14 <ericvelleman> <http://www.w3.org/WAI/ER/conformance/comments-20120920>
15:07:29 <Detlev> Zakim, mute me
15:07:29 <Zakim> Detlev should now be muted
15:07:38 <shadi> agenda: http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Dec/0003.html
15:08:10 <Zakim> +[Oracle]
15:08:16 <korn> Zakim, Oracle has Peter_Korn
15:08:16 <Zakim> +Peter_Korn; got it
15:08:56 <Detlev> looked at it but not its entirety
15:09:00 <MoeKraft> eric: Please look in document and look for Please Review in column.
15:09:10 <MoeKraft> Eric: Second column is status
15:09:51 <MoeKraft> Eric: Id 8 Please review telco discussion, commenter, location, terms and conditions. Proposed resolution. Rationale.
15:10:18 <korn> q+
15:10:23 <Detlev> Eric, can you post link to the questionnaire?
15:10:29 <MoeKraft> Eric: Please review document let me know if you do not agree with any of them. We have a few more days to look at.
15:10:39 <ericvelleman> <http://www.w3.org/WAI/ER/conformance/comments-20120920>
15:10:49 <ericvelleman> q?
15:10:59 <MoeKraft> Eric: This link is the updated disposition of comments
15:11:17 <MoeKraft> Peter: How do we determine the actual comment
15:11:37 <Ryladog> Ryladog has joined #eval
15:11:52 <Detlev> This is the link to the W3C survey: https://www.w3.org/2002/09/wbs/48225/evaltfq6/
15:12:34 <MoeKraft> Eric: First column id, 3rd column commenter, 4th column link to their comment
15:12:43 <shadi> q+
15:12:52 <MoeKraft> Peter: This is a bit cumbersome while their link may go to several comments.
15:13:20 <MoeKraft> Eric: Places text of comment in the document and proposed changed.
15:13:28 <shadi> q-
15:14:26 <MoeKraft> Peter: Change column regarding Suggested Change and Rationale to Commenter's Suggested Change and Commenter's Rationale
15:14:59 <ericvelleman> q?
15:15:32 <MoeKraft> Eric: Look for "Please review outcome" in column 2
15:15:40 <Detlev> q+
15:15:41 <Vivienne> can we have a couple of days longer?
15:15:47 <Sarah_Swierenga> +1 monday deadline
15:15:47 <ericvelleman> q?
15:15:52 <MoeKraft> Eric: Is Monday, Dec. 10 a good day for deadline to review?
15:15:55 <korn> q-
15:15:57 <Detlev> ack me
15:15:58 <MoeKraft> +1
15:16:17 <MoeKraft> Detlev: Please let us know what to do before Monday?
15:16:56 <MoeKraft> Eric: Just look for "Please review" in column 2. Review Proposed resolution and confirm agreement or disagreement.
15:17:20 <MartijnHoutepen> +1
15:17:22 <Vivienne> that sounds better
15:17:23 <MoeKraft> Detlev: Would like a later deadline.
15:17:32 <MoeKraft> Eric: Deadline, Wednesday, 12/12
15:18:17 <MoeKraft> Eric: Submit feedback to mailing list.
15:18:24 <Detlev> Zakim, mute me
15:18:24 <Zakim> Detlev should now be muted
15:18:33 <Detlev> fine
15:18:46 <MoeKraft> Eric: Let's return to agena
15:18:51 <Zakim> +Tim_Boland
15:20:27 <ericvelleman> q?
15:20:43 <MoeKraft> Eric: Let's finish survey by Wednesday 12/12/12 as well and we will discuss next Telco
15:21:07 <MoeKraft> Eric: Agenda item 3: #3. Extending the Scope to include websites during development Please join into the discussion we started about this on the list: <http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Dec/0001.html>
15:21:25 <shadi> Topic: Extending the Scope to include websites during development
15:21:41 <korn> q
15:21:43 <korn> q+
15:22:54 <Vivienne> q+
15:22:54 <MoeKraft> Peter: While there a 2 separate concepts, one leads to the other.
15:23:39 <Vivienne> ack me
15:23:40 <MoeKraft> Peter: Maybe the case that we should talk about exclusion in two or more cases. Not only in shipping the product. Maybe more comfortable with exclusion in one or other.
15:23:41 <korn> q-
15:24:13 <MoeKraft> Vivienne: Both extend scope. Still like to keep separate. Some people will agree with one and not the other.
15:24:17 <Detlev> agree
15:24:24 <Vivienne> zakim, mute me
15:24:24 <Zakim> Vivienne should now be muted
15:24:33 <korn> q+
15:24:37 <Vivienne> q+
15:24:45 <MoeKraft> Eric: #1 Extending scope to include websites during development
15:25:50 <MoeKraft> Peter: I think this is very important. We want to encourage use of our methodolgy during development as much as possible. Need to make a distinction about making a final report but see no reason why we should not invite developers to use methodology.
15:25:50 <Detlev> fully agree (as you know I would)
15:26:01 <korn> q-
15:26:10 <MoeKraft> Peter: Only need to address situations where one would need to make a public statement.
15:26:23 <shadi> q+
15:26:42 <MoeKraft> Peter: We have no control over how developers use public entities but would be useful to encourage it.
15:27:33 <Vivienne> ack me
15:27:41 <MoeKraft> Peter: We could add text in the introduction that would invite people to not wait until application or site is done before using this methodology. Would not be appropriate to publish final report during development while not ready for that.
15:28:28 <MoeKraft> Vivienne: Difference is intent. Development, test as you go. Cannot make a statement of compliance. Can be used during development to test as you go however a full test is needed at the end.
15:28:39 <ericvelleman> q?
15:29:02 <MoeKraft> Vivienne: Gradual inclusion at the beginning. WCAG supports incremental testing, life cycle from the beginning.
15:29:38 <Detlev> q+
15:29:58 <MoeKraft> Vivienne: Try to not include something that is not satisfactory. Knowing your website is not compliant. Intent is not to get "out of something" but rather to perform testing on an incremental basis.
15:30:04 <Vivienne> zakim, mute me
15:30:04 <Zakim> Vivienne should now be muted
15:30:05 <shadi> q-
15:30:55 <Zakim> + +1.517.353.aagg
15:30:56 <Zakim> -Mike
15:30:58 <MoeKraft> Eric: I agree. More or less what was said by Detlev during last Telco. Made difference between the exclusions. A full compliance statement is not possible during development.
15:32:01 <MoeKraft> Shadi: I feel we are repeating quite a bit that we discussed at face to face. Fully agree with Vivienne and Peter. The keyword is purpose. Looking at work statement and abstract of the document, the main purpose is to evaluate a website that exists.
15:32:13 <MoeKraft> Shadi: Not only make a statement but also deliver a report.
15:32:41 <MoeKraft> Shadi: Could be a website that is in development or a website that is completed. Think of purpose rather than target audience.
15:33:29 <MoeKraft> Shadi: Encourage evaluation throughout development otherwise one will not achieve accessibility. Primary purpose is still to evaluate existing website but does not exclude one from testing during development.
15:33:33 <korn> +1
15:33:36 <Sarah_Swierenga> +1 agree with "purpose" emphasis, rather than target audience
15:33:51 <Vivienne> +1
15:33:53 <shadi> zakim, aagg is Mike
15:33:53 <Zakim> +Mike; got it
15:33:53 <MoeKraft> +1
15:33:55 <Detlev> fine
15:33:57 <MartijnHoutepen> +1 to shadi
15:33:57 <Kathy> +1
15:33:58 <Liz>   +1
15:34:28 <mike_elledge> mike_elledge has joined #eval
15:35:04 <Detlev> ack me
15:35:43 <shadi> q+
15:36:08 <MoeKraft> Shadi: The outcome of the f2f discussion, please review the minutes. We did not have quorum.
15:36:45 <shadi> ack me
15:36:46 <MoeKraft> Martijn: Can use the same process for both phases to be ensure that you have a proper outcome.
15:36:52 <Detlev> Zakim, mute me
15:36:52 <Zakim> Detlev should now be muted
15:37:07 <shadi> s/Martijn/Detlev
15:37:13 <MoeKraft> Correction: Detlev made previous comment.
15:39:03 <MoeKraft> Shadi: If pages are known to be inaccessible, this is a method for sampling pages of an existing website. If you know website in development, you may know that the outcome will ... Don't want to exclude someone who wants to use this in development but do not want to expand scope of document.
15:39:15 <MoeKraft> Shadi: Not sure that would need to be a change in the body.
15:39:23 <MoeKraft> Eric: Let's propose a redraft.
15:39:37 <MoeKraft> Eric: Want to give a bit more time for folks to review the comments.
15:39:59 <korn> q_
15:40:01 <korn> q+
15:40:13 <MoeKraft> Eric: #4. Extending the scope to make exclusions possible Please join into the discussion started about this on the list: <http://lists.w3.org/Archives/Public/public-wai-evaltf/2012Dec/0002.html>
15:40:53 <MoeKraft> Peter: Makes sense if someone performing the evaluation, especially during development, to make a statement that they will limit the scope of their evaluation.
15:41:38 <MoeKraft> Peter: If we know a certain class of failures will happen, it may make sense at the outset to exclude things while it is known that they will fail. It is recognized that people will do it.
15:41:58 <MoeKraft> Peter: Real quesiton: If you do this, are you able to make a conformance claim?
15:42:10 <korn> q-
15:42:11 <Detlev> q+
15:42:18 <Detlev> ack me
15:42:19 <MoeKraft> Peter: Doing exclusions is a matter of practicality. Very useful and normal thing to do.
15:42:36 <MoeKraft> Detlev: Agree with Peter. The real issue involves conformace claim.
15:42:59 <MoeKraft> Detlev: Minimum: on a page level, everything has to comply or conform.
15:43:16 <shadi> q+
15:43:46 <MoeKraft> Detlev: Are cases in limiting certain things, like PDF document. Would need to document that the inaccessible PDF is available. Clearly flagged as inaccessible.
15:43:58 <MoeKraft> Detlev: Need to clearly draw the line.
15:44:19 <MoeKraft> Detlev: Should not raise the impression that everything is fine and conforms.
15:44:23 <Vivienne> q+
15:44:32 <shadi> q-
15:44:36 <Detlev> Zakim, mute me
15:44:36 <Zakim> Detlev should now be muted
15:45:25 <MoeKraft> Shadi: May come to completely different results and this raises confusion. We are providing a standardized procedure for doing that.
15:46:03 <korn> q+
15:46:23 <MoeKraft> Shadi: Don't think we should claim that we follow the procedure unless it is truly followed. Reminder: This document is informative and not normative. Can use the guidance but to make a statement about conformance need to use all parts.
15:46:28 <Vivienne> ack me
15:46:30 <ericvelleman> q?
15:46:50 <MoeKraft> Vivienne: I agree with Shadi. Uncomfortable to making this guidance messy as to what it can be used for.
15:47:32 <MoeKraft> Vivienne: The purpose of this methodology is to evaluate entire websites. While one can use this methodolgy in any way they choose, it is designed to audit a complete website once established.
15:47:51 <ericvelleman> q?
15:48:21 <Kathy> q+
15:48:25 <MoeKraft> Vivienne: Don't want to encourage folks to exclude parts of the websites if they do not comply. Ok to add a statement in the intro that you could use methodology as a guide but cannot make a statement of conformance or compliance.
15:48:30 <Vivienne> zakim, mute me
15:48:30 <Zakim> Vivienne should now be muted
15:49:19 <MoeKraft> Peter: I think real crux of the matter is compliance statement. The world is messy already. When look at single url web applications that extremely complex which has accessibility bugs that are not hit during review pass.
15:49:35 <MoeKraft> Peter: Take slight exception of purpose is to audit. The purpose is to evaluate.
15:50:02 <Vivienne> sorry Peter, poor choice of words with 'audit' meant 'evaluation'
15:50:22 <MoeKraft> Peter: There might be other statements that could be made just short of a compliance statement. Is there other output that one might make other than 100% compliance statement?
15:51:01 <MoeKraft> Peter: Don't have a problem if you are making a compliance claim that you should not exclude anything
15:51:08 <Kathy> ack me
15:51:10 <korn> q-
15:51:17 <MoeKraft> Peter: Purpose of report should have more than just a compliance claim.
15:51:49 <shadi> q+
15:52:07 <shadi> q-
15:52:15 <MoeKraft> Kathy: Majority of accessibility reviews are not on complete products. We review as we go through. Would be good to have methodology apply to development as well. And important to indicate that this cannot be a compliance claim.
15:52:20 <ericvelleman> q?
15:52:23 <shadi> q+
15:52:43 <Kathy> zakim, mute me
15:52:43 <Zakim> Kathy should now be muted
15:52:44 <MoeKraft> Kathy: Majority of accessibility evaluations that happen are not on full products or full websites, especially when evaluating applications.
15:53:04 <ericvelleman> q?
15:53:08 <Detlev> q+
15:53:13 <MoeKraft> Eric: Agree if one wants to make a compliance statement there should not be a possibility of excluding things.
15:53:15 <shadi> q-
15:53:53 <MoeKraft> Shadi: I am interested by Kathy's statement. Maybe there is something that we should consider. Could use methodology in different ways but the question is around conformance statement.
15:54:15 <Detlev> q-
15:54:30 <Kathy> ack me
15:54:38 <MoeKraft> Shadi: When evaluating products during development, how important is it to follow a methodology such as Eval TF? As compared to after website dev is completed?
15:55:20 <MoeKraft> Kathy: The methodology is a good way of saying this is how you go about testing a website or application. Here is how to select sample. This procedure is valid whether looking at subset or entire product.
15:55:59 <MoeKraft> Kathy: Challenge is that you have multiple development teams working on different parts and finish at different times. Ususally you have a contained set of functions per review.
15:56:09 <MoeKraft> Kathy: But still follow the same methodology.
15:56:31 <Detlev> Agree
15:56:39 <MoeKraft> Kathy: Report on all reviews the same way as if doing a full review. We combine results into a much larger review for conformance claim.
15:57:01 <Detlev> q+
15:57:05 <Kathy> zakim, mute me
15:57:05 <Zakim> Kathy should now be muted
15:57:35 <MoeKraft> Eric: Biggest item is conformance claim and whether you can or cannot exclude things. Let's take discussion to the list. When would we want exclusions? What would the compliance statement depend upon?
15:57:45 <Detlev> ack me
15:57:45 <MoeKraft> Eric: Focus on this during the coming week.
15:58:54 <MoeKraft> Detlev: Eric you state, "only point to make is that it is clear for final conformance statement we should not have exclusions". Feel this is still an open question. Still will have inaccessible PDFs etc.
15:59:19 <MoeKraft> Eric: I understand the legacy part also. Maybe we can point to WCAG statement on legacy parts.
15:59:21 <Detlev> zakim mute me
15:59:36 <MoeKraft> Eric: #5. Other issues
15:59:56 <MoeKraft> Eric: If not, wish everyone a good week and encourage discussion around these topics.
16:00:04 <mike_elledge> Thanks! Bye!
16:00:07 <MartijnHoutepen> bye
16:00:07 <Zakim> -[Oracle]
16:00:08 <Vivienne> good night all
16:00:08 <Zakim> -Sarah_Swierenga
16:00:09 <Sarah_Swierenga> bye
16:00:09 <Zakim> -Kathy
16:00:10 <Zakim> -Tim_Boland
16:00:11 <Zakim> -Detlev
16:00:12 <Zakim> -shadi
16:00:13 <Zakim> -Katie_Haritos-Shea
16:00:13 <Zakim> -Eric_Velleman
16:00:13 <MoeKraft> Eric: Plan to launch an edited draft next Wednesday.
16:00:14 <Zakim> -Vivienne
16:00:14 <Zakim> -MartijnHoutepen
16:00:14 <korn> korn has left #eval
16:00:16 <Zakim> -Mike
16:00:17 <Zakim> -Liz
16:00:19 <ericvelleman> ericvelleman has left #eval
16:00:22 <Zakim> -MoeKraft
16:00:23 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
16:00:23 <Zakim> Attendees were +1.248.342.aaaa, shadi, +31.30.239.aabb, Eric_Velleman, MartijnHoutepen, Vivienne, Mike, +1.301.975.aacc, +1.978.443.aadd, Liz, MoeKraft, Kathy, +1.517.432.aaee,
16:00:23 <Zakim> ... Katie_Haritos-Shea, +49.404.318.aaff, Sarah_Swierenga, Detlev, Peter_Korn, Tim_Boland, +1.517.353.aagg
16:00:26 <Liz>  bye
16:00:41 <MartijnHoutepen> MartijnHoutepen has left #eval
16:00:41 <Tim> Tim has joined #eval
16:02:13 <shadi> trackbot, end meeting
16:02:13 <trackbot> Zakim, list attendees
16:02:14 <Zakim> sorry, trackbot, I don't know what conference this is
16:02:21 <trackbot> RRSAgent, please draft minutes
16:02:21 <RRSAgent> I have made the request to generate http://www.w3.org/2012/12/06-eval-minutes.html trackbot
16:02:22 <trackbot> RRSAgent, bye
16:02:22 <RRSAgent> I see no action items