13:55:44 <RRSAgent> RRSAgent has joined #eval
13:55:44 <RRSAgent> logging to http://www.w3.org/2012/05/31-eval-irc
13:55:46 <trackbot> RRSAgent, make logs world
13:55:46 <Zakim> Zakim has joined #eval
13:55:48 <trackbot> Zakim, this will be 3825
13:55:48 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 5 minutes
13:55:49 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
13:55:49 <trackbot> Date: 31 May 2012
13:57:01 <Detlev> Detlev has joined #eval
13:57:29 <richard> richard has joined #eval
13:57:39 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
13:57:46 <Zakim> +Detlev
13:57:55 <Zakim> +MartijnHoutepen
13:58:13 <Detlev> Zakim, mute me
13:58:13 <Zakim> Detlev should now be muted
13:58:23 <MartijnHoutepen> zakim, mute me
13:58:23 <Zakim> MartijnHoutepen should now be muted
13:58:38 <ericvelleman> ericvelleman has joined #eval
13:58:55 <Zakim> +??P13
13:59:05 <Kathy> Kathy has joined #eval
13:59:10 <richard> p13 is richard
13:59:24 <richard> zakim p13 is richard
13:59:42 <Zakim> +ericvelleman
13:59:43 <Zakim> +Liz
13:59:50 <MartijnHoutepen> zakim, ??P13 is richard
13:59:50 <Zakim> +richard; got it
13:59:50 <Zakim> +Kathy
13:59:59 <Zakim> +Shadi
14:00:00 <Zakim> +??P21
14:00:28 <vivienne> vivienne has joined #eval
14:00:35 <shadi> zakim, ??p21 is Alistair
14:00:35 <Zakim> +Alistair; got it
14:00:59 <Zakim> +??P31
14:01:07 <vivienne> zakim, ??P31 is me
14:01:09 <Zakim> +vivienne; got it
14:01:17 <vivienne> zakim, ??P31 is vivienne
14:01:17 <Zakim> I already had ??P31 as vivienne, vivienne
14:02:13 <shadi> regrets: Kerstin, Moe, Elle, Sarah
14:02:47 <shadi> scribe: Vivienne
14:02:53 <shadi> zakim, mute me
14:02:53 <Zakim> Shadi should now be muted
14:02:54 <vivienne> zakim, mute me
14:02:54 <Zakim> vivienne should now be muted
14:03:11 <Kathy_> Kathy_ has joined #eval
14:03:14 <vivienne> ev: distribution of comments circulated
14:03:15 <Zakim> +??P39
14:03:28 <Kathy_> zakim, mute me
14:03:28 <Zakim> sorry, Kathy_, I do not know which phone connection belongs to you
14:03:29 <shadi> Topic: Processing Comments
14:03:37 <shadi> http://www.w3.org/WAI/ER/conformance/comments
14:04:02 <ericvelleman> q?
14:04:03 <vivienne> zakim, next agendum
14:04:03 <Zakim> I see nothing on the agenda
14:04:12 <shadi> ack me
14:04:13 <Aurelien> yes
14:04:15 <Detlev> yes
14:04:22 <ericvelleman> http://www.w3.org/WAI/ER/conformance/comments
14:04:25 <shadi> zakim, who is on the phone?
14:04:25 <Zakim> On the phone I see Detlev (muted), MartijnHoutepen (muted), richard, ericvelleman, Liz, Kathy, Shadi, Alistair, vivienne (muted), ??P39
14:04:54 <Aurelien> i'm ??P39 I think
14:05:18 <vivienne> ack me
14:05:35 <Kathy_> zakim, ??p39 is me
14:05:35 <Zakim> +Kathy_; got it
14:05:52 <shadi> zakim, Kathy_ is Aurelien
14:05:52 <Zakim> +Aurelien; got it
14:06:12 <shadi> zakim, Kathy is Kathy_
14:06:12 <Zakim> +Kathy_; got it
14:06:23 <Kathy_> zakim, mute me
14:06:23 <Zakim> Kathy_ should now be muted
14:07:11 <vivienne> topic: disposition of comments
14:07:20 <vivienne> zakim, mute me
14:07:20 <Zakim> vivienne should now be muted
14:07:20 <shadi> zakim, mute me
14:07:22 <Zakim> Shadi should now be muted
14:08:11 <vivienne> ev: id5: 1.1, 2.1 - scores better defined - add a note about website parts and clarification of including/excluding parts
14:08:51 <vivienne> also to add a note in 5c a method for aggregating performance scores
14:08:54 <ericvelleman> q?
14:08:59 <Detlev> q*
14:09:00 <shadi> q+
14:09:03 <vivienne> Anyone against?
14:09:03 <Detlev> q+
14:09:06 <shadi> ack me
14:10:21 <vivienne> SA: not seeing anythng about exclusion.  The case is about different portlets and what we can say about the entire portal.  This may need to be addressed in 3.1 or nearby.  Disagree with excluding parts - that is a different discussion
14:10:46 <vivienne> EV: will take out the comment about excluding.  Definition is a bit unclear.  We will add a clarification in 5c on aggregation.
14:10:57 <vivienne> SA: define the performance score first in 5c
14:11:02 <shadi> zakim, mute me
14:11:02 <Zakim> Shadi should now be muted
14:11:04 <richard> q+
14:11:32 <vivienne> EV: add a note about website parts to be explained in 3.1
14:12:00 <ericvelleman> q?
14:12:02 <vivienne> EV: clarify the performance scores in 5c and then how to aggregate
14:12:07 <Detlev> ack me
14:12:12 <Zakim> -richard
14:12:46 <richard> Sorry lost phone
14:13:39 <Zakim> +??P45
14:13:53 <vivienne> Detlev: the issue is that you can single out a part of the site like a library for evaluation, but you can't use the overall university evaluation and exclude library.  Regarding the performance score, we can't make any decision on that until we decide on the score function.  Putting in a note now could be confusing about putting in information about score when there isn't anything there yet.  
14:13:53 <vivienne> We can't say anything about aggregation methods yet.
14:14:11 <shadi> zalom, ??p45 is richard
14:14:16 <shadi> ack richard
14:14:26 <Detlev> Zakim, mute me
14:14:26 <Zakim> Detlev should now be muted
14:14:30 <shadi> q+
14:14:40 <vivienne> Richard: Are we talking about aggregate eg Level A or Level AA, we can't have bits being one and bits being another.  There is no inbetween, so not sure what we mean about aggregating
14:15:02 <Aurelien> maybe we can ask him some more details on his comment
14:15:32 <ericvelleman> q?
14:15:34 <vivienne> EV: Francois' comment is about 3 sections, which is making it confusing.  You cannot aggregate A, AA, AAA, but we'll come back to that in 5c.  We could say if part of a website conform to A and others to AAA, you can't say the website is AAA.  You would say the website is A with parts at AAA
14:15:34 <shadi> ack me
14:16:36 <Zakim> +Mike_Elledge
14:16:52 <ericvelleman> q?
14:17:20 <vivienne> SA: agree with Detlev.  We need to have the algorithm for scoring in 5C that we provide optional, in addition to the Level A, AA, AAA conformance claim.  Before that we can't talk about aggregation till we have pieces. What belongs in 3.1 is the note as Detlev suggests. As to aggregating n the levels basis, we can do that easily by talking about the least common denominator - if one is A and the 
14:17:20 <vivienne> others AA, then the overall accessibility is single A - could add a note that parts are better. But the aggregated value is only A.
14:17:27 <Detlev> agree with Shadi
14:17:28 <Mike_Elledge> Mike_Elledge has joined #eval
14:17:34 <richard> I agree with Shadi
14:17:36 <shadi> zakim, mute me
14:17:36 <Zakim> Shadi should now be muted
14:17:38 <vivienne> SA: 5C is required to be looked at separately
14:18:01 <shadi> q+
14:18:08 <shadi> ack me
14:18:09 <Detlev> fine
14:18:16 <vivienne> EV: in 3.1 we don't have anything about the conformance aggregation - propose then to just clarify the definition of website parts and to make a note to continue discussion on the performance scores later.  Is that okay with everyone?
14:18:35 <vivienne> SA: let's talk about clarification of the website parts separately
14:19:33 <vivienne> SA: propose to add all of this to section 5 
14:20:27 <vivienne> Sa: we can say that aggregation on a performance value is the least common denominator.  With regard to aggregating the performance score we need a note saying we need to address this later.  Not sure where the 2 pieces belong - more in section 5
14:20:37 <vivienne> EV: 5C says option to provide a performance score
14:21:45 <vivienne> EV: need to decide what you want to score when you then want to re-aggregate - hasn't been described yet
14:21:49 <ericvelleman> q?
14:22:58 <vivienne> SA: 1 defines a portal - maybe a note saying that 1.a - if you already know the accessibilit of a website part, then you don't need to re-evaluate that, you can consider that outcome along with your entire evaluation
14:24:04 <vivienne> SA: section 5: need 1 step that talks about the conformance claim itself - 5.8 provide documention for each step and part of that is to provide the actual outcome and in there will be the aggregation on a conformance status.
14:24:14 <richard> The conformance claim for the whole site can only claim the lowest score of any and all parts
14:24:18 <vivienne> SA: there will then be more stuff in 5c
14:24:36 <agarrison> It should be noted that you would only be able to aggregate evaluations which are done within a certain time period 
14:24:55 <ericvelleman> q?
14:24:56 <Tim> Tim has joined #eval
14:25:44 <vivienne> I agree with what Shadi said, along with Alistair's comment - that's good
14:25:56 <Aurelien> +q
14:25:57 <shadi> zakim, mute me
14:25:58 <Zakim> Shadi should now be muted
14:26:08 <shadi> ack Aurelien 
14:26:42 <vivienne> Aurelien: agree with Shadi.  
14:26:53 <vivienne> Sorry, Aurelien, I couldn't understand that - could you clarify
14:27:02 <vivienne> ack me
14:28:00 <Detlev> q+
14:28:07 <Detlev> ack me
14:28:09 <vivienne> EV: if you have looked at website parts that you do not need to re-evaluate them as long as they are within a certain time frame and in step 5 add a short proposed text about how to aggregate the parts.  Start putting in info about the lowest common denominator eg A
14:28:15 <vivienne> zakim, mute me
14:28:15 <Zakim> vivienne should now be muted
14:28:39 <Aurelien> yes that's it shadi
14:28:54 <shadi> q+
14:28:55 <vivienne> Detlev: lowest common denominator - if you have a site which has 15 portlets and you evaluated 14 and they were all AA and 1 was single A, if you aggregated it, that would be single A.  Is that meaningful?
14:29:23 <shadi> ack me
14:29:24 <vivienne> Detlev: you need the detail so you need to work on the one that wasn't AA
14:29:30 <richard> q+
14:29:32 <Detlev> Zakim, mute me
14:29:32 <Zakim> Detlev should now be muted
14:30:29 <Kathy_> q+
14:30:45 <Detlev> q+
14:30:52 <shadi> zakim, mute me
14:30:52 <Zakim> Shadi should now be muted
14:30:53 <vivienne> SA: agree with Detlev - use cases are difficult - same as when we evaluate a website and it 'nearly meets AA on all occasions' but the conformance reallity is that it is A.  The additional documentation allows you to add more information - but we only have the 4 discrete values of conformance defined by WCAG.  The report gives more information, but the performance score provides additional 
14:30:54 <vivienne> information so that they can clean up the last one
14:31:01 <shadi> ack richard
14:31:46 <vivienne> Richard: you need to be really clear about who the performance claim is aimed at - if it's the user then we go with the WCAG - lowest common. If you're looking at the developer's point of view, but it's different - not a conformance claim but a report on the progress towards compliance. Aimed at the owner/developer and not the user.
14:31:47 <Kathy_> ack me
14:32:26 <shadi> +1 to Kathy
14:32:26 <vivienne> Kathy: agree with Richard.  Regarding not having to review a portlet that we'd done in the past within a certain time - but need to add a restriction on whether there was a major change - need to be careful on wording
14:32:33 <vivienne> +1 also
14:32:34 <ericvelleman> q?
14:32:37 <Detlev> ack me
14:32:37 <Kathy_> zakim, mute me
14:32:38 <Zakim> Kathy_ should now be muted
14:32:50 <Mike_Elledge> Mike_Elledge has joined #eval
14:33:34 <vivienne> Detlev: benchmark might be - not aggregated score where you try to put the performance on a level - if conformance has not been reached on that level, then how many of the success criteria have been met - eg say 3 failed and the rest passed.  You could do that for any scope
14:33:48 <ericvelleman> q?
14:33:50 <vivienne> q+
14:34:07 <vivienne> ack me
14:34:31 <vivienne> EV: report on more than the AA or AAA.  This is feasible - maybe add something in section 5
14:34:33 <Detlev> Zakim, mute me
14:34:33 <Zakim> Detlev should now be muted
14:34:46 <shadi> q+
14:35:31 <vivienne> zakim, mute me
14:35:31 <Zakim> vivienne should now be muted
14:35:56 <vivienne> VC: it is good to add information about what has been met for each of the levels - can be encouraging
14:35:57 <agarrison> q+
14:35:59 <shadi> ack me
14:37:00 <Zakim> +Tim_Boland
14:37:11 <vivienne> SA: Canadian Treasury Board's aggregation was circulated.  This was simple and quite effective, and there is an on-going counter.  In addition to the methodology we have resources to develop an evaluator interface.  Similar to "How to Meet..." the idea is to develop this interface.
14:37:23 <vivienne> SA: we could make this more clear in our document
14:37:32 <shadi> zakim, mute me
14:37:32 <Zakim> Shadi should now be muted
14:37:36 <Kathy> Kathy has joined #eval
14:37:37 <ericvelleman> q?
14:37:38 <shadi> ack agarrison 
14:37:48 <Kathy> zakim, Kathy_ is Kathy
14:37:48 <Zakim> +Kathy; got it
14:38:17 <vivienne> Alistair: not convinced that we should be looking at AAA, when we are asked to look at AA.  If people ask for AA, then I stick with AA and not look for AAA, this is too big an increase in time required.
14:38:47 <vivienne> Alistair: you have to add sign language for AAA, that is too expensive for most
14:38:50 <shadi> ack me
14:38:59 <vivienne> EV: currently marked as optional?  
14:39:12 <vivienne> Alistair: don't want to encourage people to expect AAA when we put that in
14:39:14 <shadi> zakim, mute me
14:39:14 <Zakim> Shadi should now be muted
14:39:41 <vivienne> EV: in 3.1c there is a sentence about that it can be useful to do AAA - need to clarify
14:40:15 <richard> zakim, mute me
14:40:15 <Zakim> sorry, richard, I do not know which phone connection belongs to you
14:40:21 <ericvelleman> q?
14:40:26 <Aurelien> no
14:42:10 <vivienne> EV: what do we want to do with the remark by Francois - wants the scores better defined.  We need more discussion about that.  I can write a new proposal on the mailing list and see if we can agree on a sentence to add for the scores n section 5.c
14:42:38 <shadi> ack me
14:42:39 <vivienne> EV: all agree that I make a clarification on website parts?
14:43:56 <vivienne> SA: will write on some actions
14:44:15 <shadi> [Suggested actions: (1) provide a section that is more clear about "determining conformance outcome" (Level A, AA, AAA); (2) add a note in there about aggregating such conformance outcomes from individual website parts; (3) add an editor note to section 5.c about aggregation of individual website parts (for future, after we agree on the actual algorithms); (4) add pointers to these concepts early in the document (eg. step 1.a)]
14:46:10 <ericvelleman> +1
14:46:14 <vivienne> Thanks, Shadi.  Makes my job easier.
14:46:15 <Detlev> fine
14:46:15 <Aurelien> +1
14:46:17 <vivienne> +1 from me
14:46:18 <MartijnHoutepen> +1
14:46:19 <Kathy> +1
14:46:38 <shadi> zakim, mute me
14:46:38 <Zakim> Shadi should now be muted
14:47:38 <shadi> q+
14:47:42 <vivienne> Ev: ID 20 - other audiences - 
14:47:44 <Kathy> zakim, mute me
14:47:44 <Zakim> Kathy was already muted, Kathy
14:47:48 <shadi> ack me
14:48:18 <Detlev> q+
14:48:22 <vivienne> SA: 1.3 says that it is expected that the reader is aware of the resources
14:48:45 <vivienne> EV: could add it just about 1.4 - a short and concise pointing to the business case
14:49:23 <vivienne> SA: seems to be out of scope of the document - should assume they already want to do an evaluation - this is the guidance on how to do it.  Maybe reference this later in the document.
14:49:23 <Detlev> ack me
14:49:25 <shadi> zakim, mute me
14:49:26 <Zakim> Shadi should now be muted
14:50:07 <vivienne> Detlev: Peter was getting at the need for business to progress and making that comparable at different stages.  We can re-write it later and upgrade it later when we know what we can offer for the business side.
14:50:28 <vivienne> EV: it is in the scoring - for comparison
14:50:44 <vivienne> Detlev: if you do the evaluations 3 times, then you have a score you can compare it yourself
14:51:02 <Detlev> Zakim, mute me
14:51:02 <Zakim> Detlev should now be muted
14:51:05 <ericvelleman> q?
14:51:11 <shadi> q+
14:51:14 <shadi> ack me
14:51:29 <shadi> http://lists.w3.org/Archives/Public/public-wcag-em-comments/2012Apr/0001.html
14:53:11 <vivienne> SA: Peter is talking about the progress - not only does it meet/not meet, but how is the support for WCAG improving over time.  This is the performance score section that is still to be developed. We need to finalise that first and talk about how we can motivate and inform developers even if they have minimal support for WCAG 2.  It will become later what we can say at the front.  Need a resolution 
14:53:11 <vivienne> to come back to it later after 5C has been developed.
14:54:09 <vivienne> EV: ID:21 Add definition of "ancillary functionality" we agreed that as a concept it might be useful. But just hold the thought and not currently add it to the document.
14:54:12 <ericvelleman> q?
14:54:18 <Detlev> Fine, can be added later
14:54:27 <vivienne> SA: mark the comment as 'closed' - first one!
14:54:38 <agarrison> I don't know whether allowing people to say they meet 59 out of the 61 checkpoints is encouraging them - or allow a status quo to develop.  If you only talk about A or AA that might be more encouraging...
14:55:03 <vivienne> EV: ID 27: add partial conformance.  It is already inside WCAG 2.0 - could refer to it in WCAG 2.0 - add 3.5b and c 
14:55:11 <Detlev> q+
14:55:16 <Detlev> ack me
14:55:28 <vivienne> Detlev: can you explain this?
14:55:49 <vivienne> EV: in WCAG 2 there is partial conformance for third party and language - just point to the existence of these
14:55:59 <Detlev> Zakim, mute me
14:55:59 <Zakim> Detlev should now be muted
14:56:02 <shadi> http://www.w3.org/TR/WCAG20/#conformance-partial
14:56:11 <shadi> http://www.w3.org/TR/WCAG20/#conformance-partial-lang
14:56:13 <shadi> q+
14:56:18 <vivienne> EV: the clarification would be good - thoughts?
14:56:21 <shadi> ack me
14:58:39 <vivienne> SA: this could be a longer discussion.  There is a bit of a miss-match.  Partial conformance does not mean excluding a part from the evalution.  Part of the comment seems to say 'exclude the ability to remove part of a website' .  It is more about how you write a conformance statement, but it does assume you have done the evaluation.  We'll need more in section 5 about the documentation.  Removing 
14:58:39 <vivienne> part of a website would not allow you to make a statement of partial conformance because you don't know what that missing piece does or does not meet.
14:58:47 <vivienne> Ev: thoughts/
14:58:48 <Aurelien> good for me
14:58:51 <vivienne> +1
14:59:00 <ericvelleman> q?
14:59:01 <Detlev> Vivienne you are a brillant scribe!
14:59:07 <vivienne> TA!
14:59:17 <vivienne> who was just speaking?
14:59:48 <vivienne> Thanks Shadi.  Missed Tim's comment - Tim can you please write in your comment?
15:00:16 <ericvelleman> q?
15:00:19 <vivienne> EV: it is a clarification that you can't leave something out - partial conformance is meant in a different way
15:01:01 <vivienne> EV: will see if we can speed it up by doing part of the discussion on the mailing list and put comments and resolutions in that mailing list.  All please put your comments down for discussion.  We can go through these in the next call.
15:01:03 <Tim> how does partial conformance apply to websites instead of webpages as WCAG defines? 
15:01:07 <shadi> Topic: F2F Meeting
15:02:22 <agarrison> CSUN would be better - given the choice
15:02:57 <vivienne> SA: news about Lyon.  Survey - we have 13 likely attending, 13 can't likely attend.  Some might be able to travel from the US, but also another maybe a CSUN in the US and we would have the opposite figures there.  Would still recommend that 13 people would be a good turnout for a sub-group meeting - a good idea still.  We could do a lot of ironing out over the course of 2 days.  Are there any 
15:02:57 <vivienne> objections?
15:03:03 <vivienne> I'm still going to see if I can get there.
15:03:18 <vivienne> Sa: we will still have some dial-in discussion
15:03:31 <vivienne> SA: any objections to this sub-meeting?
15:03:57 <vivienne> EV: no objections noted - will try to organise it
15:04:08 <vivienne> EV: other issues?
15:04:14 <agarrison> q+
15:04:20 <ericvelleman> q?
15:04:50 <vivienne> Alistair: re techniques being optional?  Can we put this forward at a later date?
15:05:33 <vivienne> Ev: added ID 47 which was a missing comment in the disposition.  So I'll add another one for Alistair's comment.
15:06:06 <vivienne> ack me
15:06:14 <Detlev> bye bye
15:06:17 <vivienne> bye
15:06:17 <Zakim> -Liz
15:06:18 <Mike_Elledge> bye
15:06:19 <Zakim> -Tim_Boland
15:06:21 <Zakim> -Alistair
15:06:22 <Kathy> bye
15:06:23 <vivienne> -vivienne
15:06:24 <MartijnHoutepen> ack me
15:06:25 <vivienne> vivienne has left #eval
15:06:28 <Zakim> -Detlev
15:06:41 <Zakim> -Shadi
15:06:45 <Zakim> -Aurelien
15:06:50 <Zakim> -vivienne
15:06:51 <Zakim> -??P45
15:06:54 <Zakim> -Mike_Elledge
15:06:55 <Zakim> -Kathy
15:06:59 <Zakim> -ericvelleman
15:07:20 <Zakim> -MartijnHoutepen
15:07:21 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
15:07:24 <Zakim> Attendees were Detlev, MartijnHoutepen, ericvelleman, Liz, richard, Shadi, Alistair, vivienne, Aurelien, Mike_Elledge, Tim_Boland, Kathy
15:08:41 <shadi> trackbot, end meeting
15:08:41 <trackbot> Zakim, list attendees
15:08:41 <Zakim> sorry, trackbot, I don't know what conference this is
15:08:49 <trackbot> RRSAgent, please draft minutes
15:08:49 <RRSAgent> I have made the request to generate http://www.w3.org/2012/05/31-eval-minutes.html trackbot
15:08:50 <ericvelleman> ericvelleman has left #eval
15:08:50 <trackbot> RRSAgent, bye
15:08:50 <RRSAgent> I see no action items