13:40:51 <RRSAgent> RRSAgent has joined #eval
13:40:51 <RRSAgent> logging to http://www.w3.org/2011/08/25-eval-irc
13:40:56 <Zakim> Zakim has joined #eval
13:41:03 <shadi> zakim, this will be eval
13:41:03 <Zakim> ok, shadi; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 19 minutes
13:41:28 <shadi> meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference Meeting
13:41:38 <shadi> chair: Eric Velleman
13:42:18 <shadi> agenda: http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Aug/0039.html
13:43:25 <shadi> agenda+ Welcome
13:43:25 <shadi> agenda+ Minutes of previous meeting
13:43:25 <shadi> agenda+ Recap of the proposed work
13:43:25 <shadi> agenda+ Current best practices
13:43:27 <shadi> agenda+ Discussion on the requirements
13:43:29 <shadi> agenda+ Overview on upcoming schedule
13:43:31 <shadi> agenda+ Any other business
13:43:37 <shadi> regrets+ Leonie
13:55:56 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
13:56:03 <Zakim> +??P1
13:56:05 <shadi> zakim, ? is me
13:56:05 <Zakim> +shadi; got it
13:56:46 <kp> kp has joined #eval
13:56:47 <sds> sds has joined #eval
13:57:39 <Zakim> +dboudreau
13:58:09 <Liz> Liz has joined #eval
13:58:27 <shadi> zakim, dboudreau is really Samuel
13:58:28 <Zakim> +Samuel; got it
13:58:45 <shadi> scribe: shadi
13:58:53 <shadi> zakim, take up agendum 1
13:58:53 <Zakim> agendum 1. "Welcome" taken up [from shadi]
13:59:00 <Zakim> +Liz
13:59:13 <Zakim> + +31.30.239.aaaa
13:59:18 <shadi> zakim, samuel is really sds
13:59:18 <Zakim> +sds; got it
13:59:30 <shadi> zakim, aaaa is Eric
13:59:31 <Zakim> +Eric; got it
14:00:03 <Detlev> Detlev has joined #eval
14:00:29 <eric> eric has joined #eval
14:00:36 <Zakim> +??P27
14:00:37 <Zakim> +Detlev_Fischer
14:01:02 <shadi> zakim, ? is Kerstin
14:01:02 <Zakim> +Kerstin; got it
14:01:35 <vincent> vincent has joined #eval
14:01:39 <Zakim> +Kathy
14:01:41 <Zakim> +??P29
14:01:48 <Zakim> +Tim_Boland
14:01:54 <kerstin> hi all
14:02:02 <Kathy> Kathy has joined #eval
14:02:03 <Zakim> +??P37
14:02:04 <Detlev> Zakim, mute Detlev
14:02:04 <Zakim> Detlev_Fischer should now be muted
14:02:05 <shadi> zakim, ? is Alistair
14:02:05 <Zakim> sorry, shadi, I do not recognize a party named '?'
14:02:14 <shadi> zakim, ??p37 is Alistair
14:02:14 <Zakim> +Alistair; got it
14:02:25 <shadi> zakim, ??p37 is Vivienne
14:02:25 <Zakim> I already had ??P37 as Alistair, shadi
14:02:44 <shadi> zakim, Alistair is really Vivienne
14:02:44 <Zakim> +Vivienne; got it
14:02:51 <shadi> zakim, ??p29 is Alistair
14:02:52 <Zakim> +Alistair; got it
14:03:01 <shadi> zakim, who is on the phone?
14:03:01 <Zakim> On the phone I see shadi, sds, Liz, Eric, Kerstin, Detlev_Fischer (muted), Kathy, Alistair, Tim_Boland, Vivienne
14:03:05 <Zakim> +Katie_Haritos-Shea
14:03:21 <agarrison> agarrison has joined #eval
14:03:40 <Tim> Tim has joined #eval
14:03:44 <sds> zakim, mute sds
14:03:46 <Zakim> sds should now be muted
14:05:34 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Aug/0011.html
14:05:37 <shadi> zakim, take up next
14:05:38 <Zakim> agendum 2. "Minutes of previous meeting" taken up [from shadi]
14:05:54 <shadi> http://www.w3.org/2011/08/18-eval-minutes.html
14:06:09 <shadi> EV: please let me know if you disagree with the minutes
14:06:14 <sds> nothing to change on my side
14:06:19 <shadi> [no objections]
14:06:22 <shadi> zakim, take up next
14:06:22 <Zakim> agendum 2 was just opened, shadi
14:06:28 <shadi> zakim, close agendum 2
14:06:28 <Zakim> agendum 2, Minutes of previous meeting, closed
14:06:29 <Zakim> I see 5 items remaining on the agenda; the next one is
14:06:31 <vincent> Hi, I'm trying to call using a calling card but I have some problems with it.  I'll call you directly finally. 
14:06:31 <Zakim> 3. Recap of the proposed work [from shadi]
14:06:32 <shadi> zakim, take up next
14:06:32 <Zakim> agendum 3. "Recap of the proposed work" taken up [from shadi]
14:06:34 <Ryladog_> Ryladog_ has joined #eval
14:06:43 <shadi> http://www.w3.org/WAI/ER/2011/eval/eval-tf
14:07:16 <Zakim> + +1.514.487.aabb
14:07:28 <shadi> EV: shadi sent around some questions, i just sent around some too
14:07:34 <shadi> zakim, aabb is Vincent
14:07:34 <Zakim> +Vincent; got it
14:07:43 <shadi> ...also found previous requirements
14:08:01 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Aug/0041.html
14:08:13 <Zakim> +Kostas
14:08:17 <Zakim> + +1.662.084.aacc
14:08:21 <shadi> EV: any questions on the Eval TF page?
14:08:29 <shadi> Liz: no questions
14:08:51 <shadi> zakim, aacc is Emanuelle
14:08:51 <Zakim> +Emanuelle; got it
14:08:55 <Ryladog_> Hello All
14:08:56 <kostas> kostas has joined #eval
14:09:13 <Ryladog_> Ryladog is Katie Haritos-Shea
14:10:15 <shadi> SAZ: will work on a group page with the minutes and stuff
14:11:04 <shadi> TB: will be using some form of issues tracking?
14:11:16 <sinarmaya> sinarmaya has joined #eval
14:11:18 <shadi> SAZ: an instance will be provided, will explain in one of the upcoming calls
14:11:53 <shadi> EV: one of the objectives is collecting information
14:12:48 <shadi> ...maybe we can use a wiki?
14:13:02 <shadi> SAZ: need clearer structure for a wiki or it gets difficult to track
14:13:10 <Detlev> Zakim, unmute me
14:13:10 <Zakim> Detlev_Fischer should no longer be muted
14:13:11 <shadi> ...let's talk more offline
14:13:20 <vivienne> okay
14:13:39 <Detlev> Zakim, mute me
14:13:39 <Zakim> Detlev_Fischer should now be muted
14:13:49 <sds> throwing best pratices to mailing list = something like http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Aug/0013.html ? or do we prefer URI to reference documents?
14:14:02 <shadi> EV: will collect information through the mailing list for a start
14:14:16 <shadi> Kostas: different languages ok?
14:14:27 <shadi> EV: probably ok if some of us can understand it
14:14:47 <shadi> Liz: would be good to provide a small paragraph explanation
14:15:04 <shadi> Kerstin: can all the tests be best-practices?
14:15:42 <vivienne> zakim, mute me
14:15:42 <Zakim> Vivienne should now be muted
14:15:52 <shadi> EV: might be good to have survey, studies, and such
14:17:16 <shadi> Kerstin: was just referring to the term "best practices"
14:17:41 <shadi> SAZ: concerned about quality and getting disoriented by tracking external stuff
14:18:03 <shadi> ...better to start by our own requirements and see what existing experiences we can use
14:18:12 <vivienne> that makes sense
14:19:09 <shadi> ...feel uncomfortable starting out with someone elses requirements
14:19:24 <shadi> ...need a better common understanding of what we are looking for
14:20:31 <shadi> EV: work on requirements
14:20:49 <shadi> zakim, take up agendum 5
14:20:49 <Zakim> agendum 5. "Discussion on the requirements" taken up [from shadi]
14:21:12 <shadi> EV: some discussion on the list already
14:21:24 <Detlev> Zakim, unmute me
14:21:24 <Zakim> Detlev_Fischer should no longer be muted
14:21:29 <vivienne> yes
14:21:31 <shadi> http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Aug/0041.html
14:21:43 <shadi> [[Is the "evaluation methodology" expected to be carried out by one person or by a group of more than one persons?]]
14:21:57 <Detlev> q+
14:22:04 <vivienne> q+
14:22:06 <sds> q+
14:22:46 <vivienne> zakim, unmute me
14:22:46 <Zakim> Vivienne should no longer be muted
14:22:51 <Kathy> q+
14:22:54 <vincent> q+
14:23:08 <shadi> EV: number of people probably dependent on their level of expertise
14:23:11 <Ryladog_> q+
14:23:13 <shadi> ack det
14:23:15 <Tim> q+
14:23:42 <shadi> Detlev: maybe have one tester, than aggregate more than that
14:24:03 <shadi> ...or usability testing or other improvements
14:24:07 <shadi> ack viv
14:24:15 <sinarmaya> I think it depends on the scope of the review and the stage where the web is. Stage design, development, publishing, updating. Scope: Self-evaluation, external evaluation, or audit.
14:24:26 <sds> zakim, unmute me
14:24:26 <Zakim> sds should no longer be muted
14:24:32 <shadi> Vivienne: I usually use a group of users, doesn't matter if by one person or a group of people
14:24:32 <vivienne> zakim, mute me
14:24:32 <Zakim> Vivienne should now be muted
14:24:39 <Detlev> Zakim, mute me
14:24:39 <Zakim> Detlev_Fischer should now be muted
14:25:03 <shadi> ack sds
14:25:18 <shadi> Samuel: we do an audit with only 1 or 2 people
14:25:19 <sam> http://alpha.gcwwwtemplates.tbs-sct.ircan.gc.ca/theme-clf2-nsi2/accessRespBreakdown-eng.html
14:25:48 <shadi> ...we do define "roles" though
14:26:14 <shadi> q?
14:26:16 <sam> zakim, mute me
14:26:16 <Zakim> sorry, sam, I do not know which phone connection belongs to you
14:26:23 <sinarmaya> I think that we need define "audit" ;-) For me an audit can't be doing but one person only ...
14:26:24 <shadi> ack Kathy 
14:26:57 <sam> zakim, sds is sam
14:26:57 <Zakim> +sam; got it
14:26:58 <shadi> Kathy: often have multiple people involved in the review, depending on their roles
14:27:02 <sam> zakim, mute me
14:27:02 <Zakim> sam should now be muted
14:27:05 <shadi> ...role-based is goof
14:27:11 <shadi> s/goof/good
14:27:29 <shadi> ...but also important to have one-person mode
14:27:51 <shadi> ack vin
14:28:12 <shadi> Vincent: need to support people with less experience to be able to do some tests
14:28:18 <vivienne> I agree, we can train people to do some of the tests.  We don't have to have experts doing everything.
14:28:40 <shadi> ack Ryladog_ 
14:29:12 <shadi> Katie: not disagreeing but needs to be a test methodology, regarding who is doing it
14:29:21 <shadi> ...has to work for everybody
14:29:23 <sam> we have to make sure to eliminate the negative issues of the "hero" factor i guess. everybody can have a little part of the responsability
14:29:58 <shadi> ...important to have comparability between evaluations
14:30:11 <vivienne> agreed
14:30:16 <shadi> q?
14:30:22 <kerstin> agreed
14:30:33 <shadi> ack tim
14:30:44 <shadi> Tim: agree with everyone else
14:30:59 <shadi> ...the more perspectives, the better
14:31:04 <Zakim> +??P49
14:31:06 <shadi> ...assistive technology use, etc
14:31:24 <shadi> zakim, ??p49 is Richard
14:31:24 <Zakim> +Richard; got it
14:32:11 <shadi> EV: so, at least one person should be able to do it but can be supported by others
14:32:14 <richard> richard has joined #eval
14:32:22 <shadi> ...that person would need some level of expertise
14:32:25 <shadi> [[What is the expected level of expertise (in accessibility, in web technologies etc) of persons carrying out an evaluation?]]
14:32:26 <vivienne> q+
14:32:31 <Detlev> Zakim, unmute me
14:32:31 <Zakim> Detlev_Fischer should no longer be muted
14:32:35 <vivienne> zakim, unmute me
14:32:35 <Zakim> Vivienne should no longer be muted
14:32:36 <Detlev> q+
14:32:40 <sam> q+
14:32:50 <shadi> EV: anyone from the street can carry out an evaluation or do we need some level of requirements?
14:32:53 <shadi> ack viv
14:33:11 <shadi> Vivienne: people have to be trained, and have to know what they are looking for
14:33:38 <shadi> ack det
14:33:40 <vivienne> zakim,unmute me
14:33:40 <Zakim> Vivienne was not muted, vivienne
14:33:50 <sam> Zakim, unmute me
14:33:50 <Zakim> sam should no longer be muted
14:34:05 <shadi> Detlev: often quite good to conduct test be two testers and have them talk about it
14:34:16 <shadi> ...often helps to make better decisions
14:34:17 <agarrison> q+
14:34:31 <shadi> ...to get consensus on results
14:34:50 <vincent> whois
14:34:50 <shadi> ...agree need some level of expertise
14:34:56 <kostas> maybe basic technical expertise of the person could be important in order to understand the results 
14:35:04 <shadi> ...but with technical knowledge can get up to speed quite quickly
14:35:20 <shadi> ...HTML, CSS, and scripting knoweldge
14:35:28 <shadi> ...scripting gets quite tricky
14:35:29 <sinarmaya> I believe that anyone can perform certain tests, but a person who assumes full assessment must meet the specifications of the languages ??or technologies used in the web, know the WCAG, understand the most commonly used technical aids, learn strategies adaptation of the users.
14:35:47 <shadi> ack sam 
14:35:54 <Detlev> Zakim, unmute me
14:35:54 <Zakim> Detlev_Fischer was not muted, Detlev
14:35:57 <kostas> q+
14:36:02 <Detlev> Zakim, mute me
14:36:02 <Zakim> Detlev_Fischer should now be muted
14:36:02 <shadi> Samuel: need some level of technical knowledge
14:36:14 <shadi> ...but need basic understanding of the reality
14:36:32 <shadi> ...so know basic use of browsers, assistive technologies, etc
14:36:44 <sam> zakim, mute me
14:36:44 <Zakim> sam should now be muted
14:36:47 <shadi> q?
14:36:55 <shadi> ack aga
14:36:59 <Ryladog_> q+
14:37:23 <shadi> Alistair: often find that experts have different understanding of the requirements
14:37:33 <Detlev> Different understandings: quite normal, unavoidable
14:37:49 <shadi> ...too much room for the expert than that would influence the results
14:37:58 <Detlev> q+
14:38:16 <Detlev> Zakim, unmute me
14:38:16 <Zakim> Detlev_Fischer should no longer be muted
14:38:25 <shadi> q?
14:38:49 <shadi> EV: what would be the solution to reduce room for interpretation?
14:39:00 <shadi> Alistair: maybe some form of a training
14:39:05 <shadi> ack kos
14:39:22 <shadi> Kostas: some level of understanding of WCAG is important
14:39:31 <shadi> ...additional instructions important too
14:39:39 <vincent> q?
14:39:42 <shadi> ack Ryladog_ 
14:39:53 <vincent> q?
14:40:05 <agarrison> q+
14:40:06 <shadi> Katie: don't want to divide things up between different types of disabilities
14:40:19 <shadi> ...but need knoweldge about the impact of barriers
14:40:36 <vivienne> I agree, some of the guidelines are more specifically helpful to different groups of people
14:40:48 <vincent> msg sam Petit test
14:41:08 <shadi> EV: is this about prioritization?
14:41:22 <shadi> Katie: no, would not want to prioritize based on disability
14:41:47 <shadi> ...want to build for everyone, but want to optimize for specific users too
14:41:59 <shadi> ack det
14:42:20 <shadi> Detlev: would not need to be part of the methodology itself
14:42:38 <shadi> ...could map the results to how it impacts specific groups of people
14:43:30 <shadi> ...in each of every test there can be different assessments
14:43:46 <shadi> ...depending on the context, particularly when aggregating the tests
14:44:20 <Detlev> Zakim, unmute me
14:44:20 <Zakim> Detlev_Fischer was not muted, Detlev
14:44:21 <shadi> EV: aggregation is one of the points that we need to discuss more in-depth
14:44:25 <shadi> ack aga
14:44:30 <Detlev> Zakim, mute me
14:44:30 <Zakim> Detlev_Fischer should now be muted
14:44:57 <vivienne> that's rather where we are now
14:45:22 <shadi> Alistair: if people's expectation is that the methodology leads to 100% certainty about conformance but indeed it is not, then you have strong ramifications
14:45:41 <vivienne> q+
14:45:44 <Detlev> q+
14:45:50 <vivienne> zakim, unmute me
14:45:50 <Zakim> Vivienne was not muted, vivienne
14:45:51 <Detlev> Zakim, unmute me
14:45:52 <Zakim> Detlev_Fischer should no longer be muted
14:45:53 <shadi> ...methodology should not provide information about how to fix issues but just the outcome
14:45:59 <shadi> ack vivienne 
14:46:31 <shadi> Vivienne: often assessments include suggestions for improvement
14:46:43 <shadi> ...something that is helpful
14:46:50 <shadi> ack det
14:46:52 <vivienne> zakim, mute me
14:46:52 <Zakim> Vivienne should now be muted
14:47:28 <Tim> q+
14:47:38 <shadi> Detlev: only theoretical to have 100% compliance, so there has to be some degree of difference
14:47:45 <Detlev> Zakim, mute me
14:47:45 <Zakim> Detlev_Fischer should now be muted
14:47:59 <shadi> ack tim
14:48:10 <vivienne> aagreed
14:48:16 <shadi> Tim: should facilitate education and opportunities to learn about accessibility
14:48:41 <vivienne> I agreed that we should have recommendations, sorry
14:49:08 <agarrison> q+
14:49:17 <shadi> ack a
14:49:33 <vincent> zakim, mute me
14:49:33 <Zakim> Vincent should now be muted
14:49:42 <shadi> Alistair: goal of WCAG2 is to reduce uncertainty
14:50:23 <shadi> EV: will need to address that in the methodology
14:50:35 <shadi> ...how to increase inter-rater reliability
14:50:44 <shadi> [[Is the involvement of people with disabilities a necessary part of carrying out an evaluation versus an improvement of the quality?]]
14:50:52 <vivienne> q+
14:51:00 <Ryladog_> q+
14:51:03 <shadi> ack v
14:51:09 <vivienne> zakim, unmute me
14:51:09 <Zakim> Vivienne should no longer be muted
14:51:34 <Kathy> q+
14:51:36 <shadi> Vivienne: don't think it is essential but very helpful
14:51:51 <shadi> ...not everybody will have access to a group of people to test
14:51:56 <kerstin> q+
14:51:58 <vivienne> zakim, mute me
14:51:58 <Zakim> Vivienne should now be muted
14:51:59 <shadi> ack ry
14:52:01 <sam> q+
14:52:19 <shadi> Katie: it is a recommendation rather than a requirement
14:52:23 <kostas> q+
14:52:35 <shadi> ack kathy
14:52:49 <shadi> Kathy: agree with everyone else
14:53:15 <shadi> ...testing with users with disabilities certainly helps a lot to find additional issues but maybe not a requirements
14:53:16 <agarrison> q+
14:53:30 <shadi> ...but maybe use of assistive technologies is a necessity
14:53:32 <Detlev> +q
14:53:33 <shadi> ack kers
14:53:44 <Detlev> Zakim, unmute me
14:53:44 <Zakim> Detlev_Fischer should no longer be muted
14:53:51 <shadi> Kerstin: depends on the complexity of the subject
14:53:53 <sam> zakim, unmute me
14:53:53 <Zakim> sam should no longer be muted
14:54:04 <shadi> ...may need to involve people with disabilities
14:54:15 <shadi> EV: would need to define "complexity" of websites
14:54:20 <shadi> ack sam
14:54:36 <shadi> zakim, close queue
14:54:38 <Zakim> ok, shadi, the speaker queue is closed
14:55:09 <shadi> Samuel: the way actual people use assistive technology is essential to understand how people use the web
14:55:48 <shadi> ...was important learning exercise
14:55:57 <shadi> ...need to keep up-to-date too
14:56:12 <shadi> EV: needs to be in the methodology or a requirement for the evaluator?
14:56:31 <shadi> Samuel: recommendation but not necessarily an obligation
14:56:38 <Ryladog_> q+
14:56:46 <sam> zakim, mute me
14:56:46 <Zakim> sam should now be muted
14:56:50 <shadi> ack kostas
14:57:15 <shadi> Kostas: involvement of people with disabilities in the evaluation phase may not be helpful
14:57:30 <shadi> ...maybe better to use people with disabilities to validate the methodology
14:57:44 <Ryladog_> q+
14:58:06 <shadi> ...results categorized by people with disabilities quite important
14:58:10 <shadi> ack aga
14:58:17 <Zakim> -Tim_Boland
14:58:28 <kerstin> q+
14:58:35 <shadi> Alistair: the more people you ask, the more opinions you will get
14:59:12 <shadi> ...like the section508 model where you run down the tests then do functional testing too
14:59:16 <shadi> ack det
14:59:25 <Ryladog_> q+
14:59:40 <shadi> Detlev: good for the usability of the methodology to keep it simple
14:59:50 <kostas> q+
15:00:11 <shadi> ...but agree that some complex sites may require involvement of people with disabilities
15:00:59 <vincent> zakim, unmute me
15:00:59 <Zakim> Vincent should no longer be muted
15:00:59 <shadi> EV: out of time
15:01:20 <shadi> ...please continue discussion on the mailing list
15:01:22 <sam> zakim, unmute me
15:01:22 <Zakim> sam should no longer be muted
15:01:25 <vivienne> thanks bye
15:01:29 <sam> by all
15:01:30 <Kathy> bye
15:01:30 <Zakim> -Kostas
15:01:31 <shadi> ...will draft first set of requirements
15:01:31 <sinarmaya> bye
15:01:32 <vivienne> vivienne has left #eval
15:01:32 <Zakim> -Liz
15:01:32 <vincent> Bye
15:01:33 <Zakim> -Alistair
15:01:33 <Zakim> -Detlev_Fischer
15:01:33 <sinarmaya> sinarmaya has left #eval
15:01:34 <Zakim> -Katie_Haritos-Shea
15:01:36 <Zakim> -Kathy
15:01:40 <Zakim> -Emanuelle
15:01:44 <Zakim> -Eric
15:01:46 <Zakim> -Vincent
15:01:49 <Zakim> -Kerstin
15:01:50 <Zakim> -shadi
15:01:54 <Zakim> -Richard
15:02:05 <eric> eric has left #eval
15:02:34 <Zakim> -Vivienne
15:03:53 <shadi> zakim, who is on the phone?
15:04:45 <Zakim> On the phone I see sam
15:05:06 <Zakim> -sam
15:05:10 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
15:05:12 <Zakim> Attendees were shadi, Liz, +31.30.239.aaaa, Eric, Detlev_Fischer, Kerstin, Kathy, Tim_Boland, Vivienne, Alistair, Katie_Haritos-Shea, +1.514.487.aabb, Vincent, Kostas,
15:05:19 <Zakim> ... +1.662.084.aacc, Emanuelle, sam, Richard
15:06:06 <shadi> zakim, bye
15:07:01 <Zakim> Zakim has left #eval
15:07:11 <shadi> rrsagent, make logs world
15:07:16 <shadi> rrsagent, make minutes
15:07:16 <RRSAgent> I have made the request to generate http://www.w3.org/2011/08/25-eval-minutes.html shadi
15:07:18 <shadi> rrsagent, make logs world
15:07:22 <shadi> rrsagent, bye
15:07:22 <RRSAgent> I see no action items