13:50:46 <RRSAgent> RRSAgent has joined #eval
13:50:46 <RRSAgent> logging to http://www.w3.org/2013/04/25-eval-irc
13:50:48 <trackbot> RRSAgent, make logs world
13:50:50 <trackbot> Zakim, this will be 3825
13:50:50 <Zakim> ok, trackbot; I see WAI_ERTWG(Eval TF)10:00AM scheduled to start in 10 minutes
13:50:51 <trackbot> Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference
13:50:51 <trackbot> Date: 25 April 2013
13:53:02 <MartijnHoutepen> MartijnHoutepen has joined #eval
13:54:59 <shadi> hi martijn
13:55:15 <shadi> do you know if eric is back yet?
13:57:21 <MartijnHoutepen> yes, he is
13:57:22 <Zakim> WAI_ERTWG(Eval TF)10:00AM has now started
13:57:29 <Zakim> +Shadi
13:57:37 <MartijnHoutepen> dropped him off myself :-)
13:57:51 <shadi> ok, great :)
13:57:58 <shadi> chair: Eric
13:57:59 <Liz> Liz has joined #eval
13:58:32 <Zakim> + +1.301.975.aaaa
13:59:46 <Zakim> +MartijnHoutepen
14:00:06 <Liz> zakim, aaaa is Liz
14:00:06 <Zakim> +Liz; got it
14:00:19 <ericvelleman> ericvelleman has joined #eval
14:00:28 <ericvelleman> Hi there
14:00:29 <Mike_Elledge> Mike_Elledge has joined #eval
14:00:56 <Zakim> +Mike_Elledge
14:00:57 <shadi> regrets: Tim, Moe, Detlev, Alistair
14:01:37 <Kathy> Kathy has joined #eval
14:01:45 <Zakim> +Eric_Velleman
14:02:05 <Zakim> + +1.978.443.aabb
14:02:14 <Kathy> zakim, aabb is me
14:02:14 <Zakim> +Kathy; got it
14:03:18 <Zakim> +Peter_Korn
14:04:33 <MartijnHoutepen> zakim, mute me
14:04:33 <Zakim> MartijnHoutepen should now be muted
14:04:53 <Kathy> scribe: Kathy
14:05:04 <MartijnHoutepen> ack me
14:05:46 <MartijnHoutepen> zakim, please mute me
14:05:46 <Zakim> MartijnHoutepen should now be muted
14:05:52 <MartijnHoutepen> agenda?
14:06:08 <ericvelleman> <http://lists.w3.org/Archives/Public/public-wcag-em-comments/>
14:06:21 <MartijnHoutepen> topic: current comments
14:06:27 <Kathy> Eric:  Welcome.  Not a large agenda.  We are still working on the comments.  We can look at the current state of comments.  Not alot of change yet.  There are 9 comments
14:07:22 <Kathy> Eric:  We can encourage people to submit comments
14:07:46 <richard> richard has joined #eval
14:07:52 <Kathy> Peter:  Another person from Oracle interested in providing comments, we will get this within a week
14:08:36 <Kathy> Topic:  Testing Methodology
14:08:39 <Zakim> +[IPcaller]
14:09:40 <ericvelleman> q?
14:09:43 <richard> zakim, ??ipcaller is Richard
14:09:43 <Zakim> sorry, richard, I do not recognize a party named '??ipcaller'
14:10:00 <Kathy> Eric:  may be some issues with the survey.  Kathy and Tim are having problems with the results being posted.
14:10:23 <MartijnHoutepen> zakim, IPcaller is richard
14:10:23 <Zakim> +richard; got it
14:10:54 <richard> Thanks Martijn
14:11:22 <Kathy> Eric:  We will try to locate the error.  Kathy will send comments to Eric
14:11:25 <MartijnHoutepen> No problem
14:11:35 <ericvelleman> https://www.w3.org/2002/09/wbs/48225/testrun/results
14:11:47 <Vivienne> Vivienne has joined #eval
14:12:17 <Kathy> Eric: What has been your general impression for Steps 1-3
14:12:22 <MartijnHoutepen> q+
14:12:23 <Zakim> +[IPcaller]
14:12:25 <MartijnHoutepen> ack me
14:12:27 <Kathy> q+
14:12:28 <Vivienne> zakim, IPcaller is me
14:12:28 <Zakim> +Vivienne; got it
14:13:23 <Kathy> Martijn: Took a bit more time for defining the scope in the evaluation.  Sample was slightly larger than usual.  May be due to the website selected
14:13:27 <Kathy> ack me
14:13:29 <MartijnHoutepen> zakim, please mute me
14:13:29 <Zakim> MartijnHoutepen should now be muted
14:13:45 <Zakim> +[Oracle]
14:13:51 <Zakim> -Peter_Korn
14:14:03 <korn> korn has joined #eval
14:14:14 <korn> Zakim, who is here?
14:14:14 <Zakim> On the phone I see Shadi, Liz, MartijnHoutepen (muted), Mike_Elledge, Eric_Velleman, Kathy, richard, Vivienne, [Oracle]
14:14:17 <Zakim> On IRC I see korn, Vivienne, richard, Kathy, Mike_Elledge, ericvelleman, Liz, MartijnHoutepen, RRSAgent, Bim, shadi_brb, Zakim, trackbot
14:14:26 <korn> Zakim, Oracle has Peter_Korn
14:14:26 <Zakim> +Peter_Korn; got it
14:16:12 <ericvelleman> Went through it too. Sometimes hare because you have your won methodology. We need to tighten up the language. This is not well for applications. Ok for websites, but sample also lot larger.. Common page elements adds a lot. Last step replace with .. NO: take them out and not replace… three times larger sample than normal or needed. We need more guidance depending on type of review (do you need random sample? Maybe link this to three types of ev[CUT]
14:16:19 <ericvelleman> Says Kathy
14:16:30 <MartijnHoutepen> q?
14:16:33 <MartijnHoutepen> q+
14:16:57 <MartijnHoutepen> ack me
14:17:38 <MartijnHoutepen> zakim, please mute me
14:17:38 <Zakim> MartijnHoutepen should now be muted
14:18:25 <Vivienne> zakim, mute me
14:18:25 <Zakim> Vivienne should now be muted
14:19:14 <Kathy> Liz:  Did not take too much time selecting a random sample.  Looked diagram that Vivienne sent.  Came up with a smaller sample.
14:19:18 <Kathy> q+
14:19:58 <Kathy> Liz:  Guide did not say how much is enough
14:20:04 <Kathy> ack me
14:22:43 <Kathy> Kathy;  May want to look at the language random sample.
14:23:08 <Vivienne> q+
14:25:03 <Vivienne> ack me
14:25:05 <Kathy> Liz:  started with Vivienne's list.  Then selected a list of pages based on the different types of items that were different
14:26:33 <Kathy> Vivienne - the document was based on Step 1 and Step 2.  It was to find out the types of web pages and exploring the website so you can see the different key pages.  Automated scan was run and identified the problem areas
14:28:21 <Kathy> Vivienne - Automated tool gave a starting point for identifying the sample.  Looking at sitemap and key pages will help with Step 3
14:28:39 <Kathy> Vivienne - need more language about how to explore the target website
14:29:25 <Kathy> Eric:  We have the feedback from some people, we should walk though the questions in the survey
14:29:27 <ericvelleman> https://www.w3.org/2002/09/wbs/48225/testrun/results
14:30:12 <Vivienne> zakim, mute me
14:30:12 <Zakim> Vivienne should now be muted
14:30:31 <Kathy> Eric:  Questions are asking for more clarification
14:30:59 <MartijnHoutepen> q+
14:31:15 <MartijnHoutepen> ack me
14:31:29 <Kathy> Eric:  Some questions are not answered
14:31:30 <Vivienne> q+
14:31:55 <Vivienne> ack me
14:32:01 <MartijnHoutepen> zakim, mute me
14:32:01 <Zakim> MartijnHoutepen should now be muted
14:32:02 <Kathy> Martijn: Some questions do not apply to this website.  We need to review other sites
14:32:16 <Kathy> Vivienne:  Will the survey be open
14:32:35 <Kathy> Eric:  Will be open for 2 more weeks.  We will open a new survey for IBM site
14:32:46 <Kathy> Eric:  New website is more of an application
14:33:36 <Vivienne> zakim, mute me
14:33:36 <Zakim> Vivienne should now be muted
14:34:14 <MartijnHoutepen> I believe it was 2
14:34:22 <korn> /IBM/site #2
14:34:42 <korn> s/IBM/site #2
14:35:51 <Kathy> Eric:  We may take a different step from the methodology for the other sites
14:36:14 <Kathy> Eric:  I will send a link around.  We will focus on a different step
14:36:45 <shadi> regrets+ Sarah
14:37:37 <ericvelleman> q?
14:37:37 <Vivienne> q+
14:37:43 <Vivienne> ack me
14:37:59 <Mike_Elledge> q+
14:38:58 <Kathy> Vivienne:  Step 2 is where you explore the website and know what technologies they have used, where they come from, landing areas, use cases with key functionality.  This is where you have to find out about the site.  You need to get the information about the websites
14:39:16 <Kathy> Vivienne:  fair bit that needs to be done in this step
14:39:21 <MartijnHoutepen> q+
14:39:33 <Kathy> Eric:  Is this the output we should have for this step?
14:39:44 <Kathy> Vivienne:  I would have put more information there
14:40:08 <richard> q
14:40:18 <richard> q+
14:40:33 <Kathy> Vivienne:  Add more metrics about the site.  The report should have information about the website
14:40:42 <Vivienne> zakim, mute me
14:40:42 <Zakim> Vivienne should now be muted
14:40:53 <Kathy> Mike:  Do we have to save everything all at once or as we go through
14:41:49 <Kathy> Eric;  Yes, you can go back.  It will be open for 2 weeks
14:42:06 <MartijnHoutepen> ack me
14:43:02 <Kathy> Martijn:  Only thing in step 2 we ask to be reported is the types of technologies used.  We should have more information in step 2.  That way we have  a clear picture of what was done
14:43:49 <Kathy> Martijn:  We should rephrase what we are asking
14:44:02 <MartijnHoutepen> zakim, mute me
14:44:02 <Zakim> MartijnHoutepen should now be muted
14:44:15 <ericvelleman> q?
14:44:48 <Kathy> Richard:  First sentence asks to understand what the site is supposed to do is the most important part
14:45:26 <shadi> q- mike
14:45:28 <Kathy> Richard:  What is that the site does .. the purpose of the site is what we need to define
14:45:31 <shadi> q- richard
14:46:01 <Kathy> Richard: Important for the tester but also those who are reading the report
14:46:13 <Vivienne> q+
14:46:16 <Vivienne> ack me
14:46:17 <Kathy> Eric:  I don't think this is in Step 5
14:46:24 <Vivienne> ack me
14:46:39 <Vivienne> q+
14:46:48 <Kathy> Eric:  we should think about adding.
14:47:00 <Kathy> Richard:  It is in Step 2 but you skip to 2a
14:47:14 <shadi> zakim, who is making noise?
14:47:28 <Zakim> shadi, listening for 11 seconds I heard sound from the following: Vivienne (66%)
14:47:38 <Kathy> Vivienne:  that is why I did the diagrams to understand the website
14:48:02 <Kathy> Eric:  this would be a valuable information for reporting
14:48:12 <shadi> q+
14:48:13 <Kathy> Eric:  also determine the pages
14:48:16 <shadi> q- viv
14:48:59 <Vivienne> zakim, mute me
14:48:59 <Zakim> Vivienne should now be muted
14:49:00 <Kathy> Vivienne:  Need to know this to be able to select the pages.  To create use cases and complete process you need to know what the website is
14:49:09 <Vivienne> zakim, mute me
14:49:09 <Zakim> Vivienne was already muted, Vivienne
14:49:15 <shadi> ack me
14:49:50 <korn> q+
14:49:55 <Kathy> Shadi:  good point, we need to document what the evaluator sees as the main functions.  Do these need to be steps?
14:49:56 <Zakim> -[Oracle]
14:50:03 <ericvelleman> q?
14:50:21 <Zakim> +[Oracle]
14:50:23 <Vivienne> I rather think we have that in Step 2 already, but it could be better explained
14:50:29 <Kathy> Eric:  This is not in section 5, we should add it maybe as optional
14:50:30 <korn> Zakim, Oracle has Peter_Korn
14:50:30 <Zakim> +Peter_Korn; got it
14:51:06 <Kathy> Shadi:  Should this be all in step 5 or as a concrete step in section 2.  As Richard pointed out, it is gets lost at the top of step 2
14:51:51 <Vivienne> q+
14:52:02 <shadi> ack ko
14:52:05 <Kathy> Peter:  What purpose of these new steps?  How does that clearly help the evaluation?  We need be clear on how this helps or supports the purpose of the evaluation
14:52:08 <Vivienne> ack me
14:52:15 <shadi> ack vi
14:52:53 <korn> q+
14:53:25 <Kathy> Vivienne:  we already have this in Step 2 but evaluator needs to understand the site in order to do select the sample.  In particular it is hard to define complete processes
14:53:26 <Vivienne> ack me
14:54:35 <korn> q-
14:54:49 <Kathy> Peter:  I don't disagree with the logic.  We need to describe how this will help.  Maybe hard for those not familar with evaluating websites
14:54:50 <shadi> q+
14:55:08 <Kathy> Peter:  We need to give them enough to be effective
14:55:32 <korn> [And the question is, what is "enough", what is "effective"]
14:56:25 <Kathy> Shadi:  Agree with Peter.  What do we need for those who are not as experience?
14:56:26 <shadi> ack me
14:56:30 <Kathy> q+
14:57:01 <Kathy> Peter:  And the question is, what is "enough", what is "effective"
14:57:34 <Kathy> Peter:  what is the reproducability of the methodology?  How likely are they to come up with the same results
14:58:11 <Kathy> Shadi:  if there is a different sample than the other person, this could be because of what they did during Step 2
14:59:06 <ericvelleman> sorry for not leading this more strict :)
14:59:14 <shadi> Kathy: main thing throughout step 2 and step 3 is clear points without telling people they need to use specific tools
14:59:24 <shadi> ...different ways of achieving these points
14:59:38 <Vivienne> q+
14:59:39 <shadi> ...can work with the website owner, or commissioner or whatever
14:59:56 <shadi> ...should ideally be done collaboratively, not by  the evaluator alone
15:00:12 <shadi> ...information is very important
15:00:24 <shadi> ...but if we require too much information it may not be practical
15:00:40 <shadi> ...may also interfere with privacy and legal requirements
15:01:15 <shadi> ...it is part of what we should be doing, but maybe addressed differently in the reporting
15:01:28 <shadi> ...not about collecting data because may be sensitive
15:01:46 <shadi> ...also can't dictate the way to do this, there many different approaches
15:02:03 <shadi> ...usually best done with input from the website owners
15:02:57 <ericvelleman> How about adding a step specifically for the evaluator to define the main purpose of the website (for the evaluation)..Could be more of a collaborative step.. Shall we prepare this  for a next editor draft?
15:03:04 <Kathy> Eric:  propose to prepare the next step for review in the next editor draft
15:03:06 <MartijnHoutepen> +1
15:03:11 <Kathy> +1
15:03:12 <Vivienne> +1
15:03:18 <Liz> +1
15:03:27 <Mike_Elledge> +1 But perhaps as optional item
15:03:43 <Vivienne> q-
15:03:48 <shadi> +1
15:03:56 <shadi> q- kath
15:04:04 <Zakim> -[Oracle]
15:04:06 <Mike_Elledge> Thx! Bye
15:04:09 <Kathy> Eric:  This finishes the call.  We can pick this up on the next call.  We will keep the WBS survey open.  A new one will be opened for Site #2
15:04:10 <Vivienne> good night all.
15:04:11 <Liz> Bye
15:04:11 <MartijnHoutepen> ack me
15:04:14 <Kathy> thanks bye
15:04:14 <Zakim> -Mike_Elledge
15:04:19 <Zakim> -MartijnHoutepen
15:04:21 <Zakim> -Shadi
15:04:23 <Zakim> -Kathy
15:04:24 <shadi> trackbot, end meeting
15:04:24 <trackbot> Zakim, list attendees
15:04:24 <Zakim> -Liz
15:04:24 <Zakim> -Vivienne
15:04:25 <Zakim> As of this point the attendees have been Shadi, +1.301.975.aaaa, MartijnHoutepen, Liz, Mike_Elledge, Eric_Velleman, +1.978.443.aabb, Kathy, Peter_Korn, richard, Vivienne
15:04:25 <ericvelleman> ericvelleman has left #eval
15:04:28 <Zakim> -richard
15:04:28 <Zakim> -Eric_Velleman
15:04:28 <Zakim> WAI_ERTWG(Eval TF)10:00AM has ended
15:04:28 <Zakim> Attendees were Shadi, +1.301.975.aaaa, MartijnHoutepen, Liz, Mike_Elledge, Eric_Velleman, +1.978.443.aabb, Kathy, Peter_Korn, richard, Vivienne
15:04:32 <trackbot> RRSAgent, please draft minutes
15:04:32 <RRSAgent> I have made the request to generate http://www.w3.org/2013/04/25-eval-minutes.html trackbot
15:04:33 <trackbot> RRSAgent, bye
15:04:33 <RRSAgent> I see no action items