See also: IRC log
<ericvelleman> <http://lists.w3.org/Archives/Public/public-wcag-em-comments/>
Eric: Welcome. Not a large agenda. We are still
working on the comments. We can look at the current state of comments. Not alot
of change yet. There are 9 comments
... We can encourage people to submit comments
Peter: Another person from Oracle interested in providing comments, we will get this within a week
Eric: may be some issues with the survey. Kathy and Tim are having problems with the results being posted.
Eric: We will try to locate the error. Kathy will send comments to Eric
<ericvelleman> https://www.w3.org/2002/09/wbs/48225/testrun/results
Eric: What has been your general impression for Steps 1-3
Martijn: Took a bit more time for defining the scope in the evaluation. Sample was slightly larger than usual. May be due to the website selected
<ericvelleman> Went through it too. Sometimes hare because you have your won methodology. We need to tighten up the language. This is not well for applications. Ok for websites, but sample also lot larger.. Common page elements adds a lot. Last step replace with .. NO: take them out and not replace… three times larger sample than normal or needed. We need more guidance depending on type of review (do you need random sample? Maybe link this to three types of ev[CUT]
<ericvelleman> Says Kathy
Liz: Did not take too much time selecting a
random sample. Looked diagram that Vivienne sent. Came up with a smaller
sample.
... Guide did not say how much is enough
Kathy; May want to look at the language random sample.
Liz: started with Vivienne's list. Then selected a list of pages based on the different types of items that were different
Vivienne - the document was based on Step 1 and Step 2. It was to find out the types of web pages and exploring the website so you can see the different key pages. Automated scan was run and identified the problem areas
Vivienne - Automated tool gave a starting point for identifying the sample. Looking at sitemap and key pages will help with Step 3
Vivienne - need more language about how to explore the target website
Eric: We have the feedback from some people, we should walk though the questions in the survey
<ericvelleman> https://www.w3.org/2002/09/wbs/48225/testrun/results
Eric: Questions are asking for more
clarification
... Some questions are not answered
Martijn: Some questions do not apply to this website. We need to review other sites
Vivienne: Will the survey be open
Eric: Will be open for 2 more weeks. We will open
a new survey for site #2
... New website is more of an application
Eric: We may take a different step from the
methodology for the other sites
... I will send a link around. We will focus on a different step
Vivienne: Step 2 is where you explore the website
and know what technologies they have used, where they come from, landing areas,
use cases with key functionality. This is where you have to find out about the
site. You need to get the information about the websites
... fair bit that needs to be done in this step
Eric: Is this the output we should have for this step?
Vivienne: I would have put more information there
Vivienne: Add more metrics about the site. The report should have information about the website
Mike: Do we have to save everything all at once or as we go through
Eric; Yes, you can go back. It will be open for 2 weeks
Martijn: Only thing in step 2 we ask to be
reported is the types of technologies used. We should have more information in
step 2. That way we have a clear picture of what was done
... We should rephrase what we are asking
Richard: First sentence asks to understand what
the site is supposed to do is the most important part
... What is that the site does .. the purpose of the site is what we need to
define
... Important for the tester but also those who are reading the report
Eric: I don't think this is in Step 5
... we should think about adding.
Richard: It is in Step 2 but you skip to 2a
Vivienne: that is why I did the diagrams to understand the website
Eric: this would be a valuable information for
reporting
... also determine the pages
Vivienne: Need to know this to be able to select the pages. To create use cases and complete process you need to know what the website is
Shadi: good point, we need to document what the evaluator sees as the main functions. Do these need to be steps?
<Vivienne> I rather think we have that in Step 2 already, but it could be better explained
Eric: This is not in section 5, we should add it maybe as optional
Shadi: Should this be all in step 5 or as a concrete step in section 2. As Richard pointed out, it is gets lost at the top of step 2
Peter: What purpose of these new steps? How does that clearly help the evaluation? We need be clear on how this helps or supports the purpose of the evaluation
Vivienne: we already have this in Step 2 but evaluator needs to understand the site in order to do select the sample. In particular it is hard to define complete processes
Peter: I don't disagree with the logic. We need
to describe how this will help. Maybe hard for those not familar with
evaluating websites
... We need to give them enough to be effective
<korn> [And the question is, what is "enough", what is "effective"]
Shadi: Agree with Peter. What do we need for those who are not as experience?
Peter: And the question is, what is "enough",
what is "effective"
... what is the reproducability of the methodology? How likely are they to
come up with the same results
Shadi: if there is a different sample than the other person, this could be because of what they did during Step 2
<ericvelleman> sorry for not leading this more strict :)
<shadi> Kathy: main thing throughout step 2 and step 3 is clear points without telling people they need to use specific tools
<shadi> ...different ways of achieving these points
<shadi> ...can work with the website owner, or commissioner or whatever
<shadi> ...should ideally be done collaboratively, not by the evaluator alone
<shadi> ...information is very important
<shadi> ...but if we require too much information it may not be practical
<shadi> ...may also interfere with privacy and legal requirements
<shadi> ...it is part of what we should be doing, but maybe addressed differently in the reporting
<shadi> ...not about collecting data because may be sensitive
<shadi> ...also can't dictate the way to do this, there many different approaches
<shadi> ...usually best done with input from the website owners
<ericvelleman> How about adding a step specifically for the evaluator to define the main purpose of the website (for the evaluation)..Could be more of a collaborative step.. Shall we prepare this for a next editor draft?
Eric: propose to prepare the next step for review in the next editor draft
<MartijnHoutepen> +1
+1
<Vivienne> +1
<Liz> +1
<Mike_Elledge> +1 But perhaps as optional item
<shadi> +1
Eric: This finishes the call. We can pick this up on the next call. We will keep the WBS survey open. A new one will be opened for Site #2