14:53:47 RRSAgent has joined #eval 14:53:47 logging to http://www.w3.org/2013/10/31-eval-irc 14:53:49 RRSAgent, make logs world 14:53:49 Zakim has joined #eval 14:53:51 Zakim, this will be 3825 14:53:51 ok, trackbot; I see WAI_ERTWG(Eval TF)11:00AM scheduled to start in 7 minutes 14:53:52 Meeting: WCAG 2.0 Evaluation Methodology Task Force Teleconference 14:53:52 Date: 31 October 2013 14:54:40 WAI_ERTWG(Eval TF)11:00AM has now started 14:54:48 +[IPcaller] 14:54:55 zakim, IPcaller is me 14:54:55 +Vivienne; got it 14:55:40 MartijnHoutepen has changed the topic to: Eval TF 14:57:07 Detlev has joined #eval 14:57:23 zakim, mute me 14:57:23 sorry, Vivienne, muting is not permitted when only one person is present 14:57:52 agarrison has joined #eval 14:58:09 +MartijnHoutepen 14:59:08 +[IPcaller] 14:59:16 zakim, ipcaller is me 14:59:16 +shadi; got it 14:59:40 +[IPcaller] 15:00:27 zakim, ipcaller is me 15:00:27 +agarrison; got it 15:00:30 zakim, who is on the phone? 15:00:30 On the phone I see Vivienne, MartijnHoutepen, shadi, agarrison 15:01:02 +Mike_Elledge 15:01:24 regrets: Eric, Sarah 15:01:34 Mike_Elledge has joined #eval 15:01:53 agenda+ tep 3.f: Eliminate Redundancies in the Sample (Optional) 15:02:04 agenda 1 = step 3.f: Eliminate Redundancies in the Sample (Optional) 15:02:16 agenda+ Step 4.a: Check web pages for WCAG2.0 conformance requirements 15:02:20 agenda? 15:02:43 I'll try not to fall asleep! 15:03:21 zakim, ack me 15:03:21 I see no one on the speaker queue 15:04:56 zakim, who is on the phone? 15:04:56 On the phone I see Vivienne, MartijnHoutepen, shadi, agarrison, Mike_Elledge 15:06:16 zakim, mute me 15:06:16 MartijnHoutepen should now be muted 15:06:26 scribe: vivienne 15:07:42 Sorry - we have a new ISDN phone, I am trying to dial in, but the phone does not accept the entry of th conference code (or zero for assistence) not sure whether I manage to get in... 15:07:49 SA: apologies for time confusion and lack of formal agenda. Hope to have a new version soon to put into a survey to get feedback 15:09:45 SA: it will be a new editor draft to allow for comments in a survey 15:09:48 richard has joined #eval 15:10:28 SA: today focusing on 3F and 4A, particularly as 4A is still a bit unclear 15:10:58 zakim, take up next 15:10:58 agendum 1. "step 3.f: Eliminate Redundancies in the Sample (Optional)" taken up 15:11:17 http://www.w3.org/WAI/ER/conformance/ED-methodology-20131014#step3f 15:11:32 +[IPcaller] 15:11:48 SA: Step 3F eliminates redundancies and the idea is that we have a fairly elaborate selection process where we select different types of pages and in addition we do a random selection as well. 15:12:20 zakim, ipcaller is richard 15:12:20 +richard; got it 15:12:41 SA: we added in section 3F to eliminate redundancies that might come about - particularly as the random has a likliehood to create redundancies 15:12:44 zakim, mute me 15:12:44 Vivienne should now be muted 15:13:15 q+ 15:13:19 SA: section A & B don't likely involve redundancies, but it is the random selection addition that may introduce redundancies 15:13:38 http://www.w3.org/2013/09/19-eval-minutes.html#item03 15:13:53 SA: Peter suggested 19 September to formulte the sections in such a way that you avoid duplicates from the beginning so you wouldn't need that section. 15:13:53 + +49.403.17.aaaa 15:14:15 SA: would eliminate an extra step, as you would eliminate them as you go along 15:14:18 Zakim, aaaa is Detlev 15:14:18 +Detlev; got it 15:14:35 SA: 19 September there was support for this, and Moe has sent an email as well 15:15:25 ack rich 15:15:32 Richard: from a practical point of view it is easier. If I get duplicates I may see that I've covered the site well. It acts as a double-check, as a true random sample may give me duplicates anyway and it's not a problem 15:15:36 q+ 15:16:11 Richard: this would narrow down what I'm doing. When I do a random sample I know I have randomly sampled the whole site. It does no harm to double-check something. 15:16:53 SA: There is a point here about double-checking. Once you select the random sample in addition to the structured and identify duplicates, you actually throw away the duplicates and re-select new pages for those duplicates. 15:17:13 SA: the question is not to get rid of the concept but to get rid of that section from the document. It won't change the selection procedures. 15:17:21 ack mike 15:18:24 ME: I was confused also. We're talking about eliminating redundancies among all the pages we're viewing not just the random sample. Maybe we should consider at the beginning out for not choosing duplicate pages or processes and have something as a final check to make sure you don't have multiple instances of the same process, feature or page. 15:18:30 http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Oct/0020.html 15:19:10 Tim has joined #eval 15:19:11 SA: also Moe says something along the same lines in her email. There is an aspect of having redundancy and that might be an indication that the website is homogeneous and that can be important to note. 15:19:43 q+ 15:19:52 ack me 15:20:16 http://lists.w3.org/Archives/Public/public-wai-evaltf/2013Oct/0021.html 15:22:11 DF: I'm aware there is no point in the sampling process or the checking process where the evaluator is told that if a page has been selected and is part of a larger process we need to track back or forward to get the entire process, simply because the page you've found is part of that process. If we interpret check all processes meaning that if you check one part of the process you'd have to 15:22:11 check all parts of the process. We cannot for every page/state we've picked that if it's part of a process we need to check every part of that process. You're always dealing with processes of some sort, this is lacking at the moment. 15:22:20 http://www.w3.org/WAI/ER/conformance/ED-methodology-20131014#step3d 15:23:08 SA: isn't that what step 3e does - if you select the page and its part of a random then you also need to include all of the pages inthe processes. Step 3A,b,c, we select pages based on a particular criteria. in Step 3E you do a random selected sample on top of that. 15:24:17 DF: can you expect people in the exploration part of the process that you need to tell them they need to do this. You may have to call up things repeatedly and tht can become time consuming. Does that issue just come up much later when you see tht it's part of the process and then you have to add the other pages/states. You may need to add them while testing and this changes the exploratory 15:24:17 stages. 15:24:37 DF: don't know if everyone would want to go through the process itself - time consuming and laborious 15:24:44 MoeKraft has joined #eval 15:25:03 SA: need to think of what the natural process is. You're saying a lot of the selection happens while there is testing happening. 15:25:20 SA: the tests overlap in a way, but we may need to consider that more when we write the sections 15:26:06 +MoeKraft 15:26:38 q+ 15:26:45 DF: there are different ways of tackling that. You make a selection based on pages, but as you do it it may lead to other pages that need to be tested. You may also need to check the page initiating a search as well as the search results page which would need to be tested. You may not be able to generate a url and it is just generated and you have to list the path. We need to give advice on 15:26:45 that, as it's a very pracitcal issue of sampling. 15:27:20 zakim, mute me 15:27:20 Detlev should now be muted 15:27:21 ack agarrison 15:28:53 Alistair: don't have a problem with taking out 3F. Problem is picking another random sample and there may not be a point in selecting yet another page. 3F becomes rather redundant as you're telling people to do what is done naturally. 3E in the 2nd sentence - 10-15% of the number of pages - is that in the website or the sample. Is that the number of pages you'd select? If there is 20 pages 15:28:53 in the sample you add 2 random. 15:28:54 +Tim_Boland 15:28:58 SA: a minimum of 5 15:29:14 AG: wouldn't make a major impact then 15:30:14 SA: 10-15% is not clear. 3E basic idea is to have a double-check mechanism - structured selection and then a random selection which should show a similar structure to the structured selection 15:30:35 AG: you could say you randomly select 5 pages from the remaining pages of the website and then you can get rid of 3F 15:30:50 +1 15:30:50 SA: that would eliminate this and leave the mechanism to the individual 15:31:23 SA: it seems we want to integrate the content of 3F into primarily 3E but may others as well 15:31:56 q+ 15:32:03 SA: replace duplicates as you go along in each of the sections and it would make the document easier to use in different situations. 15:32:31 SA: Detlev also relates to the testing and things can change while you're testing. 15:32:39 ack mike 15:33:47 ME: it makes sense that we mention in section 3 (introduction) that it is an on-going process of selecting sample and auditing may be rather concurrent activity. It's not only eliminating duplicate pages, but also duplicate processes - expectation that the code that folows of the feature that is presented duplicates a function. 15:34:01 q+ 15:34:27 SA: yes that needs to be explicitly mentioned. We don't want people to waste time testing stuff that's already tested. Depends on the heterogeneity or homogeneity of the website. 15:34:29 ack ag 15:35:20 AG: in terms of coming on a page in the middle of a process, and the requirement of the whole process being in it. If the process is some bit that's included in every page of the site such as search. If you're randomly finding a process and then having to check back. 15:35:55 SA: if you check the search function and the search results, you should be able to assume that it works the same on every single page regardless of where you've decided to search from. You don't need to test the search function on every single page. 15:36:23 q+ 15:36:24 AG: the way it reads at the moment you have to have every page in a process which would mean then every page that has the search box. Wouldn't you just need 1 clear example of the process. 15:37:02 AG: say you come to the page 3/4 of the way along and you have to check every single page in the process you'd have to check every single other page that has the front part of the process in it. 15:37:25 AG: need to clarify that you need to selection 1 path through the website that represents every way the process could be displayed to the users 15:37:49 ack me 15:37:49 Sa: focus on checking the function not necessarily all the pages 15:38:00 q+ 15:39:38 DF: there are different ways of the way processes work where you have to start at page 1 or pages that progressively unfold in the DOM based on your choices. It makes sense to define the start point and then define the input which will cause a certain output. The search result page would also be evaluated. You'd have to explain that you are going to check all 7 pages for a registration process 15:39:39 and it would be different for pages that change. We need to give some guidance, perhaps specific types of processes you could encounter. Otherwise it would be perceived as out-dated. 15:40:11 SA: you mentioned 2 types of different processes - 1 that has a defined start point that you can't jump into the middle, and the other where you can link back from the middle. 15:40:22 Sa: we're now getting into 4A. 15:40:36 ack moe 15:41:02 Moe: no problem integrating 3F into the other stuff, but don't want to lose the optionality. In regression testing there are times you may pick the same pages or procedures, and in that case it's not a bad thing. 15:41:20 Moe: if appropriate remove redundant pages 15:41:37 SA: are you saying it's okay if we pick the same pages 15:42:22 Moe: envisioning the testing more from a user perspective. Making sure all of the pages are clean and then conduct a regression test. I'm not going to hone in on the fact that as an end-user I'm seeing a page twice. 15:42:28 Sa: would you test it again 15:42:36 Moe: in regression test, yes I would 15:42:39 q+ 15:42:47 SA: there are then situatins where you mgiht want to re-test something 15:42:52 ack ag 15:43:10 AG: you're not regression testing to make sure nothing's changed. 15:43:18 AG: they are 2 different things. 15:43:43 Moe: it may be scenario testing. We dn't want to make it so stringent they have to remove redundancies 15:43:47 zakim, take up next 15:43:47 agendum 2. "Step 4.a: Check web pages for WCAG2.0 conformance requirements" taken up [from shadi] 15:44:19 [[Step 4.a: Check for the Broadest Variety of Use Cases]] 15:44:20 SA: the old title for 4A was something like 'broadest use cases possible' 15:44:52 SA: that was the intent at the beginning. In most situations you are not going to be able to check everything. 15:46:00 SA: in checking use cases there is a back and forth between selecting and checking. We need to check for WCAG 2 conformance for each of the pages we check. How many use cases, what kind of use cases. This section is increasingly changing back into check web pages for wcag conformance 15:46:30 SA: we keep getting back to scenarios, use cases, functions. What kind of advice can we give here and how does it relate to step 3. 15:46:54 q+ 15:47:24 ack ag 15:47:28 Sa: when you are doing the testing, where do you start testing the function. You end up on a page tht has a search function. How do you go about testing on a web page including the functins that will take you away from that page 15:48:38 AG: you take the use case and you do each of the actions in terms of the users. After each action you look at the dom and see what changes have occurred. You use the use case as a way to determine how the dom will change with each interaction the user has. It is the dom state that is changing. 15:48:44 so the use case is a particular instance of a process 15:48:55 SA: do you conduct use cases ahead and that's what you check for - to help you. 15:50:01 AG: most of the time people have an idea of how a web page is going to be used. You capture the use case and then you transfer that into a use case person with disability might have. A blind person wouldn't be using a mouse and would be tabbing. By doing those key presses they may come across changes that change the dom. We need to keep the dom in mind for accessibility. All changes have to 15:50:02 be assessed. 15:51:33 q+ 15:51:40 SA: in step 3 the evlauator has selected an initial sample. Not all processes might be apparent yet and there may be some modifications to that sample. I start evaluating and the advice in 4A is start constructing use cases based on visual keyboard users, screen reader users, voice command etc. and construct certain use cases and check using those use cases. While you're doing that and checking 15:51:40 for the use cases you might identify processes or use cases that need to be part of the sample. 15:51:50 AG: any new DOM state needs to be re-assessed. 15:52:02 SA: we call for a different state of a page, and that is a new page 15:52:03 ack me 15:52:06 ack det 15:52:53 q+ 15:54:41 DF: from my experiencemost changes in dom are really activations of buttons etc that are normally the same that will be triggered by keyboard users. Be aware during page samplilng that there may be cases where dom changes occur - holds for all types of users. We can simplify this and tell them to check the relative results of the page, what is the process they are trying to achieve e.g. booking 15:54:41 process. It is more important to define for the tester how you specify those processes - during sampling or later on. Do you record it with the base page, or do you take the process and it becomes more pages for the sample. It has an impact on how you send your customer a quote -for how many pages are you testing. You can have problems because you didn't charge the customer forthat. 15:55:24 SA: the balance is to try to keep it as light as possible (the procedure) but we want to make sure that there aren't too many holes. People will use common sense but others will only try to do the minimum. 15:55:29 ack ag 15:56:29 AG: there were 2 reasons for forcing people to tab through the page. It gives people a chance to see how usable the page it - if they have to tab to the search function. With javascript you can attach events to something like an item in a tab index. You can tell people to follow the use case for amouse, but they won't find that the person using keyboard only can't get to it. 15:56:55 DF: that is part of the discretionof the tester - it's part of SC2.1.1 that you have to get to all actionable actions via the keyboard. 15:57:18 SA: in some instances tabbing through - not only getting to it but also the visual indicator 15:58:25 SA: the conclusion of the discussion is that in section 4A to provide more information - say something about use cases, functionality. Relates back to the selection of pages for the samle and the need to go back and forth and think about use cases, keyboard users and other types of users. 15:58:38 SA: would people agree we need to provide more information? 15:58:42 +1 15:58:47 +1 15:58:52 +1 15:58:54 +1 15:59:06 +1 15:59:11 yes draft something - but don't overdo it... 15:59:17 Sa: need to thnk of how to provide more detail without it become too laborious 15:59:21 q+ 15:59:29 q+ 15:59:33 SA: other aspect is to be clear 15:59:44 ack mike 16:00:10 ME: we mentioned that it's important to understand the different use cases. We can talk about being aware that no only web page types, but the type of use that web pages will receive 16:00:40 Sa: it all ties together - the purpose of a web page is not just picking at random - what does the page actually do and how do the user act with it? 16:00:40 ack ag 16:01:19 Ag: use cases are the only way to drive mobile apps as you can't run tools over the page - the state is changng all the time. 16:01:31 agree with Alistair here! 16:01:35 Sa: more sohisticated applications make it impossible to test every single event 16:01:48 +1 16:01:59 zakim, unmute me 16:01:59 Vivienne should no longer be muted 16:02:32 bye 16:02:34 ack me 16:02:36 bye! 16:02:39 -agarrison 16:02:40 -Mike_Elledge 16:02:40 -MartijnHoutepen 16:02:41 -MoeKraft 16:02:43 bye 16:02:43 -Detlev 16:02:49 -richard 16:03:08 -shadi 16:03:10 -Tim_Boland 16:03:11 -Vivienne 16:03:11 WAI_ERTWG(Eval TF)11:00AM has ended 16:03:11 Attendees were Vivienne, MartijnHoutepen, shadi, agarrison, Mike_Elledge, richard, +49.403.17.aaaa, Detlev, MoeKraft, Tim_Boland 16:03:20 MartijnHoutepen has left #eval 16:06:35 trackbot, end meeting 16:06:35 Zakim, list attendees 16:06:35 sorry, trackbot, I don't know what conference this is 16:06:43 RRSAgent, please draft minutes 16:06:43 I have made the request to generate http://www.w3.org/2013/10/31-eval-minutes.html trackbot 16:06:44 RRSAgent, bye 16:06:44 I see no action items