EOWG Home
Page
Sunday, March 26, 2000
9:00 AM - 5:30 PM
LAX Marriott Hotel, Los Angeles, California, USA
Action Item Summary
Attendees
Scope for EO re-charter
Use and promotion of WAI Curricula
Web site review teams
Planning for next face-to-face meeting
Standards coordination
Promotion of WAI materials
User Agent FAQ
Next teleconference
CURRICULUM
Wendy Chisholm, Judy Brewer: point to WCAG Curriculum from more places on the WAI site - e.g. WCAG techniques document and WCAG WG home page.
Judy Brewer: investigate if we can/should get the WCAG Curriculum published as a Note.
Wendy Chisholm volunteers to organize a pre-conference session for CSUN next year.
Chuck Letourneau to check with Laurie Harrison (UofT) for better way of developing on-line curricula.
WEB REVIEW PROCESS:
Action in 3 weeks - Sylvie, Mark, Charles, Gregory: draft a general process to be used to evaluate web sites that considers the following list: @@copy from judy.
STANDARDS:
Due within a week. Fill in the required information about each standard and mail to the list.
George Kersher: OEB, NISO/Daisy
Charles McCathieNevile: WAP, OASIS
Judy Brewer: SEN
Sheela Sethuraman: Instructional Management System
Karl Hebenstreight: NCITS
Tim Noonan: Voice XML
Wendy Chisholm: W3D
Who will take? WEBTV/IPTV
OTHER ACTION ITEMS:
Helle Bjarnø: to provide a pointer to the World Health Organization definitions of disabilities
JB: Judy Brewer (chair)
WC: Wendy Chisholm (part-time recorder)
KH: Karl Hebenstreit, Jr.
KC: Kevin Carey
JH: Julie Howell
CL: Chuck Letourneau (part-time recorder)
HB: Helle Bjarnø
MU: Mark Urban
MRK: Marja-Riitta Koivunen
TN: Tim Noonen
AC: Alan Cantor
GK: George Kersher
MN: Masafume Nakane
SS: Sheela Sethuraman
SD: Sylvie Duchateau
JA: Jim Allen
JT: Jutta Treviranus
CMN: Charles McCathieNevile
GR: Gregory Rosmaita
DD: Daniel Dardailler
JR: Jan Richards
JB: summarized the agenda
JB: The EO group is about to recharter. This has been talked about recently in conference calls but here we want to improve the precision of dependencies with other working groups. When we create a charter we must state the dependencies with other working groups. Does everyone understand the issue of dependencies?
No
JB: when an idea comes up in another working group, who is responsible for handing it? E.g. the concept of a simplified version of the guidelines, should it be EO or WCAG? Probably WCAG but with help from EO. Another example is the review team approach, which is clearly something that is in EO's mandate, but review is also a part of the E&R WG's scope so discussions are needed.
GK: I am a little lost, perhaps because I am not on the EO list. Can you give us an update and review of where we are right now?
JB: almost everyone else in the room has been participating to some degree in EO and some are regular participants. What EOWG has done so far includes: fact sheets for WCAG, ATAG; WCAG Curriculum; WAI Overview Curriculum; Getting Started page; tracking and updating policy issues; revised quicktips// gave out 80,000; coordinated translation of them into multiple languages; brailled address quick tips; updated literature stuffer. Items in progress include: Myth FAQ; Recommended review-by-human process; coordination for review teams; discussion about set-up of model gallery; resource page for Web accessibility contests; how people with disabilities use the Web; business case for accessible Web sites; promotion activities on WCAG curric; standards coordination; discussion w/ WCAG on kinder gentler WCAG guidelines; before/after demo.
JB: lets talk about dependencies with WCAG and E&R
KC: [off topic] what relationship should EO have with whatever the official W3C line is about things? E.g. a piece in the press about the WAI damaging web creation for commercial sites. WAI would use the press to support the W3C guideline, but there is probably already an official press line in W3C and how do we coordinate with that?
JB: there is a careful process in the W3C for press response, but the WAI has a much larger population interested and looking for press response from WAI than from W3C. Did people see the note asking if the media-monitoring thing should be taken off the IG list because there were press listings posted and very long, often heavy, discussions. Do we want separate notated message headers to identify these types of discussions? Judy has heard from some press recently who are sorry for getting things wrong in the past and are trying to get things right.
KC: we should be a resource to the official W3C spokes people and spend less time doing our own rebuttals. The advocacy people can do it well enough and just be an official resource to the press team so we are not a ghettoize group doing our own corrective action.
SS: what is the official policy on press in W3C and WAI?
JB: interesting question. This is how a press reaction works: 1st level, we don't have as much effective media monitoring as Judy would like. Trying to improve that. So we are currently relying on people pinging Judy or Janet and they will prepare a response quickly if an erroneous statement is found (quite a bit lately) and most, when contacted will listen to the facts. W3C has some press clout. On the other hand, W3C doesn't have the only perspective on the issues. Judy is thinking because we have quite limited staff resources and since the issues are reported internationally, it has been effecting to say that some response coming from different organizations (disability, advocacy groups, etc,) maybe we could ask for more collaboration.
KC: you can't really tell advocacy groups what to say, but would just prefer for comments to come from W3C than from WAI as the official spokespersons.
JB: maybe we need information about putting how to deal with the press on the web site.
JB: Back to dependencies with WCAG -
WC: in general all of the other groups are working with different target audiences. Wondering if there was some way to borrow strategies from other groups. How is the audience defined for this group? Her group has 30 audiences and that is only a subset of what EO might have. What are the people in this group's skills that might make it easier to develop dependency?
JB: there are no constraints on EO's audience. The importance of any community may change as our priorities emerge, but all are targets. We have done much around WCAG to date, but the new guidelines are just out and we are moving in those areas.
AC: who is it we are addressing? He would like to see Wendy's list to get a handle on the kinds of communities. Thinks it would be interesting.
JB: Should WCAG or EO be responsible for simplified guidelines?
TN: doesn't recall what Monday (WCAG) meeting decided about the simplified guidelines. There was some discussion about a briefer version of the WCAG that would be the official document. His sense is that although the WCAG group has changed somewhat recently, reading the existing guidelines as a relatively non-technical person, he agrees that WCAG needs the help of EO to make a user friendlier version. WCAG is very good at the technical stuff, and there should be definite crossover from EO to help.
GK: I understand that we are trying to reach a limitless audience, but we must be strategic. E.g. the AU group reaches a significantly smaller audience than the content author group, which itself could be sliced in many different ways.
AC: If the people who produce the guidelines thinks they have the ability to produce the versions then he doesn't think we should second guess them. But it would be good to have a review group to "translate" the document with subsequent re-review by WCAG.
JB: it still may be useful to figure out "what kind of help would your WG want from us"
JH: one main target group (that isn't being served well) is the visually impaired developers themselves. There is no demand in the UK from blind developers. We do get a lot of press about accessibility, but the blind community themselves are not fully cognizant of the issues because the guidelines are too difficult to explain for the lay person.
MRK: we must really know about the target group before we can decide how to make them "better"
JB: lets get the thirty audiences from the minutes of the WCAG minutes.
From CSUN 2000 WCAG face-to-face meeting notes [ http://www.w3.org/WAI/GL/meetings/20000320 ]
- Specification-reading developers,
- Developers,
- Designers,
- Testers
- Policy design and development
- Interpreters - people like EO.
- Innovators and integrators (those not of the choir)
- Government departments who commission accessible sites
- Technically competent lawyers
- Expert witnesses
- Educators and authors
- Managers/executives
- Activists
JB: But for EO the audiences include: designers of sites, managers of design teams, information technology policy enforcers, software developers (browser , and multimedia developers) authoring tool develops, persons with disabilities.
AC: do we need to break down audience by the three sets of guidelines? We need to deal with them differently.
KC: it may be slightly different in the US, but in Europe the people we need to advise are politician, standard setters and policy makers. That is where we are being asked to help and we need solid overview material not technical stuff, which can be added as appendices.
JB: one way to handle this would be to add another bullet point to the project list that indicated the audiences. This might help build the "portal" version of the WAI web site that will help people find what they need.
HB: re Kevin's comment, in Denmark we have the library community. Also on the Web site get an idea of how the different guidelines relate to one another. I.e. more general information about the different products.
JB: I don't understand how that helps.
HB: it isn't clear that you don't have to read everything that is there that is necessary for, say, web accessibility design. I.e. the site needs better structuring and internal references.
MU: I am a policy maker who is also a systems engineer, so I understand both the guidelines and how the policy and industry people look at this stuff and don't have time to make major reviews. Policy makers often make decisions on 5-mingute conversations. So targeted information at that high level is probably key.
JB: lets note that as a priority but get back to the actual topic about dependencies.
WC: comment on audience. Likes the definition of audience for each product. But it must also include researchers, clinicians, faculty resource people, language developers (all the xml dialects). EO's clients are really a superset of all the other working groups.
SS: other audiences- web trainers, education technology specialists
AC: can we learn more about our audience by looking at the guidelines that others are producing... for whom are they putting up their guidelines. E.g project EASI.
EASI sees us a "geeks" and they want something in plain language rather than as it is currently framed.
JB: so it sounds like EASI people should be recruited into helping.
AC: or just use them as research targets to find out why.
JB: thinks the issue of plain language will continue to come up.
TN: one of his employers is a blindness company, the audience that would be relevant are the high level technical people but also the marketing people (non-technical) want to promote accessibility but haven't enough low-level stuff to send out.
GK: in thinking about trainers - people who teach courses on web page design and access would have to put together his or her own curriculum to teach the course, drawing heavily on the guidelines, but they would have to chunk it up to deliver it to other people. What would I have to put together?
JB: would suggest downloading the most recent version of the WCAG curriculum. It is done, ready and available. It has guidelines, checkpoints and illustrations of those checkpoints and thinks it is an excellent resource.
GK: has it been taught anywhere yet?
JB: lots of places.
JB: but back to dependencies
WC: part of reason its so obvious for WCAG because it has been out there for a year and we have lots of comments. Until the others have been out there for a while we may not really know the audiences. We need usability testing that will help us anticipate the user needs for the other working groups.
MU: sees another dependent to the authoring tool group. Once those are in place, EO should promote those tools that are working towards implementing them. Lets promote them as being good participants. The tools are key to future use of the WCAG.
KH: Echo Mark's comments. These are the strategic leverage for, also the Research Group.
HB: AU and UA and tools, we should be reviewing the lists and matrices that describe browsers, ER tools, etc, and ensure that these tools are useable by the audience.
MRK: more about feedback: how are we really gathering feedback. We don't have linked for users to provide information back to us.
SS: on dependencies. We know issues for WCAG. We will find the same thing happening once the others are really out there. We may not have the same issues, but is sure that for UA we will have a language-level problem as well. We know the FAQ. Identify primary needs for each primary audience category (cluster) and reshape some of the common information for all. Learn from the experience of WCAG and all the dependencies we are finding that apply to the new guidelines. E.g. FAQ and feedback and usability of the guidelines and then maybe do the simpler versions of the other guidelines. EO has a critical role to play. There are curriculum modules created for WCAG: is there something relevant like that for other guidelines? Reviewing the current status of the tools. E.g. the ER group has a list of tools and how accessible they are, etc. does EO have a role to play in this sort of list? E.g. current status of technologies is out there.
MRK: maybe we could have a feedback form for the material
JB: Avenues to promotion? Usage opportunities that we could drive? Various countries? Also, online WAI overview. not packaged for off-line use. similar to what I do for an overview but w/out my commentary. Are there priority areas for new developments? ATAG is out, do we want to take some opportunities to develop curriculum around that. Key design requirements for maximum usability of materials. e.g. something in the tool area. Len Kasday has done good reviews and demo usage of tools, should we try to capture that? Ways to design curriculum? We ended up with something that is very comprehensive. We decided not to do speaker notes and put the info on the slides. It therefore might be more suited for self-study versus classroom.
CL: we get about one comment per week on people using the curriculum. it has been praised for content. About 1 of every 5 says, "it is not really suited to stand-up delivery." It is most easily used for self-study. We might want to repackage those so that they fit on slides and use in front of the classroom.
KH: within the GSA there are 4 parts of the agency. there are professional services available. a number of these are training services. to get the curriculum integrated into the web curriculum is a strategy we'll be working on. Greg Gay's presentation on process awareness opens opportunities to organize the courses.
MU: we need to use it to develop a set of reviewers who can review sites. It would be worthwhile to develop a performance tool to determine how well someone has understood the curriculum.
JB: I would like to encourage us not to think about new things we could develop, but how to promote what we do have. The amount of things that come up is infinite. We have things that need to get out there and be used. Please, continue.
MU: The curriculum is an excellent resource, however in terms of promotion and usability, we need to call on people that we know understand it. Therefore, i'm looking for materials that we can use immediately and to know who to call on. If i want someone to create Visual Basic, I hire someone who is certified. Therefore, if someone has gone through the curriculum then we ought to be able to certify so that we can call on them.
MK: an important part of promotion is translation. current w3c policy does not recognize official translations. it is important for us to be clear about the policy that we will use to translate our materials. Perhaps we could even change the w3c process.
JB: specific suggestion?
MK: it ought to be in a language that we can review. then we can review and give approval and say that it has been reviewed by EO group. Then we can put it in our site.
JB: EO reviews particular translations? But there are not multiple people that speak any given language?
MN: some sort of approval will help so that people can determine which version of translation they want to do.
MK: I point to the curriculum. How well do we point to them in our site? If someone goes to TR can they easily get to the curriculum?
JB: top of WAI home page. Not on TR since not official note.
MK: but that's how people get to the guidelines. can we point to them from the techniques? from the WCAG WG home page?
Action WC, JB: point to WCAG Curriculum from more places on the site - techniques document and WCAG WG home page.
JB only way to get them on TR is to publish as a Note.
Action JB: investigate if we can get the WCAG Curriculum published as a Note.
SD: the movie is very helpful. once they have seen the video they are happy to read the guidelines. what's happening with the video? is it mentioned in the curriculum? Distribution in other countries?
JB: there is captioning and description that has to happen. the production was more intensive that we thought. we're waiting for the redesign of the jacket. the EU collision reprimanded us since they were not properly acknowledged. Helen Petrie and others are working on redraft of jacket. I can't do anything for promotion and distribution until those are addressed. Continued promotion elsewhere, jeopardizes our funding from EU.
GK: could the video be made into a SMIL presentation?
JB: we have an agreement with NCAM to do that. waiting for wrap-up of other issues.
GK: then it could be on the SMIL test suite to ensure that all of the SMIL players will play it.
AC: what is the process for bringing things into translation. Translations are vital. Anything to speed up the process to the major languages is an important consideration. If it is a horrible process is there a way to streamline it? What about additional curricula related to WCAG? It may be too early to determine. People need to use what is available and it is not meeting their needs. If I were going to teach a class, I would take them and modify them. I don't think we can anticipate every person's needs. There may need to be a "geek-free" version. But, for the moment perhaps sit back and let people let us know what they need.
JB: explicitly defined process to translate W3C documents. It primarily addresses issues that arise around a translated specification is normative or not. The essence is that the W3C has only one version and language that is normative, due to subtleties with translations. Some organizations (W3C members) have wanted to submit translations through the W3C process, but it seems that it will be problematic. Curriculum would not have to be so strict.
AC: could we apply the same method to translating curriculum that we used to translate quicktips?
JB: yes, we often had 2 or 3 people independently translating then bringing translations together. therefore, we had a lot of resources. the major thing we looked at is outdated and offensive language regarding disabilities. we missed it on a couple of them. it's hard if you don't know what the options are. we will use the same process with the WAI flyers. To translate the curriculum will be a major project since there is so much there.
SS: promotion - post to interest groups. not just web accessibility, but web design. do we have a process to do that? preconference workshops also another avenue. although, if self-study then perhaps not. definitely we need to streamline the feedback process. chuck and geoff may be gathering, but it ought to be collected, and categorized. it would be good for all of our work. Re: use of our materials, use the amazon.com approach to get reader reviews.
KC: if an authority legislates, then it needs to provide the means to the end. If several governments are legislating, then they ought to pay what they are requiring.
JB: we are attempting to make all of our materials online so that they are free. I see a conflict there.
HB: re: translation: i had thought about translating but it is too big. we have a danish set of recommendations that relate to WCAG. if you want details go to W3C originals.
JB preconference workshops seem to be an important thing to address. internally, we are concerned with the accuracy of the material that is getting out. Sometimes I see this on our own lists, e.g. our guidelines prevent the use of tables. Therefore, it might be useful to have a concerted effort to ensure that there are preconference sessions that are presented by people who are involved in the process. perhaps a workshop of the common presenters to get them on the same page. Therefore concerted effort to increase the quality and reduce misinformation.
AC: re: preconference workshops: perhaps we need to decide *now* who will commit to CTG, CSUN, and RESNA. These will be decided upon in the fall.
/* WC volunteers to organize preconference CSUN next year. */
JB: actually, those conversations are going on now.
AC: we have to decide right away then that we will be doing this.
re: language: translation and enormous task. We ought to prioritize the languages. Japanese may be a higher priority than Danish. Therefore, target countries where English is less commonly used.
TN: we need develop materials that point people to the key ideas - conference stuffers. for example, all of the materials on a disk. A cd-rom stuffer. There is a huge interest for 1 or 2 day courses on web accessibility.
GK: I agree, 1 or 2 day course materials could be helpful. A question on how this relates to educators is that to teach a course you will probably pick an authoring tool. You might want to associate the authoring tool with that course. Get in touch with companies that do training on their tool. E.g. Microsoft FrontPage or HoTMetaL. You might leverage those company efforts to incorporate our materials into their curricula. Likewise, get educators to incorporate our materials into their courses.
WC: yes, that's one of the most common questions i hear. It's part of ATAG and we ought to promote that.
JB: we're industry-neutral, but we can leverage the corporate relationships that we do have. this is important because it gets beyond the edges of the choir. Are there ways to leverage usability design courses?
MK: thinking that when XLINK comes out we could ask the companies to use this. you can have the links separate from the document.
JB: do other people know XLINK? i'm not sure how it works.
MK: the starting point and ending point in a document can be separate.
JB: you can point to a specific section of a document.
KH: concerned that we will look like we are endorsing certain tools.
TN: we can use inclusion of tools in curricula as incentive for tools to increase conformance to ATAG. we should talk with AU and tell them that they should mention to tool developers that if their tools are accessible they are more likely to do ATAG. Part of the business case.
JB: it sounds like another deliverable to me.
MU: re: university curricula: it is important that we do outreach w/the universities. Specifically community colleges and non-traditional education systems (web authoring trainers, private companies).
WC: authoring tool - reviewing tools, techniques document, it's like an "easy read version" how to apply WCAG. Almost 3-way coord needed between WCAG, AU, and EO.
JB /* summarizes key points @@get list from judy */ Note that I don't want to replace other presenters but increase the accuracy.
AC: If WC or someone else produces a curriculum that is approved by the group, is it then a deliverable?
JB we try to take things developed by staff that are developed quickly and clean them up. this is a good group for review. the EO has a better ability to take something developed by a staff and evolve. But it doesn't automatically mean that work by staff becomes a deliverable.
AC: perhaps we create a course, and then certify others to use it.
JB: certification could be a possibility. However, I am concerned that we often need to manage expectations. Resources are limited. W3C is interested in certification across groups and materials. This probably a year or more down the line. Other comments?
SS: to promote within universities: target conferences and mailing lists. Future needs: good to record Geoff and Chuck's experiences so we don't repeat their mistakes.
CL: Any future curriculum should avoid the W3C slidemaker tool. It works well for small slide shows (fewer than 8 tools) but not for greater than 200. Therefore, we may want to use a database and generate slides and links.
KH: I attended EduCause last year. there was only one panel that discussed accessibility issues. We have submitted for a session this year.
JB what about tools to use?
WC: we use a fair number of scripting for other guidelines.
JB yes, but are there commercial tools?
Action CL: check with Laurie Harrison on the state of the art.
JB: someone from a review team, please discuss the steps you went through for reviewing sites. We want to draft first round of steps. Need to work with WCAG WG and ER WG.
HB: it was slow. based on Bobby tests. screen reader (jaws). magnifier (zoomtext). didn't have time to do review via the checklist. Therefore, have not yet said, "WAI compliant" yet. over the next year will set up guidelines for reviewing that use the wai guidelines.
TN: it is common for people to do that type of brief audit, to get that general sense of the accessibility. we did that in australia as well. used bobby to get gut feel then users to verify. not conclusive results. but that is often only way to go with large volume.
[Lunch]
The Government of Canada, in its continuing effort to make its electronic information and on-line services accessible to all Canadians, has just concluded a pilot project operating a "Web site Accessibility Testing Service". The service is sponsored by the Chief Information Officer's Branch of the Treasury Board Secretariat. Government of Canada Webmasters or design teams can make an appointment to watch as persons using enabling technologies experience their designs under real-world conditions. Also on hand is an expert in accessible Web site design to advise the client on how to make their designs conform to the Treasury Board and W3C's guidelines for universal accessibility. Since the testing center is fully equipped as a teleconference site, this service has the potential to be rolled out across the country. During the pilot project, four departmental website designs have been evaluated. The feedback from the participating webmasters has been very positive. They now feel that they understand why the accessibility requirements are being demanded of them. For many, it is the first time they have seen assistive technology in use and feel the entire process is very enlightening. Seeing it applied to their Web site designs is especially interesting. There is a strong indication that the pilot project will continue to be funded by the Treasury Board for the upcoming Government of Canada fiscal year.
Process:
There was some discussion of the level of implementation of Web Content Accessibility Guidelines in Canada. Chuck indicated that all new web design and current re-design projects are following the GoC "Common Look and Feel specifications". This draft Treasury Board Internet policy requires Double-A compliance with the WCAG 1.0 recommendation.
TN: are you offering an endorsement service? are you charging for the service?
CL: Evaluation - yes, endorsement - no. We tell them what they have or have not done correctly. You do not get a full, detailed evaluation. It is a very top-level discussion about what is observable in about an hour. No charge for this service. Canada wants all governmental services online by 2004. As part of this commitment, the center has been set up.
AC: I was in Ottawa 2 weeks ago and witnessed it first-hand. it was one of the most powerful presentations I had ever seen.
JB: I am amazed at the packaging. It is very professional. The front-end and back-end: request come in, they get a request, expectations are set. I still don't understand what happens in the 2 hours: is there a consistent sequence of events? How many sites have you done? How many hours per site?
CL: in the pilot we have done 4 sites to the depth that is possible in the time available (1.5 hour). Pre-site evaluation (survey) that identifies potential issues, such as frames, allows us to focus on those pages. We look at general navigation issues and general issues with a particular assistive technology.
AC: speech recognition to get to the address line took 6 minutes. But not a very sophisticated user.
JB did you use any evaluation tools or was it pure usability?
CL: Pure usability.
JB: no checklist?
CL: That's in the package of material we give them. They know they must adhere to the guidelines. We don't have to give them a complete checklist because they know what and why they need to do. What is not in place is a follow-up to see if they have actually worked to make the site accessible.
JB: So the service is a convincer, not a validator.
CL: Right. The report is a summary of the comments to give them something to remember. Something to show to bosses. Rresources: less than 50K Canadian for salaries, computers, etc. This is one afternoon per week. Sometimes we can do 2 sites in one afternoon. If the government wants more we will have to negotiate that. 50K is minimum.
Context: UK government published first set of guidelines in December. All local and federal government must be accessible to persons with disabilities. These do not now reference WAI but will soon. Better-connected Society of IT management a responsible for ICT policy. Last year produce "Well Connected", a report on the accessibility of sites, concentrating now on access for people with visual disability. That shifted the list of accessible sites.
Julie looked at 58 sites (?) These were the criteria she used (followed by reasons for why this is important in the project description)
Concentrated on the accessibility of the home page. IF bobby didn't pass the home page then didn't; go further. We are promoting good design (e.g. use markup correctly). Marked out of thirty. IF they passed all, get a 3. If most things were covered, would get a 1. Ranged from 2 - 28 final scores. For political reasons they released a nasty report that stung a lot of departments. Extremely useful way to get designers aware.
JB: looks like doing some negative press in this fashion is not a bad idea although WAI might not be the best group to do it. If we could find some credible review team to do it.
JH: did something similar for commercial sites and they mostly failed miserably. But TESCO will soon be very accessible for on-line food shopping.
JB: to clarify steps: review page against criteria, focus on home page, score against each criterion.
JL: using Jaws, PWWebspeak and with partially sighted person. Time spent is no more than half and hour on each site. Probably three people with different abilities reviewing a site and then combine and compare reports.
DD: re: not going beyond home page, why not? Do you assume that the rest of the site is not accessible?
JH: if the home page is not accessible, some people may not go further period, so it is a reasonable choice.
AC: would like to add a criterion to Julie Howell's list: would be ease of keyboard navigation, especially of image maps.
JB: interested to see how different the evaluation processes are.
CL: Government of Canada methodology will be on line, but not the evaluations themselves.
The University of Toronto (ATRC) has a formal cost recovery process: the Web Savvy Service. The evaluations are multistage: a quick review highlighting the obvious difficulties, which enables them to make a detailed contract for formal review. We do a full review which takes up to 5 weeks. We provide full report. Do a human pass of following the WAI guidelines. We identify which changes are needed to meet checkpoint. Then do a usability review (e.g. cognitive and usability). Then provide sample code for some areas. While they are making the changes, ATRC consults on an ongoing basis, and when the changes are done, Web Savvy gives a seal of approval that the client can display. A second person goes through to check on the evaluation of the evaluator. Also looks at it with different AT technologies etc. to look at usability.
CMN: have any met triple A?
JT: no although one has committed to do so.
CMN: does someone just look at the site for general visual and cognitive usability?
JT: yes.
TN: any idea of the total number of person hours to do a site?
JT: not easy since it varies immensely from site to site. Doesn't think have any have had more than a day. Amount of work may be reduced by the number of template based pages on a site.
MU: from a standpoint of assessments, what kinds of costs are you incurring?
JT and CL: it depends on the complexity of the site. From hundreds to many thousands of dollars.
CMN: what about the data that fills the templates. Templates may be accessible while content is not.
Our goal is to identify accessibility of French Web sites for the citizens of France. Examined sites belong to different fields, railway reservation service, government sites, site fro job placement, want to evaluate commercial sites, but haven't done that yet, and hope to evaluate search engines as well. Methods: Tried to define according to the guidelines.
Created a form to fill out for each page, starting at home page starting at level 1. Also skipped common template pages. Stopped at levels where only text pages started appearing (e.g. level 4 or 5). Then wrote report. Team included a blind user, human factor analyst, web-site designer knowledgeable in HTML. Each looked at the site then compared notes to fill out the form for each page. Looked at style sheets, frames, etc. Tried to give a mark like Julie did, but in her case it was not easy. Chose to use stars up to four stars (top level access) three stars (guidelines not always respected, but still usable) 2stars lots of difficulty. 1 star not accessible without assistance of sighted person. Has been given to person with physical disabilities who will come up with own criteria and hopes to have those results integrated within the next month. Now beginning to work with users who are not terribly familiar with the Web for usability issues. We have an English version that only describes the process. The evaluations are in French only.
Judy and Sylvie demonstrated the evaluation site. http://www.braillenet.jussieu.fr and one bad example site: http://www.laposte.fr (a 1-star site)
KC: 5 aspects.
JB: criteria for the review which is a subset of process for the review. in what kind of process to you envision.
KC: would use some of julie's 10. multi-scoring. at least 3 people to even out the wrinkles.
JB 5: criteria?
KC:
JB: those are usability criteria.
Action Kevin: send paper to the group re: usability criteria.
JB: candidate steps: set expectations, test usability criteria, multiple testers.
GK: might depend what instigates the review. a user requesting versus a testing service. that might define the scope of the test.
JB: ability to match review method to the source or purpose of the review.
DD: transparency - whatever process we come up with should be described completely so that people can read and understand it.
JB: methodology is completely available and documented.
TN: open to scrutiny.
CMN: validate to spec. compare full-blown multimedia to text version, when all things are turned off. go through the checkpoints. check semantic structure and navigability. tool like opera can allow you to navigate to discover. cut your window size down to try the site.
MU: process for output of evaluation: report should have 2 areas: quantitative (manual review of checkpoints as well as tool report, like Bobby) and qualitative (1. navigation comprehension, 2. separation of content from presentation, 3. general fulfillment of stated intent).
JB: beginning step: set expectations including showing testing criteria, team, and depth of evaluation. then report.
DD: question of review team: how do we select people or guarantee expertise. we should have a process to qualify the knowledge of the reviewers.
JT: inter-rater testing: every few sites we will have another person independently review the site and see if the evaluations match. helps determine the subjective component and consistency between raters.
JB: anyone interested in working on a draft that says, "here's the process?"
hands: charles, sylvia, mark, gregory.
GR: checking text equivalents are part of p1 and very important. as p1 it is a, "if all else fails."
SS: in setting expectations: need to make it clear how the report will be used. does it only go to the designer or released to public or ??
JB: so many purposes for reviews. Could write something generic enough so that can applied in a variety of situations. e.g., "when setting expectations, ensure that it is clear how the results will be displayed." Can we generalize the middle part any more?
HB: using checklist. start w/a meeting w/representatives from various disability areas to determine what to do for each specific case.
JB: could that be generalized?
HB: I do a lot of reviewing by myself. Use the checklist to go through the site. use html validation tools.
JB: candidate steps:
MU: it's not just that it validates, but there are tests w/adaptive software. need functional testing.
CMN: part of functional testing is: make sure that what on the pages works with the software that is out there. this feedback makes sure that the guidelines are viable.
SS: rationale component in the report. multiple users test each site. that ought to be constant - multiple testers.
WL: why?
JB: quality control issue.
GK: part of machine test is Bobby?
/* some nods yes, some nods no */
GK: promoting its use could get good feedback on the tool and promote its use.
JB /* reading list */ input back into wcag and tools.
TN: inter-rater reliability testing: should imply that there is a primary tester but they may have a secondary tester to validate as some points. the functional tests do not have to be by people with the disabilities the tool is created for. sometimes you may only test a site for 10-15 minutes and only do a Bobby test. therefore, need a list of thing you can choose from.
JB: yes, generalized to say, "you need quality control."and "need functional test with either person with disability or not"
AC: having actual users: experienced screen reader user can do a good job. however, don't know many sighted people that can use a braille display.
GR: shouldn't specify which evaluation tool to use, a number exist. however, running it through W3C HTML validator and CSS validator.
DD: language specific review: we have to account for cultural things. a site may not be accessible if you don't understand an image. For example, an alt could be considered "meaningless" if don't know the language or culture.
CMN: some tools are better internationalized than others. I can check if every image is comprehensible. I can validate manual. Ii can listen to the music, but I'm not good at that. There are tools to help with all of these. It's not what tools are there, but how well you know how to use them.
Action in 3 weeks - Sylvie, Mark, Charles, Greg, Gregory: draft a general process to be used to evaluate web sites that considers the following list: @@copy from judy.
JB: other agenda items that we have glossed by: press/myths/messaging, standards coordination, UAAG FAQ (urgent), and promotion of WAI materials.
Possible examples:
CMN: what about Portugal?
JB: no.
AC: what about Resna in Florida in June? A five day meeting of an assistive technology (not necessarily information processing)
MN: what about the INET conference in Japan 18-21 of July in Yokohama?
GK: a conference in Rome in July but he doesn't know much about it.
CMN: Sophia, Bulgaria, impact of computers on users in late August.
KC: first week of August international council for the visually impaired in Krakov.
GK: what about the Frankfurt Book Fair - electronic publishing - in the fall.
GK: World Blind Union in Melbourne AU in November.
HJ: what about REHA in the fall in Dusseldorf Germany
WC: Spain in September on virtual reality
JB: Edmedai in Montreal
GK: Two initiatives with books are Open E-book forum and the NISO/DAISY consortium. Open E-Book is setting themselves up as a standards organization. George is proposing working hand in glove with the W3C. Currently a subset of HTML, CSS and XML. The draft points to the W3C on accessibility features. OEB pointers for technical production matters. There is interest in the OEB also doing multimedia. This ties in to the NISO/DAISY consortium work on digital talking books which are also XML based. George is going to try to move them toward convergence of DTDs to bring them in line. Hopes to have these reviewed by W3C with respect to namespaces and fragments. The Open E-Book forum may become a huge initiative, Microsoft is moving into it, as well as Gemstar (many users).
CMN: the Wireless Access Protocol WAP) forum uses an xml format with big backing from European mobile phone groups (not xhtml, but there is work going on to converge WML to xhtml).
KH: the NCITS (National Committee on Information Technology Standards) (formally X3) IT Accommodation Study Group - alternate accesses interface protocol. Another group looking at metadata registration issues. Also distance learning standards (IMSproject.org)
JB: what should we be doing with our limited resources with this information? George is sort of a liaison from his other groups to the WAI. Should we be linking to them and to people who are knowledgeable about the convergence of standards?
JB: saw some stuff in Europe about FENN
TN: voice xml, being developed by Lucent (bell labs). A markup language for text to speech for control of telephony resources (xml based) to separate user interface on client side from server stuff.
CMN: There is VRML, which may or may not be recast in to XML (run by W3D now). Oasis and Biztalk collects information about standards
JB: Here is the standard information we want under each of these?
ACTION ITEMS to fill this in: within a week.
GK will fill in the boxes for OEB, NISO/Daisy
CMN: WAP
JB: SEN
SS: Instructional Management System
KH: NCITS
TN: Voice XML
WC: W3D
CMN: will take OASIS
? WEBTV/IPTV
CMN: would it be useful to have information about what things are actually used on the WAI Site?
JB: we have the capability. IT would be nice to have a desired hit-count analysis. So we need to ask them to target the information. How many hits to the home page, how many hits to quicktips, curriculum, guidelines?
CMN: which are straight hits or are referrals from other pages. What domain do they originate from? Do they download anything?
AC: why are we gathering this information?
JB: would lkike to know what resources are being used and by what kinds of people, what things aren't being found. Sort of market research.
CMN: constancy of hits or periodic jumps.
KH: relate it ot particular outreach events.
SS: can we gather info on what is being searched for?
JB: Hits to different translations.
CMN: accept language headers
JB: the WAI home page is being slowly redesigned.
HB: should we take another look at the World Health Organization definitions of disability to bring the new definitions into line?
ACTION ITEM for Helle
KC: Helen Petrie and I have done a lot of slicing of visual and hearing impairment so that remedial action can be taken. It will be published shortly and Kevin will make it available.
CMN: availability of translations and where this fits on the web site and how we get them in the first place.
AC: how are quick tips being advertised? we've given quite a few out, right?
JB: EASI list. after CSUN we will circulate on WAI lists, then on a wider circulation. sometimes people cross-post on their own. do to staff limitations we have not been widely publisizing them.
AC: market research will help us target who to send them to.
GR: do the braille quick tip cards prototypes or actually available?
JB: available in the past and will be available again. post-csun will have more run and announce on the site that they are available. there are 2 pieces: 1. full set on a large piece of paper, and, 2. small card.
MN: on the topic of quicktips, we need comments from Braille readers on the Braille cards, please send your comments to Judy or Max. We can discuss later whether more with the Braille should be created.
JB: lets not talk about the quicktips anymore. We are contrarily trying to restrict the distribution of quicktips, not increase the distribution.
KH: from discussion in his office, the FOSE show in Washington is a wonderful opportunity to let the major vendors and government offices what EO stuff is available.
JB: is the new WAI flyer a good looking document (with quicktips in Larger print on one side) and some resources listed (without URLS). Is this an effective vehicle? Are people going to use this, like a FOSE.
KH: thinks it would be good material. We will be able to see how effective the distribution would be if we also start tracking site hits.
CMN: getting the curriculum into schools and into how-to books out there.
JB: how do we do that?
CMN: by asking people
HB: the other thing would be a good candidate for translation. A bit more describing the work we are doing in a more brochure-like format (but not the guidelines).
JB: what about prioritization?
JB: Ian Jacobs had batched questions from discussion in one of the working group meetings and some thoughts he was having as well. Has anybody been looking back at the other FAQ's from other WG's.
CMN: if the press are looking at them then it is a useful product.
MN: thinks the contents of the FAQ are very useful. But they are hard to locate on the web site.
JB: now they are better referenced from the WAI home page.
CL: the Canadian Government heavily promotes the use of the FAQ for general questions about the WCAG.
JB: first question on UA's list is:
1. Can my screen reader conform? (yes, but the spec not designed for this)
Discussion: How do I find out if my screen-reader conforms to the guidelines or
What sort of applications do these things cover?
1) What is a user agent?
JB: doesn't' like what is a user agent at the top.
JB: then "what is this guideline good for?"
GR: can my assistive technology conform?
2) Who are these guidelines for? Testers, UA developers, software and hardware developers.
3) What products do conform?
4) What do these guidelines cover (what kind of tools conform)?
5) GR: question: who assesses conformance? Who checks products to see if they meet the guidelines?
WC: earlier Julie mentioned needed information to advocate for better accessibility and these would be good to promote development of accessible user agents.
JH: not really applicable.
JB: are there ways the people with disabilities can advocate for these guidelines. In some European languages there is a lag between US development of a tool and internationalization.
AC: another question: that doesn't get covered by these guidelines and why?
GR: question: when will these affect me - address the localization questions in the same questions: What does it mean to conform?
AC: What will a conformant product do for me? How will a conformant product benefit people?
JB: any closing comments..
JB: read over the remaiing UA suggestions. Some humor.
/* begin exerpt from UA Email */
2. Can my UA in conjunction with another AT conform? (conformance must be
native)
3. Can I satisfy a checkpoint by using features supplied by the operating
system? (yes, using system conventions a good idea)
4. What does "applicable checkpoint" mean? (summarize definition)
5. Are these Guidelines legal requirements (no, refer to similar question
11 in [2]. Potentially discuss relationship to 508, ADA, Telecom Act.)
[1] http://lists.w3.org/Archives/Public/w3c-wai-ua/1999OctDec/0502.html
Second suggestions from Ian [2]:
* My browser doesn't support Java. How does that affect conformance?
("applicability" clause).
* Do I have to implement all of the Techniques? (No, none of them in fact.
They are only suggestions and samples)
* Which browsers conform? (Good question!)
* Does conformance mean that a browser is accessible on its own? (The goal
of the guidelines is to make general purpose UAs accessible. However, many
users require additional software -- assistive technologies -- to furnish
some services. The Guidelines have been designed to ensure communication
between user agents.)
[2] http://lists.w3.org/Archives/Public/w3c-wai-ua/1999OctDec/0718.html
Suggestions from UA Working Group:
http://lists.w3.org/Archives/Public/w3c-wai-ua/2000JanMar/0511.html
1."Will this change user agents overnight?"
2."What is a user agent?"
3.What is an accessible user agent?"
4.How does assistive technologies benefit from these guidelines?"
5.When will this affect me?"
6.What does it mean to conform?"
7.Does W3C maintain a list of accessible user agents?"
8.How do I know that a user agent meets the requirements of the Guidelines?"
9.Why is the DOM important to this document?"
10.What can I do as a user to improve user agent accessibility?"
ATAG FAQ [4]
[4] http://www.w3.org/2000/02/ATAG-FAQ.html.en
/* end exerpt from UA Email */
[Also check UA teleconference minutes from last week]
next Friday March 30 at 8:30 AM
The Thursday calls have a resource-staffing problem that Judy hopes to clear up in the near future
Judy thanked everyone for coming to CSUN and for lasting through the meeting today. She will be in touch with various individuals on various subjects.
Copyright © 2000 W3C ® ( MIT, INRIA, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply. Your interactions with this site are in accordance with our public and Member privacy statements.