Important note: This Wiki page is edited by participants of the RDWG. It does not necessarily represent consensus and it may have incorrect information or information that is not supported by other Working Group participants, WAI, or W3C. It may also have some very useful information.
From Research and Development Working Group Wiki
[Brief synopsis of 1-2 sentences]
Page author(s): VConway | Vivienne Conway
Other contact(s): Máté Pataki
Accreditation, Accessibility, Certification
There has been discussion about how accessible websites are accredited. Some methods involve an external accreditation groups, using the W3C WCAG 2.0 certificate, and self-accreditation schemes.
Does the fact that a website carried an accreditation necessarily guarantee that a website is accessible? This is probably not the case. We should look at which accreditation schemes are most reliable, and how often a website should be re-evaluated. There is also the problem that websites are evaluated on a page by page basis. Some websites are mostly WCAG 2.0 A compliant, with a number of the pages being at a greater or lesser level of compliance. Information currently states that this website is therefore compliant only to the lowest page level.
The WCAG-EM is being developed by the WCAG Evaluation Methodology Task force under the WCAG Working Group. This document is current in draft form and the Task Force is working through public comments. The document deals with the issue of how to assess a whole website as opposed to the page-by-page formal compliance testing currently stipulated by WCAG 2.0 documents.
Numerous companies offer some form of third-party or external accreditation as can be seen from the resources listed below. Other organisations self-certify their website stating that it conforms to certain guidelines. The discussion questions deal with some of the issues of accreditation and invite discussion as to whether there is merit in third-party or self-certification of websites, how this is awarded and the role W3C might play in providing guidance to organisations offering accreditation.
Accreditation is closely bounded by accessibility testing. As a result, all of the strengths and weaknesses of accessibility testing are inherited by accreditation methods. A few of the weaknesses include (references appear below the discussion questions):
- Guidelines are proposed by consensus rather than by systematic experimentation, which may hard their validity
- While guidelines address many user needs, their coverage is not full [Power, 2012]
- The over-reliance on tools, which perform badly and rely on the above mentioned guidelines [Vigo et al, 2013]
- The problem of reaching an agreement with manual reviews [Brajnik, 2011]
- The problems of sampling [Brajnik, 2007]
Yet accreditation is often sought by organisations as a means of displaying the level of work or accomplishment they have put into making sure that their website is accessible. Organisations may believe they should have some means of displaying the external validation they have sought on the accessibility of the website. Or they may believe that the effort they have put into the accessibility of the website entitles them to display some sort of self-certification or accessibility compliance claim.
On September 11 during the scheduled RDGW teleconference call, we will be hosting a Catalogue Topic Discussion about this issues. You are invited to read the proposed discussion questions below and prepare to shares your thoughts and ideas with the group. The discussion will be summarized into a Catalogue Entry after the discussion.
Because the discussion is limited to one hour, we will only be dealing with third-party or self-certification concepts, rather than delving into the issues of using certified methods to develop websites or applications. Issues such as developing workflows and processes as in ISO 9000 and BS8878 and other quality assurance schemes will need to be left for future discussions.
We can also look at 'Health on the Net' (http://www.hon.ch) as an example of an accreditation system that has been in operation for a while. It mainly relates to the reliability and validity of the information rather than the accessibility of it, but is worth looking at.
David Sloan writes: "We need to be thinking about the problem that site accreditation is trying to solve. Is accreditation a way that helps site owners be reassured and can demonstrate to third-parties that their site reaches an acceptable and recognised level of accessibility. Is it a way that customers who have commissioned the site can be sure that the site they have procured reaches an acceptable and recognised level of accessibility? Or, is it a way that people with accessibility needs can recognise and trust that the site will allow them to use it for its intended purpose? The question of trust is a critical one. Especially for disabled web users, people need to know that accreditation means something worthwhile. Accreditation must, in my view, covers as many disabilities as possible. Accreditation must, in my view, deal with the complexity that the dynamic nature of web sites brings (e.g. user generated content, third party content such as ads, which can reduce accessibility." [Sloan, 2013]
- 1. In the past, many websites carried some kind of badge such as the 'Bobby' tick and many included the W3C logo, in order to make statements about their accessibility. This has greatly diminished since WCAG 2.0 was launched. Why do you feel this has happened?
- 2. Has there ever been, or is there still, value in external or third-party accreditation schemes?
- 3. Has there ever been, or is there still, value in self-certification of websites, e.g. "This website meets WCAG 2.0 to Level AA"?
Design and Management
- 4. How do third-party accreditation schemes handle website changes?
- 5. What features are included in a third-party accreditation scheme?
- 6. What is the accepted practice for re-testing? 6 months, 12 months etc.?
- 7. What happens when a website with accreditation is found to be non-compliant?
- 8. What are some of the current pricing models for website accreditation? e.g. a yearly fee, subscription etc.
- 9. Which accreditation schemes are most effective? Are there any common elements?
- 10. Are there valid external accreditation schemes, and how are they monitored?
- 11. Is there any way to have an accreditation mean anything after the moment in which it is awarded?
- 12. Does accreditation do more harm than good?
- 13. Can third-party or self-certification lead to a false sense of security for the website owner?
- 14. Is the W3C logo being misused? If so, how can this be monitored?
- 15. What value do the current accreditation models bring to the website-owning organisation?
- 16. Does the website user receive any benefit from using an accredited website?
Future practice considerations
- 17. Should there be some kind of standardised third-party accreditation scheme for websites?
- 18. Should W3C become a certifying organisation, thus challenging its current model for vendor neutrality?
- 19. What is the opinion of WAI with regard to third-party or external accreditation schemes?
- 20. How do we have accreditation mean more than a pasted 'tick' on the website that anyone can copy?
- 21. There appears to be very little literature on the value or harm of accessibility accreditation, is this something that warrants further research?
- 22. Are you aware of scientific material that can be added to this catalogue entry? If so, please let us know.
"Specifications for a Web Accessibility Conformity Assessment Scheme and a Web Accessibility Quality Mark" CWA 15554. ICS 35.240.99 obtained from: ftp://ftp.cenorm.be/PUBLIC/CWAs/e-Europe/WAC/CWA15554-00-2006-Jun.pdf
"UWEM, Unified Web Evaluation Methodology version 1.2" 2007, WAB Cluster; obtained from: http://www.wabcluster.org/uwem1
"European eAccessbility Certification: CEN Workshop Agreement: How to assess Web Accessibility Conformance" obtained from: http://euracert.org/en/resources/cwa/
"New EU legal framework for accreditation" http://www.ukas.com/technical-information/international-role/New-EU-legal-framework-for-accreditation.asp
"Europa : Web Accessibility Policy" http://europa.eu/geninfo/accessibility_policy_en.htm
"WebAIM: Accessible Site Certification" http://webaim.org/services/certification/
"5 things you should know before buying accessibility audit and accreditation services" / Prof. Jonathan Hassell, Hassel Inclusion 14 January 2013. http://www.hassellinclusion.com/2013/01/accessibility-accreditation-value/
"BCS disability charity and RNIB offer website access accreditation" / British Computer Society - December 13, 2005 link: http://search.proquest.com.ezproxy.ecu.edu.au/docview/236990085
"Guidelines are only half of the story: accessibility problems encountered by blind users on the web" Proceeding CHI '12 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Pages 433-442 ACM New York, NY, USA ©2012 table of contents ISBN: 978-1-4503-1015-4 doi>10.1145/2207676.2207736 obtained from: http://dl.acm.org/citation.cfm?doid=2207676.2207736
"Benchmarking Web Accessibility Evaluation Tools: Measuring the Harm of Sole Reliance on Automated Tools" Vigo, Markel; Brown, Justin, Conway, Vivienne. Paper presented at the Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility, Rio de Janeiro, Brazil. http://dl.acm.org/citation.cfm?id=2461124
"The Expertise Effect on Web Accessibility Evaluation Methods". Brajnik, Giorgio; Yesilada,Yeliz; Harper, Simon. Human-Computer Interaction; Vol.26, Iss.3, 2011 obtained from: https://www.escholar.manchester.ac.uk/uk-ac-man-scw:147745
"Beyond Conformance: The Role of Accessibility Evaluation Methods" Brajnik, Giorgio. S. Hartmann et al. (Eds.): WISE 2008, LNCS 5176, pp. 63–80, 2008.c?. Springer-Verlag Berlin Heidelberg 2008, download from: http://users.dimi.uniud.it/~giorgio.brajnik/papers/iwwua08-kn.pdf.
"Health on the Net" http://www.hon.ch.
"Agenda for Teleconference on Wednesday 4 September 2013"; Sloan, David. Personal communication, email. Received 5 September 2013
Back to the list of topics.