Research and Development Working Group Teleconference

11 Sep 2013

See also: IRC log


Vivienne, Markel, Giorgio, Shadi, David, Annika, Justin, Christos, Yehya, Simon, Mark, Silvia, Klaus, Yeliz
Simon, Vivienne


Introduction of topic and explanation of format - Simon (format)

<sharper> http://www.w3.org/WAI/RD/wiki/Accreditation_Methods

SH: this is the first one of our core discussion topics
... intended to look at different ones
... outcomes of these discussion is an initial analysis
... possibly have follow-up discussion with external experts if needed

<sharper> http://www.w3.org/WAI/RD/wiki/Advice_for_Organisers_of_Catalogue_Topic_Discussions

SH: to eventually develop a catalog topic item from these discussions
... have some advice for organizers that we can evolve
... open discussion intended to collect different perspectives and ideas

VC: reminder that i'm recording this session
... so that we can put it up as part of the note
... talking particularly about 3rd party certification or self-declaration
... wondering what the value of that is
... have some background content in the wiki
... thanks for the feedback Markel and David
... thought would start with historical perspectives
... please say your name before you talk

SAZ: are the recordings publicly available or for your own use?

VC: for my own use internally

Historical perspective - 10 mins

<Yehya> many people used bobby just imitating others or using templates that included the bobby logo

MV: have Giorgio with me today!

GB: lots of the previous labels like "bobby" did not work
... because of difficulty of determining accessibility barriers
... assessment of true quality is challenged by such labels
... because people might be mislead by results
... this applies to any accreditation scheme

VC: have you seen any that had value?

GB: no
... good one would be focused on the process
... what process did you follow to ensure accessibility

<Yehya> Accreditation of websites will be credible only if the website allows users to provide their experience with the website. So an accreditation stamp should provide a feedback form, which allows users to submit identified accessibility issues to the website owner. The accreditation stamp should indicate a time frame when will be resolve the identified accessibility issues. A statistic about raised accessibility issues, resolved and pending will be as well helpful in t

GB: that would likely better indicate the quality

DS: slightly disagree with Giorgio
... previous schemes contributed to raising awareness
... maybe success in that perspective
... these earlier schemes helped people understand the importance of accessibility
... they may not have lead to accessibility of the website but they made aware of an aspect of website quality

VC: what about value of self-certification schemes?

SAZ: agree with Giorgio about importance of process quality rather than post-development evaluation only
... also agree that most previous logos were developed for awareness raising and do not follow the requirements of "self-declaration of conformity (SDoC)"

YM: only by involving the end-user in the process
... you can make sense of the accreditation
... but this would also be part of a process
... had such discussions with many customers
... use a feedback form along with the statements
... that have timeline for when the owner can fix issues

VC: yes, seeing more of these feedback forms in Australia

YM: yes, useful approach for accessibility

KM: agree with previous speakers
... need to outline the process of how the result was achieved
... together with feedback form

VC: so accessibility statements?

KM: or stored on a server for large scale accreditation

VC: need to black-box the website at the time of accreditation?

KM: yes, this might need to happen
... also need to communicate if the check was self-declared, expert, group of experts, end-users, etc
... also important to consider the level of complexity and if the process would be feasible in practice

Current practices - 30 mins

VC: how do you see 3rd party certification schemes working in practice?

MV: haven't seen the track-keeping thing in the accreditation domain
... but research papers about accessibility performance over time
... one mechanism they used is archive.org
... which has snapshot of websites
... agree with Klaus that we should keep track of how things evolve
... but difficult as websites change frequently

VC: do large organizations keep copies of their website?

DS: coming from research to consultancy
... asked to assess websites, often in a legal context

<markel> Web accessibility guidelines - A lesson from the evolving Web_http://dx.doi.org/10.1007/s11280-011-0130-8

DS: raises the question of re-testing an remediation
... maybe not written in a standard or policy

<markel> Evolution of web site design patterns: http://doi.acm.org/10.1145/1095872.1095876

DS: but need to have at regular intervals

<markel> Progress on Website Accessibility?: http://doi.acm.org/10.1145/2435215.2435217

DS: should be part of the accreditation scheme

VC: aware of scheme that is valid for only 1 year

<markel> these were papers of how we can keep track of accessibility or usability

VC: but website can change within 5 minutes

DS: things like VPAT do not seem to consider this

AN: experience with monitoring of websites in regular intervals
... approaches in the Netherlands and also in Norway
... review by experts once a year and automatic checks monthly in between
... this would indicate major changes such as a relaunch

VC: is that documented somewhere?

AN: eGovMon may have some description about the process

YM: also implemented observatory
... in which the website is copied into the database
... and compared during rechecking to only check pages that changed
... then report improvements and failures in these changes

design - 10 mins

VC: what aspects do accrediation schemes need to consider?

DS: need robust scheme that is usable for the owner
... also that ensures a quality assurance process

<annika> Regular monitoring of web sites by eGovMon project: http://accessibility.egovmon.no/en/benchmarking2.0/

DS: also need to think about technical aspects vs experiential aspects as well

VC: agree that benefit for the owner is necessary
... and ultimately also should benefit the end-user

DS: would be a fantastic research project to collect information about what would most help people report good quality

<markel> In line with what David says

<markel> http://doi.acm.org/10.1145/1639642.1639677

<markel> Collaborative web accessibility improvement: challenges and possibilities

MV: "social accessibility" by IBM Japan
... may be reliable and not as sophisticated

<davidsloan> I think that collaborative identification and fixing of accessibility problems is a great crowd-sourcing initiative, but is complementary to an accreditation system that would identify sites that *don't* have such problems

benefits - 10 mins

<annika> In the Netherlands, they use a combination of monitoring (http://versie1.webrichtlijnen.nl/monitor/) and accessiblity label "drempelvrij" (http://www.accessibility.nl/ondersteuning/het-inspectieproces).

VC: benefits for users from accredited websites?

<davidsloan> I'd like to see researchers gather more details from disabled web users on how they assess quality of web sites in terms of accessibility and user experience; and their views on accreditation schemes

SH: quite some work done confirming that website that meet standards are often built better
... and meet better quality attributes
... so maybe not immediate but would contribute to improvement
... would help promote websites that are better made

MV: agree that benefits may not be direct

<davidsloan> any accreditation scheme must be trustworthy, and as soon as trust is lost (e.g. because it isn't sufficiently rigorous and marks a site as accessible when it isn't), then the accreditation scheme fails for people with disabilities

MV: but also helps set expectation
... also side-effects of that

VC: agree with David's comment on importance of trustworthyness

DS: opens discussion on involving 3rd-party
... for example fixtheweb process had a volunteer between someone who percieves an issue and the website owner
... because also the role of people in accreditation is important
... but brings the challenge of the skills and capabilities of users

<Vivienne> SA: strongly agrees with David, trustworthiness is crucial

<davidsloan> +1 to shadi's point that trustworthiness affects the underlying standard the accreditation scheme purports to support

SAZ: agree with David on trustworthyness
... also that unreliable schemes could undo much of the work we've been doing on accessibility

Future practice considerations - 10 mins

VC: how to ensure that someone has the ability to say that a website is accessible
... Giogrio wrote about the aspect of the role of expertise

<davidsloan> http://www.hassellinclusion.com/2013/01/accessibility-accreditation-value/

DS: Jonathan Hassel posted something about selecting acreditation providers

VC: in the wiki
... also paper by Suzette Keith about overclaiming in logos

DS: discussion about need for professional acreditation
... also discussion about an international Association of Web Accessibility Professionals

VC: also discussion about W3C's role in this

<Vivienne> SA: Process is paramount, important - open and not closed to a particular group

<Vivienne> SA: who can make a claim?

<Vivienne> SA: is a need for self-certification and they have a QA process in place, others need help (technical perhaps), both can and should co-exist. Skill is importnat - training and materials - but needs to be and remain open.

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.138 (CVS log)
$Date: 2013-09-11 14:59:09 $