Context-Tailored Web Accessibility Metrics

Motivation

There is evidence that conformance to guidelines does not guarantee full access [see Sloan et al. (2006) Contextual web accessibility - maximizing the benefit of accessibility guidelines]
- Conformance to guidelines establishes a minimun requirement
- Some people still find problems though
- Considering the interaction context the is key
The interaction context consists of (at least)
- The particular abilities of the user
- The characteristics of the assistive technology in use
- The features of the accessing device
As far as metrics are concerned, if evaluation reports are context-tailored, so will be accessibility metrics

WAQM is too tied to WCAG
We propose the Logic Scores Preferences (LSP) method
- An aggregation model based on neural networks
- LSP computes a global score from intermediate scores that can be failure-rates or absolute number of accessibility problems
- Considering the interaction context the is key
E = ((W₁E₁)^p(d) +..+ (W_iE_i)^p(d)+.. +(W_nE_n)^p(d) )^(1/p(d))
- When components have a different impact, positive normalized weights are associated to each evaluation result: W₁,..,W_n where 0 ≤ W_i ≤ 1 and sum(W_i)=1
p(d) values are predefined elsewhere and they are selected upon the required logical relationship between elements of the system: be different levels of logical conjunction and disjunction
- When simultaneity in satisfying the requirements is necessary, conjunction is applied
- If the objective is to penalize the main component only if one of the subcomponents fails, disjunction is applied

Device-tailored metrics
- Testing tool: a tool that evaluates the accessibility of pages considering the specific hardware and software features of a device
- Context of use: able-bodied users accessing the web with mobile devices
- Usability metrics were compared with device-tailored metrics and non-tailored metrics
- A test with 20 users confirmed that device-tailored metrics are a more accurate approach to measure user experience in terms of efficiency and satisfaction
User-tailored metrics
- Testing tool: a tool that evaluates accessibility and usability for screen reader users
- Context of use: blind users accessing the web with Jaws screen reader
- Obtained accessibility scores were deployed as link annotations that denoted the accessibility of the page pointed
- A test with 16 users confirmed that annotations increased user orientation