This extend abstract is a contribution to the Online Symposium on Website Accessibility Metrics. The contents of this paper has not been developed by W3C Web Accessibility Initiative (WAI) and does not necessarily represent the consensus view of its membership.
Templates are highly used in Web development. Estimates indicate that 40-50% of the Web content uses templates[1]. Automatic accessibility tools report errors and warnings on pages built on those templates. Consequently, if a template presents an accessibility problem, that problem is reported as often as the template is used within the page/site.
From the developers’ perspective, standard accessibility evaluation tools provide obfuscating results. The same error is constantly repeated, producing unnecessarily long reports that confuse developers[2], concealing fundamental repairing issues.
Metrics for accessibility evaluation have the same issue. A bad result, deriving from a large number of errors in a page or site assessment, may be a consequence of a small number of problems in a frequently used template that can be rapidly corrected. Therefore, common accessibility metrics may be misleading.
We can decompose our research question in:
Template detection is often used in the fields of Information Retrieval(IR) and page Clustering [3]. Although most work on templates ignores accessibility, its use has already been proposed as a mean to improve it[4]. In fact, if templates become accessible and widely used, there is a lower probability of having inaccessible pages. Otherwise, the possible errors will propagate, causing the previously mentioned issues.
Metrics, such as UWEM[5] and WAQM[6], are invaluable to assess page accessibility. These provide different perspectives of the accessibility quality. However, none directly addresses the developers' efforts to correct the accessibility problems, in relation to the common development process. Templates are fundamental for this process and must be considered.
To detect templates, we propose the use of a simple algorithm to identify common elements amongst the HTML DOM trees - Fast Match algorithm[7]. It offers a reasonable estimate of the common elements for initial assessment. This algorithm was validated; the matches obtained were compared with the same matches conceived manually(100tests).
Then, we modified QualWeb, an automatic accessibility evaluator[8], to consider the algorithm. The tool accepts a set of pages, identifies common nodes and evaluates the pages. The evaluator itself did not influence the results; we could perform the evaluation with any other evaluator. We even perform similar evaluation in an accessibility evaluator of another project WCAG 2.0 Assessment Tool[9], but the assessments, from a templates' point of view, have not changed substantially.
Afterwards, we prepared a study comparing each Web page with its home page, to identify common elements. The results provide an indication of the percentage of accessibility issues that are detected on templates shared between those two pages, so that we can verify if templates have an important role in accessibility. For a deeper analysis, one should consider within-page templates and templates across several other pages of the same site.
To address reporting, we modified QualWeb to accommodate the template awareness. Reporting is aggregated in two sets: template(common nodes) and specific(unique page nodes). Consequently, problems(errors and warnings) are reported only once, if they occur in similar nodes. On the template set, each reported problem indicates the number of occurrences(of the common node/template).
To address the accessibility quality of a page/site, for a developer, we combine these sets’ assessment as follows:
The first equation indicates the number of accessibility problems of a page - Α(pi)-, combining the number of problems on the template set - αt- with those on the specific part of the page - αs(pi).The second applies to a site and thus sums the number of accessibility problems of specific part for each page. This way, a development team can know if the effort to correct the accessibility problems of a site/page is worthy.
The study for assessing the impact of templates was also used to produce an initial quantitative assessment of reporting and metrics. We selected four representative Web sites- Google, Wikipedia, Facebook and Amazon-, two Portuguese newspapers - DN, and Publico - and WordPress.
Major limitations:
The results show the percentage of template vs. specific outcomes from techniques(i.e. pass, warn or fail), considering all the seven sites. The average for the template set of the accessibility outcomes(e.g. pass, fail, warning) is 38.85%(σ=7.48). Of those, 34.5%(σ=7.0) were warnings and 0.8%(σ=1.0) were fails. Therefore, about 35% of issues(fail and warning) would be addressed twice for repairing if templates were not considered.
| Web Sites | %template(with problems) | Possible problems without templates | Α(S) | Α(S)/Applicability | Decrease |
|---|---|---|---|---|---|
| DN | 36% | 291476 | 192717 | 0.95 | 34% |
| WordPress | 43% | 320890 | 196706 | 0.92 | 39% |
| 19% | 15183 | 12539 | 0.95 | 17% | |
| Publico | 38% | 207531 | 131824 | 0.95 | 36% |
| Amazon | 33% | 128058 | 87720 | 0.95 | 32% |
| Wikipedia | 39% | 26985 | 17513 | 0.93 | 35% |
| 34% | 273114 | 184677 | 0.93 | 32% |
The results show a significant decrease in the scores. The results presented in each column(left to right) are:
The main directions identified were: more accurate template detection (and backtracking); intra-page and extra-page templates; page clustering; another type of analysis; and developers’ assessment.
This work was funded by Fundação para a Ciência e Tecnologia (FCT) through the QualWeb national research project PTDC/EIA-EIA/105079/2008, the Multiannual Funding Programme, and POSC/EU.