Proposed Survey Questions
Proposed Survey Questions
Adjectival rating versus granular scoring
Survey Question Instruction / Introduction
- An Adjectival rating would aim to grade web pages to one of a few possible tiers. (Perhaps as few as 3, perhaps as many as 7.)
- Granular scoring would aim to generate a grade from 0 to 100% and would probably had additional detail "under the hood".
- A granular system might also have wide range of points available, but report largely in tiers. FICO scores are one example of this.
Survey Question
Based on what you have heard so far, do you a favor adjectival ratings, or something more granular?
Survey Responses
- No opinion / not sure
- Strongly favor adjectival rating
- Strongly favor granular scoring
Depreciating scores
Survey Question Instruction / Introduction
During the Silver VF2F, there was a good bit of discussion if reported scoring should be depreciated over time. This is a topic that the Silver Task Force has discussed on multiple occasions. We think the discussion is mature and would benefit from feedback from AGWG.
Survey Question
Should Silver conformance claim be dated? If dated, should that conformance claim score depreciate over time?
- Option 1: Date and Version stamp a conformance claim. This would provide transparency to anyone viewing a claim how old the claim is, and the viewer can make their own judgement. Reducing the accessibility of a site is not a function of time, but rather is a function of the number of changes since the last time the site was tested. Depreciating a score over time will put a heavier burden on small sites with few changes. The choice of depreciating a score is more in the realm of regulation, as standards say what you have to do, not the conditions or penalties of not doing it. The Silver group passed a resolution for this option on 28 April 2020.
- Option 2: Depreciate a score over time. A claim is worth less the longer it has been since a site has been tested. Therefore, we should have a formula to depreciate the score over time. This is a widely requested feature in certain industries who want to track their accessibility progress over time.
- Option 3: No depreciation and no time stamp. W3C is writing the standards. This is a matter for regulators to decide how they want to implement it, not for a standards organization. We are not the accessibility police.
Survey Responses
- Option 1: Date and Version stamp a conformance claim
- Option 2: Depreciate the value of a conformance claim score over time
- Option 3: No date or depreciation of score
Comments
The depreciation aspect under discussion is applicable to those scores derived from user-testing (high value / high effort / high cost) as opposed to "all" test scores. Mechanical scores can be calculated (scheduled) at will (weekly, monthly, quarterly, annually, etc.) and so will likely be accurate. User test scores that are 3 years old are likely inaccurate at best, completely false at worst, and most likely somewhere in-between.
Those potential states need to be accounted for in score calculation, which proposes to add scores from both mechanical and user-testing activities towards a final score.
Reference: https://www.w3.org/WAI/GL/task-forces/silver/wiki/Survey_on_depreciating_scores