This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
The algorithm being used to compute scores is obscure. Users need to get a visual sense about the severity of the errors, to prioritize where they should invest time. - Simplify the scoring algorithm to make it easy to explain - Explain it in the help page - Display severity levels, and make critical errors easy to spot - Display the top 3 most severe errors at the beginning of the report.
More work is needed to improve the layout, but the basis is done, so I'm closing this bug. Scoring algorithm ----- Failures have different severity levels. Visually, the end user will see the following levels next to each error message: - critical: at least part of the rendering breaks on mobile devices - severe: rendering may not break but user will be lost or user experience very problematic - medium: should work but poor user experience - low: improvements possible Internally, more levels seemed needed, and each level is associated with a basic cost - critical * level 6 (highly critical), cost: 50 points * level 5 (critical), cost: 20 points - severe * level 4, cost: 10 points - medium * level 3, cost: 5 points - low * level 2, cost: 2 points * level 1, cost: 1 point Basically, the final cost is the sum of the costs of the errors, and the score is 100 minus that cost. If the cost exceeds 100, then 100 it is. 2 or more highly critical errors mean you get a 0/100. That's strict, but then highly critical is highly critical (in practice: frames or not an HTML page). Some more empirical rules to improve the score: - When an error occurs more than once, the final cost for these errors is: cost = basic_cost * sum(1/k) for k in 1 to nb_occurrences - The cost for heavy pages depends on the weight of the image. - The cost for the number of embedded resources depends on the number of exceeding resources. - The total cost of medium errors cannot exceed 50, not to be possible to get a 0/100 out of not-that-severe errors - The total cost of low errors cannot exceed 50 as well. Severity ----- Severity level is displayed next to the failure message Top 3 most severe errors are displayed at the beginning of the report.