W3C

Site Navigation

Nearby

Results of Questionnaire WCAG 3 - Headings exercise

The results of this questionnaire are available to anybody.

This questionnaire was open from 2021-05-13 to 2021-05-25.

12 answers have been received.

Jump to results for question:

Headings method/tests review

1. Headings method/tests review

In the previous meeting we agreed to start working on the guidelines / methods / tests in order to provide more concrete examples that can be used in assessing the scoring and conformance structure.

The first part of that is to review a current guideline, and we chose Headings, part of the Structured content guideline. Please review the:

Structure content guideline
Outcome: Convey hierarchy with semantic structure
Method: Headings in HTML, particularly the Tests tab.

Think through how you would apply that in practice, and please outline the top 3(ish) things you would want to change / improve.

If you have the time and an approach you would like to try out, you can make a copy of the guideline template and start working through that as an example.

Details

Responder	Headings method/tests review
Justine Pascalides	1. In Tests: Short pages/sites/etc. may not need to be divided into separate sections. Add text "if appropriate" or something along those lines to address scenarios when content doesn't need to be broken out into separate sections. 2. Leaving as a placeholder until I can complete the review.
Andrew Kirkpatrick	First, it is very confusing as to what is the normative language, when the wording of the outcome is different in the support resources when compared to the spec. What is the actual outcome? "Headings with levels organize content" or "Headings organize content" or does it include the sentence that follows these in the respective documents? Nit on the Method "Headings in HTML" - it seems odd that the platform includes PDF, word processing, slideware, and ARIA(?). One core concern that I have with this outcome is that it presumes that all content must be organized with headings, and I'm not sure that it true. Very simple pages such as a page with a <title> with Open Letter to ... and then the page goes right into the letter content - does that need a heading even though it would probably duplicate the page title? My example for the simple open letter may not be that common on the web in HTML, but it certainly is common for PDF documents. I also note that a new google doc doesn't seem that have any headings but does provide structure via role="banner" etc. Is this a problem? A second area is the logistics of scoring. To score this, someone needs to count how many headings there are on the page, how many are appropriate, how many are inappropriate, and how many additional headings are needed. In addition to this being a substantial task, there is very little margin for error in the current rating levels. For example, on the WCAG 2.1 spec, there are the following headings: H1: 1 H2: 14 H3: 29 H4: 88 H5/6: 0 Total: 132 To get a rating of 4, there can be no more than 6 headings that are incorrect or 5 headings that are missing. For a very long document this is quite possible, for example, I believe that many people might flag the bolded text at the top of the document (e.g, "This version:" as headings) - if they did then that document would be rating 3 maximum. On a shorter page such as adobe.com's home page: H1: 1 H2: 11 H3: 17 H4/5/6: 0 Total: 29 To get a rating of 4 the number of errors possible is no more than one. In the case of the w3.org site there are invisible headings such as h2 News /h2 that help justify the second most prominent text on the page being an H3 heading - is that ok? I assume that evaluators will have different opinions unless we really clarify the guidance. I would be ok with the ratings if they were as follows, but am still concerned about the subjectivity and overall effort required to evaluate. 0: <20 1: <40 2: <60 3: <80 4: >80 Third, we need to clearly define whether headings can ever constitute a critical error.
David MacDonald	===Change/Improve issue 1=== A) The requirement that headings need to be visually distinct requires some testable statements. Headings are generally made visually distinct in several ways. (1) bold (2) Large font (3) spacing (4) positioning above the content (5) occasionally the length of the string of text in the heading can be an indicator (6) Occasionally the use of Capitalization indicates visually that it is a heading How many of these would need to be present for it to pass. How are we sure something is a heading? Is it because it is introducing content below it? I think we would need to say what we mean by a heading. ===Change/Improve issue 2=== The critical error says "One or more headings necessary to locate the content needed to complete a process are not visually distinct." Does that mean that there is plain text there that says something like "Mailing address" but is not visually distinct? Where the evaluator would simply have to identify headings that are visually present but not visually distinct. I'm assuming it doesn't mean the evaluator needs to examine the process and decide whether there SHOULD be a heading when there is nothing? Is that a right assumption? This failure seems to trigger a critical failure, which gives the author 0% (0 points) for the SC in the process. Seems like a high cost and yet the language seems a bit ambiguous for such a consequence. ===Change/Improve issue 3=== RATING SCALE - these comments are both for this Guideline and rating scales in general: It seems that the rating scale requires the tester to: (1) count up all text that "should" be presented visually as headings but are not visually distinct (2) Count up all the instances of visually distinct headings that are not marked up as headings (3) Count all instances of properly marked up headings that are not visually distinct (4) count up all passing instances of text that "should" be presented visually as headings and are properly shown as visually distinct (5) count up all passing instances of visual headings that "should" be coded as headings and are properly shown as visually distinct (6) calculate the percentage of passes Scanning a page for errors is very quick but stopping for every heading that passes is going to take much longer. It should be noted that counting all the passing instances of a guideline is time used by an evaluator (and expense by a website owner) that does translate into improved accessibility. This increases the cost of evaluation. So if the company doesn't increase the budget, they may end up with less accessibility for the equivalent budget under WCAG 3 than 2.x. ===Change/Improve issue 4=== The summary says: > Headings are used as navigation features by any user who has a disability that impacts their ability to read. If the heading is not relevant to the subsequent content, it cannot successfully provide navigation support. This sounds like a new kind of failure introduced in the (non normative?) informative section which doesn't appear to be mentioned in the guideline. The failure here is a that a properly marked up heading that is also visually distinct, has a text string in the heading doesn't correctly identify the subsequent content. Is this a morphing of 2.4.6 into WCAG 3? If so, then I think the normative text would have to reflect that requirement. I think we need a fair amount of work on this Guideline
Bruce Bailey	The main thing I keep getting hung up on is that I cannot figure out the subject of our requirements. > Organizes content into logical blocks with headings relevant to the subsequent content. Who/what organizes the content? As written, is the sentence grammatically correct? Where is the subject noun? I think this should be: > Content is organized into logical blocks with headings relevant to the subsequent content. This is still a little vague, but it is grammatically correct. I agree with avoiding passive voice. But "is" is an action verb. I feel like I am missing something behind the logic. I also want to second the concern for making a tally of instances of doing things correctly. I wasn't so worried when the main concern (that I heard) was for site owners gaming the system. The more recently vocalized concerns (and reflected in responses to this survey) for the geometric increase to human evaluator testing time and labor, is a very significant issue IMHO.
Jennifer Strickland	"One or more headings necessary to locate the content needed to complete a process are not coded as headings." I would expect the "semantic structure" of headings be communicated with an order of headings in the outcome text. "Properly coded" needs to be sufficiently clear, communicating the an order to headings, such as what HTML does with H1, H2, H3 or Microsoft Word does with Heading 1, Heading 2, Heading 3, and so forth. I would expect that to be documented/conveyed in the tests as well as Outcome and Method.
Michelle Chu
Laura Carlson	For "Method: Relevant Headings "https://www.w3.org/WAI/GL/WCAG3/2020/methods/relevant-headings/ ===Change/Improve Issue 1=== I am not sure what the measurement means. I find the following difficult to understand: "Measurement: Create a percentage of the expected headings within a view that have appropriate text alternatives percentage of the expected headings that are present is provided." Readability Indices of that verbiage according to https://www.webfx.com/tools/read-able/ Flesch Kincaid Reading Ease: 27.5 Flesch Kincaid Grade Level: 15.9 Gunning Fog Score: 18.1 SMOG Index: 14.1 Coleman Liau Index: 18.2 Automated Readability Index: 18.7 Questions: * How is the "percentage of what is expected determined"? * What if the heading is text and doesn't need a text alternative? ===Change/Improve Issue 2=== The test for relevance of headings does not contain any Expected Results. Suspect that it should be Check #1 is true. ===Change/Improve Issue 3=== What is a "Heading"? It may be good to have a definition linked.
Alastair Campbell	I created a vertical slice of the heading guideline down to the HTML tests, and made some adjustments here: https://docs.google.com/document/d/1qlgM-WNSNz4DsU2I3HMRnKlwCVpKh7sKY8_dQVM2byk/edit?usp=sharing The comments/adjustments are: - I agree with Andrew's comment that headings may not be needed for all types of content. I added 'sections' as a scoping mechanism, so if you don't have multiple sections, headings are not required. - At either guideline or outcome level, shouldn't the user-requirement be the primary thing? I had thought that was the organisational principle. We only have a few guidelines, but I think it's important to establish whether the target of the guidelines (/outcomes?) is the content, or the user-requirement. - This is a much higher level of requirement than WCAG 2.x, where you only fail if you have content that appears to be a heading but is not marked up as such (1.3.1). Missing levels, or even not having headings is not necessarily a failure now. I recommend separating the 'has headings' aspect from the 'heading levels', and the 'heading descriptions'. - Should the descriptive aspect be part of this guideline? It is not that I'm wedded to how 2.x does it, but the descriptive aspect could go across many items, including heading, labels, status messages, general content. Now that content understandability has it's own guideline, should it be part of headings? - I really struggled with how to do the scoring of this. I wanted to have different tests contribute to the score in different ways. E.g. Your base level is that visual headings are marked up as headings (regardless of HTML or ARIA). Then a better score if they use a good structure, and have good descriptions. I had a go in the doc, but I'm not happy with it. I know I have confused methods/tests here, but I only got part-way through wrangling it. - I'm not sure what "can be used as navigation" really means. That's very user/AT dependant, if we took that approach there would be no point in testing the mark-up. Was that supposed to be the descriptive aspect? - On note of caution: In training I've found that it is often the case that an external person does not always know what an appropriate level should be. For example, is a section a sub-section (sub-heading) or a new section? Sometimes you have to know the content to make that determination.
Stefan Schnabel	I would spend an additional expandable section "Examples" as part of the section list "Functional categories", "Critical Errors" and "Rating"
Aimee Ubbink	Please note my comments are based on accessible documents presented online, personal and confidential as well as public content. 1) The Platform section includes reference to documents, but the Technology section does not. While WCAG is based on HTML those rules apply to documents published online albeit they will be addressed in those document base languages. There has been some confusion in the past (based on my experience) with the PDF Techniques and the 'rest' of WCAG as it applies to document conformance. Perhaps it is my misunderstanding of the Technology section. 2) When is it appropriate to apply the <h> (heading vs. H1) tag? For scenarios such as letters or short content which may not contain a typical heading structure, further guidance would be good. IE -> When a document does not contain heading structure [provide example] the <h> may\|can\|must\|should be used to be identify the purpose of the document. 3) Test for accuracy of heading levels: Item 3 -> Avoid having two headers immediately adjacent with no content in between them. I have seen this use many times in statements, invoices, and other personal/confidential materials. This is typically due to the documents having been formatted for print and/or are legacy documents and is sometimes unavoidable. The use of the term ‘avoid’ does not identify a pass or fail. Can this be further clarified for compliance?
Makoto Ueki	1. Method: Tests tab - We should put a note in "Using ARIA role=heading", something like "Use h1~h6 elements if it is HTML." 2. Method: Tests tab - In the "Using h1-h6 to identify headings", we need to clarify "2. how Check that heading markup is not used when content is not a heading" will affect the test results. 3. Method: Tests tab - I'm wondering if we could "Test that each heading can be used for navigation purposes" and how it can be repeatable by multiple testers.
Michael Gower	I've kept my comments mainly conceptual. is it possible that "Structured content" is a level higher than some of these other guidelines? (cardinal relationships not normalized?) I feel like other things, such as lists, are structured content, but they've been left out in the wording "Use headings, sections, and sub-headings to organize your text." For example, paragraphs structure content. I believe this may be because this is more about Labelling Content Structure than it is about structuring content? I also noticed that in one draft, this is written as "Structured content (headings)". Does that infer that there are multiple Structured content guidelines? But then, the same thing has been done with "Visual contrast of text (color contrast)" and there doesn't seem to be another parenthetical thing to add to that... To build on why I think something like Labelling Content Structure _might_ be word a bit better, think of a doc title. Is a title a heading? You almost have to resort to a technological analysis to answer this. But visually, one can make some case that titles tend to be a label for the content. I mentioned this on the last call, but an absence of a heading may not be an impediment to use. A surplus of them may be. So, even with the context of talking about headings, there is a risk trying to define what is sufficient -- and an even greater risk awarding points on the number used. I suspect there are internationalization considerations that would make coming up with such numbers very challenging as well.