Maturity Labeling Process

From WCAG WG

Documents

  • WCAG 3 Schedule to manage the overall work. This will include:
    • Topics raised in meetings and issues mapped to milestones
    • Links to issues
  • GitHub to manage issues, including milestone assignments that map to the schedule.
  • Editors draft / Sandbox to show the early work and direction to specific audiences. The Editor's draft will show Placeholder, Exploratory, Developing, Refining, and Mature content with labels and notes. A filter will be available that will allow the document to be filtered by stage of maturity and to see it without the notes. The editor's draft will include a warning about the changing nature of content and an explanation of the filter.
  • Working draft to show progress to the public. This will include Refining and Mature content, and some Developing content. Some Placeholder and Exploratory content will be included if the group approves.
  • Previous draft of this process AG process

Overview of process

Comparison by level of Maturity Labeling Process
Overview Placeholder Exploratory Developing Refining Mature
What it means AG has identified we need content but do not yet know what it should look like AG is exploring one or more possible directions for this content AG has high confidence in the direction and some confidence in the details AG has high confidence in the direction and moderate confidence in the details Content is believed to be ready to become a W3C Recommendation
Goal Identify needs and possible directions. Ensure content phrasing is appropriate to placeholder stage Document direction(s) Work out details and address open questions Get wide stakeholder feedback Refine and finalize
Likelihood to change Definite Very Likely Details likely to change, overall direction unlikely to change Details may change, overall direction unlikely to change Unlikely to change
Location public views content Editor’s Draft Editor’s Draft Editor’s Draft in the beginning, Working draft as it progresses to get confirmation on the direction Editors and Working Draft Editors and Working Draft
Starting point A subgroup or the working group identifies placeholder content. The subgroup develops proposals for review by the working group. This is done using wikis, google docs or whatever the subgroup finds useful to start with. It then becomes a PR to be added to the Editor’s draft.. The subgroup will use concerns raised during the exploration approval and related github issues to refine the content in the draft. After feedback from key stakeholders has been received, the subgroup will use that and the feedback from the working group to refine the content in the draft. The subgroup will use the feedback from the working group as well as any feedback from wide reviews (issues in Github) to refine the content in the draft.
Process details When the working group identifies placeholder content, the chairs will set up a subgroup or assign the placeholder content to an existing subgroup.
  • Subgroup develops initial proposal and gets feedback, usually via email process
  • Subgroup submits PR to working group including initial editor’s note
  • Survey working group
  • Subgroup works through github issues and comments in editor’s note
  • Subgroup submits revised PR to working group including a revised editor’s note
  • Some expert and key stakeholder feedback may be solicited as work progresses
  • Survey working group
  • Agreement in meeting to updates.
  • Subgroup works through github issues and comments in editor’s note
  • Subgroup submits revised PR to working group including a revised editor’s note
  • Increasingly wider feedback as work progresses
  • Survey working group
  • Agreement in meeting to updates.
  • Subgroup works through feedback from wide reviews (issues in Github) to refine the content
  • Subgroup submits revised PR to working group and discusses the results in meeting
  • Survey working group
  • Agreement in meeting to updates.
Editor’s note Usually none All outstanding questions, concerns, possible directions, etc. Include link to outstanding issues. Questions that need broader feedback and details that still need work. Questions that need broader feedback and details that still need work. Usually none
Approval to add to Editor’s Draft Agreement to the need and the initial placeholder text Agreement to explore direction and editor’s note content (see decision tree below) Agreement that the direction is correct and editor’s note content (see decision tree below). Agreement that the direction is correct, wording of direction is correct, most details are likely correct, and editor’s note content (see decision tree below). Agreement that the direction and details are correct and wording of direction and details is all correct. Publication Ready
Approval to add to Working Draft
  • Not common
  • Done by CFC after discussion at meeting
  • Not common
  • Done by CFC after discussion at meeting
  • Added to Working Draft as it develops with an editor’s note that we are looking for feedback on the direction
  • Done by CFC after discussion at meeting
  • Chairs send CFC once added to editor’s draft
  • Proof of concept(s)
  • Chairs send CFC once added to editor’s draft
Content removal By working group agreement or after 6 months if no content comes forward to the working group By working group agreement or after 6 months if no content comes forward to the working group By working group agreement in the editors draft and CFC in the Working draft By working group agreement in the editors draft and CFC in the Working draft By working group agreement in the editors draft and CFC in the Working draft

Decision tree for adding exploratory content to editor’s note during review

Step 1: Is the change presented in concrete terms, either as a PR or a specific change to a small amount of text (typo, replacement of text)?

  • If yes, continue to 2
  • If no, add concern to editor’s note or github issue

Step 2: Ask if anyone has objections or if any discussion is needed.

  • If no, make change
  • If yes, add concern to editor’s note or github issue

Additional Information about Editor’s note

In general, the editor’s note should be used for capturing general areas that need to be discussed further and github should be used for more detailed changes. The line between these can be blurry so chairs expect the editor’s note at the exploratory stage to include a variety of comments, questions, and concerns. Once content moves past exploratory the editor's note should be a more concise set of core questions that will make sense to the public. Everything else should be resolved or in Github issues.

Long content should be documented in github and a short summary placed in the editor’s note.

Content for the Editor's notes comes from the subgroup, surveys, and the AG meeting. If additional concerns or questions come up after the meeting, the participant can those concerns or questions to email group-ag-plan@w3.org. The editors will use the decision tree to add the comments to github as an issue and/or to the editor's note.

All editor’s notes should link to the related github issues.

Asynchronous Process

  • Content is sent to the ag-admin email list
  • If content receives no feedback or only +1 support within 7 days, editors will take the next step (close the issue(s), create a PR (if one does not yet exist), merge the PR into the editor's draft, etc.) and email confirmation and a link back to the list closing out the thread.
  • Content that received feedback or -1s that can’t be quickly resolved will go back to the subgroup working on it or move forward to a working group meeting for more in-depth discussion.

Acronyms and Definitions

  • AG - Accessibility Guidelines Working Group
  • CFC - Call for Consensus. The process of emailing out a decision after working group agreement for the full email list to consider. The formal objection process is used.
  • PR - Pull Request. A request to merge work into Github.
  • Working group agreement - Agreement reached in a meeting or through an asynchronous method (see outstanding decisions below). Consensus will be used to reach working group agreements Concerns that can't be resolved will be noted as editor's notes.