w3c/wbs-design
or
by mail to sysreq
.
The results of this questionnaire are available to anybody.
This questionnaire was open from 2022-07-01 to 2022-07-11.
16 answers have been received.
Jump to results for question:
The current editor's note that would be posted with the protocols content includes the following items. As a reminder, the editor's note is intended to capture questions, challenges, and other points that need to be addressed as exploratory content increases in stages of maturity. This list is independent of any particular proposal.
What changes or additions would you like to see?
Choice | All responders |
---|---|
Results |
Responder | Editor's Note | |
---|---|---|
John Foliot | ||
Rachael Bradley Montgomery | ||
Jennifer Strickland | There's a use of headings and labels as the example. This may need to be changed. My sub-group, previously known as Headings, now Structured Content, is in the middle of working on WCAG 3 outcomes, etc., and my partner has been working on a draft for about a year now. While Headings was seen as something that is solid and taken for granted my partner found issues with the draft provided so TBD what the guideline content will look like. Is there anything else that is actually stable? Something I like about Protocols is that the W3C does seem to have a shortage of volunteers and this allows organizations to contribute ideas that could become incorporated — although I wonder how the W3C will discover organizations' protocols, how WCAG could adopt them in the future if those orgs are not W3C members. Just some questions that might arise. | |
Todd Libby | ||
Shawn Lauriat | ||
Rain Breaw Michaels | I'm having a really difficult time know what I'm meant to respond to in this question. Having read these documents, though, I'm not sure that Making Content Usable should be considered a protocol since it has a combination of things that we can and should make testable, as well as things that maybe cannot become actual tests/specifications. It is also too big and diverse in its guidance to be a protocol. I would think that protocols would be more tightly scoped and not include things that should really be part of the testable specifications. For example, "ways to provide access to help" might be a candidate for a protocol, while all of Making Content Usable is too much. What are the next steps and meaningful, straightforward ways we can contribute/help? | |
Gundula Niemann | In either proposal, explain clearly what is meant by protocol or even find a different term. The term 'protocol' is widely used for different terms, like communication protocol like TCP/IP or UDP. It can also be used for the set of interfaces to be supported.Or for the way how to behave in front of the English Queeen. | |
Michael Cooper | ||
Jeanne F Spellman | Recommend: How do we address the range of potential ways that the protocol approach could be mis-used to claim accessibility? That organizations can write their own protocols following requirements set out in WCAG3 Outcomes. That organizations make a public statement as proof that an organization adhered to a protocol. | |
Michael Gower | > Is the intent of protocols to reference documents that include requirements that can be tested with high inter-rater reliability along with those that cannot? Yes, I think so. >How is it linked? What is the format of that asserstion/statement? (EARL?) These seem like implementation details that can be resolved once the overall approach and scope is defined. I don't think we need to get into that yet. >How do we ensure a protocol solves real problems? I'm not sure the AGWG has to. It depends a lot on what the protocols are built on -- what research, etc., and what the desired result is. It's entirely possible, under different scenarios (different use cases, different core user groups, etc) for the benefit/efficacy of any protocol to be judged completely differently. > Can they be used not for a claim of conformance, but an internal benchmark, for example? There are so many ways people may seek to claim benefit for people with disabilities beyond WCAG measures. Just think of many sites' best practices documentation. Little of this is offered to 'pass' or 'meet' WCAG. It is written and followed to improve user experience. We only need to think about design principles and practices, which have all developed not within standards area like WCAG, but within user-centered principles and philosophy. | |
Gregg Vanderheiden | - what measures do we have in place to keep an organization from 'gaming' these - how do we test that the assertions are real | |
Makoto Ueki | - Is it possible to localize it into different languages? WCAG is the international guideline. If it will be English-based, the instructions for localizations must be provided so that different countries/languages could adopt WCAG. | |
Laura Carlson | ||
Mary Jo Mueller | Eliminate either “Once the conformance model is developed, how will protocols fit into it?” or “How will protocols fit into conformance?” which ask the same question. Consider eliminating any conformance-related questions for now since conformance hasn’t yet been defined for WCAG 3. Questions use different terms that I thought were synonyms, but seem to imply they are not. E.g “public assertions” vs. “assertions” vs. “declarations”. General comments: There seem to be too many questions over too broad of a range of topics for this early on – very overwhelming. Some should be reserved until one or two exemplars of protocols are developed. Suggest focusing the questions on a small number of major topics that will help with rapid development of more detailed info for what a protocol is (and what it is not), how it might be used, and what a good protocol includes. For example, there could be a group of questions that focus on what might be considered essential parts of a protocol. - Do protocols include processes, procedures, best practices, tools to use for implementation? - Do protocols include “checkpoints” or simply procedures for evaluating results of following the protocol? - What are the preferred methods of documenting results? Should these documentation methods be required or recommended? | |
Wilco Fiers | From what has been provided so far, I did not feel well enough equipped to answer any of these questions. I feel the groups have gone into so much detail already that it is hard to say where to go next. I feel like it might be better to take a few steps back and come up with a more high-level proposal. Introduce the general idea as exploratory, without all the details currently attached to it. | |
Bruce Bailey |
After reviewing the proposals, how would you like to proceed?
This question will inform the draft exploratory content that is brought to the AGWG for review. A future survey will vote on adding draft content as exploratory.
Choice | All responders |
---|---|
Results | |
Add both proposals as exploratory | 5 |
Add the Protocols and Assertions proposal as exploratory | 2 |
Add the Evaluating Procedures proposal | 1 |
Add exploratory content using ideas from each of the proposals (The next questions provide an opportunity to indicate preference for specific ideas) | 6 |
Responder | Preferred starting point | Comments |
---|---|---|
John Foliot | Add the Protocols and Assertions proposal as exploratory | |
Rachael Bradley Montgomery | Add exploratory content using ideas from each of the proposals (The next questions provide an opportunity to indicate preference for specific ideas) | |
Jennifer Strickland | Add both proposals as exploratory | |
Todd Libby | Add the Protocols and Assertions proposal as exploratory | |
Shawn Lauriat | Add both proposals as exploratory | The two proposals solve very different problems and have unrelated mechanisms, both of which seem valuable to explore in more detail. |
Rain Breaw Michaels | Add exploratory content using ideas from each of the proposals (The next questions provide an opportunity to indicate preference for specific ideas) | |
Gundula Niemann | Add both proposals as exploratory | |
Michael Cooper | Add the Evaluating Procedures proposal | |
Jeanne F Spellman | Add exploratory content using ideas from each of the proposals (The next questions provide an opportunity to indicate preference for specific ideas) | |
Michael Gower | Add exploratory content using ideas from each of the proposals (The next questions provide an opportunity to indicate preference for specific ideas) | |
Gregg Vanderheiden | there is not enough information here to understand this - other than at an aspirational level. Needed are - examples (lots of them - at least enough to cover the range of topics) - examples of how they are scored and how the scores are used - discussion of how to meet the all important question of "Will points equate to actual impact" | |
Makoto Ueki | Add exploratory content using ideas from each of the proposals (The next questions provide an opportunity to indicate preference for specific ideas) | |
Laura Carlson | Add both proposals as exploratory | |
Mary Jo Mueller | Add exploratory content using ideas from each of the proposals (The next questions provide an opportunity to indicate preference for specific ideas) | |
Wilco Fiers | ||
Bruce Bailey | Add both proposals as exploratory |
Who should write a protocol?
Choice | All responders |
---|---|
Results | |
Subject domain experts write protocols and submit them for AGWG approval prior to use. | 4 |
AGWG defines requirements that protocols must meet in order to be used. | 4 |
Something else (Details in comments) | 7 |
Responder | Writing protocols | Comments |
---|---|---|
John Foliot | Subject domain experts write protocols and submit them for AGWG approval prior to use. | |
Rachael Bradley Montgomery | Something else (Details in comments) | I think AG should approve and maintain a list of approved protocols but I can see situations such as Usability testing where the approved protocol may be more like a structured set of requirements that would be used to create a more detailed protocol. |
Jennifer Strickland | AGWG defines requirements that protocols must meet in order to be used. | AGWG already has too few volunteers for there to be an approval process. Meeting WCAG is already a confusing experience for many users, so to add a submission process that will likely take AGWG a long time to get to will only make the public impression of WCAG more cumbersome. |
Todd Libby | Subject domain experts write protocols and submit them for AGWG approval prior to use. | |
Shawn Lauriat | Something else (Details in comments) | We should first get the mechanisms and details sorted out before we try to establish this aspect of responsibility. |
Rain Breaw Michaels | Something else (Details in comments) | I wish a "both" option were available. It seems that in order for subject domain experts to be able to write meaningful protocols, they need a defined set of requirements from AGWG to guide them. At the same time, it makes sense for subject domain experts to be the authors of the protocols. Additionally, there should be intersectional domain representatives who are part of drafting protocols, since accessibility requirements are not necessarily always in neat and separated packages. |
Gundula Niemann | Something else (Details in comments) | Any review by AGWG for all companies worldwide will not scale. |
Michael Cooper | Something else (Details in comments) | I think both options are valid. No matter who writes them, AG WG will have to define requirements for protocols in order to have something that can work with the established conformance model. The requirements could include a level of expertise, research, or review required to write a conforming protocol. In that case we should have a reasonable confidence in protocols we didn't write ourselves. |
Jeanne F Spellman | AGWG defines requirements that protocols must meet in order to be used. | |
Michael Gower | Something else (Details in comments) | I think both of the first options can take place. But I anticipate many more specific use cases within locations/jurisdictions, industry, user groups, etc., than could ever be fulfilled or curated by the AGWG. Just focusing on localization for a moment, I believe I just saw an announcment that the W3C has released a French language version of WCAG 2.1. How long will it take for the AGWG to vet language style guides for every language (assuming an ideal world where only one style guide for language existed). |
Gregg Vanderheiden | Subject domain experts write protocols and submit them for AGWG approval prior to use. | - the AGWG needs to maintain quality control of this *especially* since it is new and untried and we don't know all the ways it could go astray. |
Makoto Ueki | Something else (Details in comments) | Both. 1) AGWG must define the requirements for the protocols, 2) Subject domain experts write protocols and submit them and 3) AGWG will review and approve them. |
Laura Carlson | Subject domain experts write protocols and submit them for AGWG approval prior to use. | |
Mary Jo Mueller | AGWG defines requirements that protocols must meet in order to be used. | I think the AG WG has enough work to do to keep up with current technology, make improvements to the standard, develop techniques and ACT Rules, etc. I don't think it is the position of the W3C-WAI to approve every web developer's protocol for every SC. AG WG doesn't approve every test process or implementation, and protocols seem to fall into a similar category. |
Wilco Fiers | ||
Bruce Bailey | AGWG defines requirements that protocols must meet in order to be used. | AGWG might write/develop some model protocols, but "AGWG approval prior to use" seems problematic. |
Protocols are used when the results can’t be tested with high inter-rater reliability. What additional characteristics define what is accepted as a protocol?
Choice | All responders |
---|---|
Results | |
Protocols contain an articulated outcome, key expectations, and user stories or equivalent examples. | 3 |
The protocol meets specific outcome-level requirements, which will be outlined in WCAG 3 for each outcome that allows the use of protocols. | 2 |
Both | 8 |
Something else (Details in comments) | 1 |
Responder | Characteristics of a protocol | Comments |
---|---|---|
John Foliot | Protocols contain an articulated outcome, key expectations, and user stories or equivalent examples. | |
Rachael Bradley Montgomery | Both | I do not see these as mutually exclusive and both have value |
Jennifer Strickland | Both | |
Todd Libby | Both | |
Shawn Lauriat | Protocols contain an articulated outcome, key expectations, and user stories or equivalent examples. | For the second option of this question, I don't think I understand what it proposes, so examples would help me to form an opinion about it. |
Rain Breaw Michaels | Both | |
Gundula Niemann | ||
Michael Cooper | The protocol meets specific outcome-level requirements, which will be outlined in WCAG 3 for each outcome that allows the use of protocols. | |
Jeanne F Spellman | The protocol meets specific outcome-level requirements, which will be outlined in WCAG 3 for each outcome that allows the use of protocols. | |
Michael Gower | Both | I don't think a protocol would need ALL these things to qualify as a protocol. I also feel like the above is missing a procedural element which seems to me to be a core part of many protocols: do this, at this time. |
Gregg Vanderheiden | Both | |
Makoto Ueki | Both | |
Laura Carlson | Protocols contain an articulated outcome, key expectations, and user stories or equivalent examples. | |
Mary Jo Mueller | Something else (Details in comments) | The protocols should have a template to follow for essential parts. This could be a process or best practices used by design, development and/or test in support of desired end user accessibility outcomes in the content. |
Wilco Fiers | ||
Bruce Bailey | Both |
Conformance has not yet been defined so this topic requires future exploration. How may protocols fit into conformance in the future?
Choice | All responders |
---|---|
Results | |
Users earn points by adopting vetted protocols increasing their final conformance score. | 2 |
Protocols describe inputs for procedures but do not necessarily measure outcomes. The protocol must define a way to evaluate the quality of its implementation. This is expected to correlate to conformance with a WCAG 3 outcome. | 3 |
Something else (Details in comments) | 9 |
Responder | Future conformance | Comments |
---|---|---|
John Foliot | Users earn points by adopting vetted protocols increasing their final conformance score. | |
Rachael Bradley Montgomery | Something else (Details in comments) | I would prefer specify this after we better understand where we are going with conformance. |
Jennifer Strickland | Something else (Details in comments) | Protocols are not part of a baseline passing conformance score — so they cannot be "gamed" to pass. Alternatively, protocols are a required part of a baseline conformance score — as in, users must document some steps used to evaluate the subjective WCAG3 criteria. |
Todd Libby | Something else (Details in comments) | Wait until Conformance is defined so that we can determine the answer to "how many protocols fit into conformance in the future". |
Shawn Lauriat | Something else (Details in comments) | Too early to establish this, we should focus on the mechanisms and then revisit how this fits into conformance as we figure out conformance itself. |
Rain Breaw Michaels | Protocols describe inputs for procedures but do not necessarily measure outcomes. The protocol must define a way to evaluate the quality of its implementation. This is expected to correlate to conformance with a WCAG 3 outcome. | |
Gundula Niemann | ||
Michael Cooper | Protocols describe inputs for procedures but do not necessarily measure outcomes. The protocol must define a way to evaluate the quality of its implementation. This is expected to correlate to conformance with a WCAG 3 outcome. | |
Jeanne F Spellman | Protocols describe inputs for procedures but do not necessarily measure outcomes. The protocol must define a way to evaluate the quality of its implementation. This is expected to correlate to conformance with a WCAG 3 outcome. | AGWG doesn't have the bandwidth to vet protocols. |
Michael Gower | Something else (Details in comments) | In the same way that SCs are genearlly written first and afterward assigned a level (based on how the SC matures), I think the manner in which any protocol would be assessed could vary. It's also not entirely clear to me at this point whether scoring will exist in the form first pictured. But let me elaborate on a possible parallel path: at the moment the ITI template allows someone to report a site or application against many critiera. Some protocols could potentially be just another list of outcomes against which someone reports. Groups that have an interest in assessing the maturity and overall user experience could adopt their own scale for how they value things. RFIs could requirement a baseline level (including a set of w3c protocols that must be followed), and then ask groups to identify which additional features they provide based on a list of protocols (or equivalents), |
Gregg Vanderheiden | Something else (Details in comments) | Earn points to increase their RATING or SCORE. Increasing conformance sounds discordant. I have trouble with conforming as a variable. You either conform to a standard (or a level of a standard) or you don't. I'm not frozen on this - but I still have not heard a scoring proposal that doesnt have the problems that the scoring proposals we advanced in the early 2000's had - that caused us to finally give up on scoring as an approach. it sounded good to us to -- but we could never make it work. |
Makoto Ueki | Something else (Details in comments) | It is too early to discuss this at this phase. We need to get to more concrete and specific guidelines/outcomes/conformance scheme at first. And then we can revisit this question. |
Laura Carlson | Users earn points by adopting vetted protocols increasing their final conformance score. | |
Mary Jo Mueller | Something else (Details in comments) | Difficult to tell before we have any protocols to look at. If you look at the plain language protocol from the U.S. Govt. it's a good resource with great advice, but it is still difficult to provide some definitive way to score or measure against the WCAG requirement's outcome. If a protocol is a new method to try to measure "perfection" in following it and "perfection" in the protocols themselves, I fear that adding them will only increase the frustration in trying to meet WCAG requirements. |
Wilco Fiers | ||
Bruce Bailey | Something else (Details in comments) | I am not yet convinced these are not compatible. Obviously, WCAG3 needs different name/term for the approaches. |
How stringent are the controls?
Choice | All responders |
---|---|
Results | |
Content creators make a programmatically linked declarative statement commiting to adopt a protocol. The statement will note steps that taken to meet the protocol. Content creators make a declarative statement of progress. | 2 |
The protocol can only be used for a subset of outcomes that must meet W3C criteria, and must be able to be evaluated in order to be used for conformance. Protocols used to support conformance claims must be publicly documented. | 2 |
Something else (Details in comments) | 9 |
Responder | Controls | Comments |
---|---|---|
John Foliot | Content creators make a programmatically linked declarative statement commiting to adopt a protocol. The statement will note steps that taken to meet the protocol. Content creators make a declarative statement of progress. | |
Rachael Bradley Montgomery | Something else (Details in comments) | I am unsure of the difference here. I support public documentation through some form or programmatically linked statement. |
Jennifer Strickland | Something else (Details in comments) | Neither of the above options seem viable for uses. Could AGWG attempt to survey the user audience to uncover what users would be able / willing to do? |
Todd Libby | Content creators make a programmatically linked declarative statement commiting to adopt a protocol. The statement will note steps that taken to meet the protocol. Content creators make a declarative statement of progress. | |
Shawn Lauriat | Something else (Details in comments) | Too early to establish this, we should focus on the mechanisms and then revisit how this fits into conformance as we figure out conformance itself. |
Rain Breaw Michaels | The protocol can only be used for a subset of outcomes that must meet W3C criteria, and must be able to be evaluated in order to be used for conformance. Protocols used to support conformance claims must be publicly documented. | |
Gundula Niemann | ||
Michael Cooper | Something else (Details in comments) | Protocols used as part of a conformance claim must be publicly documented, allowing for vetting. The organization indicates in the public conformance claim that it is using the protocol as part of its conformance efforts, and may do so sooner to show effort prior to achieving WCAG 3 conformance. The question of whether the protocol needs to be programmatically linked should be moved to the discussion about how conformance claims are expressed. I'm not sure how tightly we should control where protocols can be used. I can imagine deciding that some outcomes are "protocol outcomes" instead of "method outcomes", but I think that misses a lot of gray areas. I'd like to see some prototype protocols, and a collection of outcomes that we think need protocols, before we make too many decision about how these should inter-relate. When organizations are not making formal WCAG conformance claims, we have no way to enforce any of this. Most orgs will take on protocols in good faith, and we should allow protocols to be useful when conformance claims are not being made. |
Jeanne F Spellman | The protocol can only be used for a subset of outcomes that must meet W3C criteria, and must be able to be evaluated in order to be used for conformance. Protocols used to support conformance claims must be publicly documented. | There are no consequences to declaring a protocol and not meeting it, except a lawsuit. If the protocol is used to gain points to claim minimum conformance, there is a high incentive to cheat. |
Michael Gower | Something else (Details in comments) | There is an important balance here between effort to prove and track, versus effort to actually improve the content. The heavier documentation lift there is, the less likely teams will adopt. |
Gregg Vanderheiden | Something else (Details in comments) | you had AGWG approval before but I don't see it here I think you need both of the above as well. |
Makoto Ueki | Something else (Details in comments) | This is also too early. Both will be needed. |
Laura Carlson | Not sure. | |
Mary Jo Mueller | Something else (Details in comments) | Putting strict and detailed controls around documentation of using a protocol will make this very cumbersome. I thought one of the goals of having protocols to be able to require and include some of the less easily implementable and testable criteria and encourage adoption. The heaver weight this whole process is, the less likely teams will be able to step up to accomplish it. |
Wilco Fiers | ||
Bruce Bailey | Something else (Details in comments) | To be determined, IMHO. |
When is the protocol applied?
Choice | All responders |
---|---|
Results | |
Protocols and assertions qualifies principles and outcomes. Protocols and Assertions leaves the solution to the contextual need of the entity who adopts the Protocol. The goal here is to get vetted, “educational” content in front of dev teams at the earliest stages - before any content is created. | 3 |
Orgs adopt protocols as part of developing their WCAG 3 conformance plan and execution. Evaluation of how well the protocol was followed is done when evaluation towards a conformance claim is done, or at other times as part of the organization’s Quality Assurance (QA) process. | 5 |
Something else (Details in comments) | 5 |
Responder | Timing | Comments |
---|---|---|
John Foliot | Protocols and assertions qualifies principles and outcomes. Protocols and Assertions leaves the solution to the contextual need of the entity who adopts the Protocol. The goal here is to get vetted, “educational” content in front of dev teams at the earliest stages - before any content is created. | |
Rachael Bradley Montgomery | Something else (Details in comments) | I agree with getting protocols in front of dev teams early. I think that is true of all parts of the standards but many protocols will be significantly more effective when adopted early. If we emphasize that, I am not sure how we would enforce that or handle situations where the protocol is adopted mid process or later on as a remediation. I think final evaluation would be during the conformance claim and that some evaluation of how well the org follows the protocol is needed. |
Jennifer Strickland | Something else (Details in comments) | Neither of the above options seem viable for uses. Could AGWG attempt to survey the user audience to uncover what users would be able / willing to do? |
Todd Libby | Protocols and assertions qualifies principles and outcomes. Protocols and Assertions leaves the solution to the contextual need of the entity who adopts the Protocol. The goal here is to get vetted, “educational” content in front of dev teams at the earliest stages - before any content is created. | |
Shawn Lauriat | Something else (Details in comments) | This seems like something we should write up afterward as a part of something like updated WCAG-EM documentation. |
Rain Breaw Michaels | Something else (Details in comments) | I'm having difficulty understanding the options above, but am very in favor of this statement: "The goal here is to get vetted, “educational” content in front of dev teams at the earliest stages - before any content is created." |
Gundula Niemann | ||
Michael Cooper | Orgs adopt protocols as part of developing their WCAG 3 conformance plan and execution. Evaluation of how well the protocol was followed is done when evaluation towards a conformance claim is done, or at other times as part of the organization’s Quality Assurance (QA) process. | |
Jeanne F Spellman | Orgs adopt protocols as part of developing their WCAG 3 conformance plan and execution. Evaluation of how well the protocol was followed is done when evaluation towards a conformance claim is done, or at other times as part of the organization’s Quality Assurance (QA) process. | Getting educational content in front of dev teams is a good goal, but it should not be tied to conformance, which is a measure of what is done. This belongs elsewhere with a different incentive than conformance score. |
Michael Gower | Orgs adopt protocols as part of developing their WCAG 3 conformance plan and execution. Evaluation of how well the protocol was followed is done when evaluation towards a conformance claim is done, or at other times as part of the organization’s Quality Assurance (QA) process. | I don't see these as diametrically opposed scenarios. |
Gregg Vanderheiden | Protocols and assertions qualifies principles and outcomes. Protocols and Assertions leaves the solution to the contextual need of the entity who adopts the Protocol. The goal here is to get vetted, “educational” content in front of dev teams at the earliest stages - before any content is created. | |
Makoto Ueki | Orgs adopt protocols as part of developing their WCAG 3 conformance plan and execution. Evaluation of how well the protocol was followed is done when evaluation towards a conformance claim is done, or at other times as part of the organization’s Quality Assurance (QA) process. | |
Laura Carlson | Not sure. | |
Mary Jo Mueller | Something else (Details in comments) | I think the timing is dependent on what the protocol is/how it is used. For example, if it is for developing understandable textual information (good alt text, plain language, etc.) the protocol could be used at time of development - giving guidance on steps to develop and steps to check the content. If I look at the example of the U.S. government's plain language information, there is part of it that descries steps to take for development (which would occur earlier in the dev process) and steps to take to test that the objectives of the requirement (to have content in plain language) that would need to occur after implementation. |
Wilco Fiers | ||
Bruce Bailey | Orgs adopt protocols as part of developing their WCAG 3 conformance plan and execution. Evaluation of how well the protocol was followed is done when evaluation towards a conformance claim is done, or at other times as part of the organization’s Quality Assurance (QA) process. |
The following persons have not answered the questionnaire:
Send an email to all the non-responders.
Compact view of the results / list of email addresses of the responders
WBS home / Questionnaires / WG questionnaires / Answer this questionnaire
w3c/wbs-design
or
by mail to sysreq
.