W3C

Site Navigation

Nearby

Results of Questionnaire Assertions text

The results of this questionnaire are available to anybody.

This questionnaire was open from 2023-01-12 to 2023-01-23.

10 answers have been received.

Jump to results for question:

  1. What is a procedure
  2. Next Step
  3. DONE: Documentation needed
  4. Level Assertions Apply
  5. Approval to add content

1. What is a procedure

summary | by responder | by choice

Different proposals for assertions have relied on different approaches to what a "procedure" is. This ranges from assertions requiring a reference to publicly published guidance like plainlanguage.gov or a W3C Note that WCAG has approved the stricter side to allowing an organization to write its own procedure based on some guidance in WCAG 3 on the less strict side. These are not mutually exclusive. The proposed text we reviewed last week allowed two options. The possibilities that have been proposed are below. Please indicate all which you would support.

Summary

ChoiceAll responders
Results
A procedure must reference publicly published guidance listed in WCAG. WCAG would maintain a list of approved procedures. 2
A procedure must reference publicly published guidance that meets the criteria listed in WCAG. In this case, WCAG would not maintain a list but rather a set of criteria for "good" guidance. 3
A procedure must follow the guidance listed in WCAG. See Example Outcome Evaluation Procedure. from the current editor's draft. 2
A procedure can be any process that improves accessibility listed in WCAG but guidance on applying it is up to the organization 3
A procedure can be any process that improves accessibility. 2

Skip to view by choice.

View by responder

Details

Responder What is a procedureComments
E.A. Draffan
Wilco Fiers
Jaunita George
Mary Jo Mueller
Gregg Vanderheiden
  • A procedure must reference publicly published guidance listed in WCAG. WCAG would maintain a list of approved procedures.
  • A procedure must reference publicly published guidance that meets the criteria listed in WCAG. In this case, WCAG would not maintain a list but rather a set of criteria for "good" guidance.
I checked top two boxes (rather than just the top box) because, although #2 lets a company game it --it has some controls and we are trying to encourage a company to do more - not force them to so something specific. Also lets this grow organically. AND we will never keep #1 up - if we ever get it up to start with.

I don't understand #3 - and looking at it raised lots of questions (see below)
#4 an#5 are so vague that one can do anything - and count it. So Silver gold would lose meaning. and I can see no end to arguments.


In item 3
Can we not use *outcome* along with *procedure* since we say that procedures do not necessarily have outcomes. Very confusing. An outcome evaluation procedure should be a way of testing an outcome - which is the ordinary interpretation of that phrase.
Also following the link in item 3 we find "Users can obtain help and support". This sounds like an outcome. but procedures don't have outcomes. (Also of course this uses "user" so you can't test it unless you specify which user - all users? - a user? users with what abilities? etc.


Jeanne F Spellman
  • A procedure must reference publicly published guidance listed in WCAG. WCAG would maintain a list of approved procedures.
  • A procedure must reference publicly published guidance that meets the criteria listed in WCAG. In this case, WCAG would not maintain a list but rather a set of criteria for "good" guidance.
  • A procedure must follow the guidance listed in WCAG. See Example Outcome Evaluation Procedure. from the current editor's draft.
I have mixed feelings about the second option. The critieria for "good" guidance will be difficult to write and maintain and may constrict organizations from using guidance that was inadvertently excluded.

I am very enthusiastic about the Example Outcome Evaluation. I'm not sure about the scoring -- I think I would need to think about that more, but I do think we should explore this example more.
Stefan Schnabel
  • A procedure can be any process that improves accessibility listed in WCAG but guidance on applying it is up to the organization
  • A procedure can be any process that improves accessibility.
Organizations should follow approved procedures but may use their own if needed, if so, they need to explain this.
Gundula Niemann
  • A procedure can be any process that improves accessibility listed in WCAG but guidance on applying it is up to the organization
  • A procedure can be any process that improves accessibility.
The first three should be recommended, but not be mandatory.
Bruce Bailey
  • A procedure can be any process that improves accessibility listed in WCAG but guidance on applying it is up to the organization
Alastair Campbell
  • A procedure must reference publicly published guidance that meets the criteria listed in WCAG. In this case, WCAG would not maintain a list but rather a set of criteria for "good" guidance.
  • A procedure must follow the guidance listed in WCAG. See Example Outcome Evaluation Procedure. from the current editor's draft.

View by choice

ChoiceResponders
A procedure must reference publicly published guidance listed in WCAG. WCAG would maintain a list of approved procedures.
  • Gregg Vanderheiden
  • Jeanne F Spellman
A procedure must reference publicly published guidance that meets the criteria listed in WCAG. In this case, WCAG would not maintain a list but rather a set of criteria for "good" guidance.
  • Gregg Vanderheiden
  • Jeanne F Spellman
  • Alastair Campbell
A procedure must follow the guidance listed in WCAG. See Example Outcome Evaluation Procedure. from the current editor's draft.
  • Jeanne F Spellman
  • Alastair Campbell
A procedure can be any process that improves accessibility listed in WCAG but guidance on applying it is up to the organization
  • Stefan Schnabel
  • Gundula Niemann
  • Bruce Bailey
A procedure can be any process that improves accessibility.
  • Stefan Schnabel
  • Gundula Niemann

2. Next Step

Last week, we agreed this proposal was moving in the correct direction. Do you approve moving this text into the WCAG 3 editor's draft as exploratory?

The next step would be to build examples and bring a pull request to the group.

Summary

ChoiceAll responders
Results
Yes, I approve moving this content into the editor's draft as exploratory. 5
Yes, I approve moving this content into the editor's draft as exploratory with the following changes (A the notes to add or text to change in comments) 1
No, I do not approve moving this content into the editor's draft. A different direction that has been proposed is a better starting point. (Indicate other prooposal in comments)
Something else

(4 responses didn't contain an answer to this question)

Details

Responder Next StepComments
E.A. Draffan
Wilco Fiers
Jaunita George
Mary Jo Mueller
Gregg Vanderheiden Yes, I approve moving this content into the editor's draft as exploratory with the following changes (A the notes to add or text to change in comments) Including this and getting comments -- is a good thing - and we should proceedl

It needs a bit of cleaning up - and we should remove things that are controversial or weak (maybe list them as "we are also considering" that come up at the meeting

Some thing to change that I noticed/ suggest:
1) inclusive design and Universal design are not procedures - they are philosophies or approaches. Not a process or procedures. [remove them or take some practices from them and put them in as procedures ]
2) "and possible at the bronze level" should be changed to "Note: we are also exploring if there is a way they might be included at the Bronze level or not." I think that expresses the level of interest and concern of the group - and would better solicit input. we could also add "and the WG solicits comments on this" to the end.
3) Finish this sentence so it is clear what is being deferred. WCAG 3 recommends additional supporting documentation that an organization can use to improve to support the procedure or assertion but defers [from requiring supplemental documentation beyond the items listed as required below] -- if that is what is meant. just guessing

Jeanne F Spellman Yes, I approve moving this content into the editor's draft as exploratory.
Stefan Schnabel Yes, I approve moving this content into the editor's draft as exploratory. Provided that "this proposal" means the "Assertions and Procedures" document
Gundula Niemann Yes, I approve moving this content into the editor's draft as exploratory.
Bruce Bailey Yes, I approve moving this content into the editor's draft as exploratory.
Alastair Campbell Yes, I approve moving this content into the editor's draft as exploratory.

3. DONE: Documentation needed

This draft assumes that organizations won't use assertions if too much documentation is required or if too much information must be made public. A third-party tester can only test content to which they have access.

Some commenters on last week's survey felt that additional documentation which proved the assertion is needed and this would likely be internal documentation available on request.

Which do you most agree with:

Summary

ChoiceAll responders
Results
Limit the documentation requirements to information about the assertion itself. Third-party testers would not have access to information to support the assertion. 3
Require additional public documentation to support the assertion. 2
Agree with requiring limited public documentation but request organizations maintain internal documentation to support the assertion that would be available on request when testing. 2
Something else 2

(1 response didn't contain an answer to this question)

Details

Responder DONE: Documentation needed
E.A. Draffan Require additional public documentation to support the assertion.
Wilco Fiers Something else I feel this is a regulatory question, not something WCAG should answer. It's more a question of transparency / openness of information than it is a question of accessibility.
Jaunita George Require additional public documentation to support the assertion. I still feel like we already have assertions that organizations make today that are largely public (accessibility statements and VPATs) that haven't been reliable and don't see how this would improve that situation unless we added some guardrails.
Mary Jo Mueller Limit the documentation requirements to information about the assertion itself. Third-party testers would not have access to information to support the assertion. I'm all for limiting burden of doing this - requiring documentation vs. voluntarily documenting. Organizations that care will likely document it as a matter of course. Organizations that don't probably won't bother making any assertions or go beyond what is absolutely required. By reducing required documentation, more might explore protocols and put them to good use.
Gregg Vanderheiden Limit the documentation requirements to information about the assertion itself. Third-party testers would not have access to information to support the assertion. My thoughts as we puzzle this through....

I think the documentation should be absolute minimum.
- the assertion
- the date the assertion is made
- the W3C listed protocol that the assertion is being done against.

Rationale
- you want people to embrace these
- the documentation is meaningless. Since it isnt outcome based (if it was it would be an outcome) there is no way to be meaningfully documented. Websites are changed daily. Saying that "we did this - and here are documents that you can't verify that says we did" -- is no better than saying "we did it".
- and requiring them to do a lot more documentation than that - will likely reduce the number of people who will bother to make an assertion -- or look at them - which means fewer that will do them.

- W3C should list the ones that count. allowing anything to be a procedure means that silver and gold can be gamed. Make up something - and assert. Do a number and done.

- Also they need to be more specific. Not big documents with lots of misc things in them. a) they then need to do everything? Some one thing in the doc? If a doc is full of recommendations -- does asserting that you followed it mean you did everything in the document? If you need to do everything it would be too much to try?
Jeanne F Spellman Agree with requiring limited public documentation but request organizations maintain internal documentation to support the assertion that would be available on request when testing.
Stefan Schnabel
Gundula Niemann Something else Agree with requiring limited public documentation but request organizations maintain internal documentation to support the assertion.
Bruce Bailey Limit the documentation requirements to information about the assertion itself. Third-party testers would not have access to information to support the assertion. I am okay with other choices, and would note that government agencies probably subject to Sunshine/FOIA (Freedom of Information Act) or equivalent -- so that is close to "available upon request".
Alastair Campbell Agree with requiring limited public documentation but request organizations maintain internal documentation to support the assertion that would be available on request when testing. I suspect that 3rd party testers would create a form for their clients to fill in that would help to fill in assertions for a conformance claim.

4. Level Assertions Apply

The argument for applying assertions to Silver and Gold levels (or something similar) is that assertions do not guarantee results so should be addressed at a higher conformance level than methods.

The argument for including assertions at the Bronze level (or something similar) is that some user needs are best supported by assertions so moving them to a higher level repeats the concerns expressed in A/AA and AAA division

Which is the lowest level assertions apply to (assuming they would also apply above that level)? Please explain your reasoning in the comments.

Summary

ChoiceAll responders
Results
Bronze 2
Silver 2
Gold 1
Something else 4

(1 response didn't contain an answer to this question)

Details

Responder Level Assertions Apply Comments
E.A. Draffan Something else It is so difficult to set a level when the issues at stake may have so many different ways of affecting outcomes
Wilco Fiers Something else Without actual outcomes for which assertions can be used, I think it's too early to say. I suspect WCAG 3 will not be equitable if we restrict it by level, but we'll just have to wait and see how things shake out I think.
Jaunita George Gold Silver may become the target standard and I feel like organizations should meet all target standards before receiving any points on a statement that could not be independently verified.
Mary Jo Mueller Bronze IMO, assertions COULD apply to any level. However, to achieve silver or gold, it seems assertions would be necessary or required.
Gregg Vanderheiden Something else I think it should only be applied at Silver or Gold levels of conformance.

BUT - I think both assertions and recommendations should be positioned mixed in with the outcomes so that you see them all when you see any of them. Even if they don't (or their company doesnt allow them to) make any assertions = just seeing them will put them in their mind and there is higher likelihood they will practice them - if not assert them. If we put them off in a separate section - they will be much less viewed
Jeanne F Spellman Bronze Assertions can have a broad use in WCAG3. Bronze level assertions can be specified clearly to meet a specific Bronze outcome and can be specified to provide specific amounts of verifiable documentation on request. An example could be using clear words. The required documentation could include the list of words that the organization is using.
Stefan Schnabel
Gundula Niemann Something else The terms 'assertion', 'process', 'protocol', and 'method' are defined and used in a very proprietary way throughout our discussions, so I cannot memorize the details. Unfortunately, not all definitions are repeated in the linked document.
Therefore I cannot answer this question.
Bruce Bailey Silver Including "web accessibility statements" is very close IMHO to an assertion, and those are not uncommon. I think that makes it a better fit for Silver.
Alastair Campbell Silver Silver / something else. If we uses the "badges" approach then presumably a site could include certain badges based on assertions that aren't sufficient to reach Silver?

I think we should have a level you can reach without assertions, so that would be bronze. However, we should aim for Silver to be the "national level" organisation level, which requires some assertions, then gold for more/better coverage.

5. Approval to add content

Do you agree to replace section 4.3.4 procedural test and 4.4.4 Outcome implementation procedures in the WCAG 3 editor's draft?

Summary

ChoiceAll responders
Results
I agree to add this content as exploratory 4
I agree to add this content as exploratory with the following changes 2
I do not believe this is an improvement over the current content in 4.3.4 procedural test and 4.4.4 Outcome implementation procedures so should not be added
Something else 3

(1 response didn't contain an answer to this question)

Details

Responder Approval to add contentComments
E.A. Draffan Something else I really am not sure as the 4.3.4 procedural test and 4.4.4 Outcome implementation procedures still have so many questions that remain unanswered and this draft document about Assertions seems to just add more complexity and imponderables.
Wilco Fiers I agree to add this content as exploratory with the following changes I'm not too clear on the difference between protocols and processes yet. I'd like a note saying we need work on if / how to define the boundary between them.
Jaunita George I agree to add this content as exploratory
Mary Jo Mueller I agree to add this content as exploratory with the following changes Are process, procedure, and protocol synonymous? It is unclear from the excerpted content in the document. While I agree this content can be added as exploratory, I think these definitions should be made clear if the terms have a different meaning. Otherwise it's a bit confusing.
Gregg Vanderheiden Something else Agree 4.3.4 with "procedure test" with "procedure assertion"

Disagree with 4.4.4 talking about *Outcome implementation procedure* -- this conflates outcomes and procedures. We should keep "outcomes" with "tests" and keep "procedures" with "assertions" and never have a sentence the conflates them like "outcome procedure" or "outcome assertion" or anything that conflates them. I will confuse people a lot.
Jeanne F Spellman I agree to add this content as exploratory
Stefan Schnabel
Gundula Niemann Something else I can't see two versions, so I cannot decide on a replacement.
Options 1 and 2 talk about adding, not about replacement.
Bruce Bailey I agree to add this content as exploratory
Alastair Campbell I agree to add this content as exploratory Under documenting assertions, I was wondering why the assertions wouldn't have the same scope as the conformance statement that they are presumably documented within? Would different assertions have different scopes? Perhaps, but shouldn't the default (easiest way) be for the same scope across assertions?

The transition to "protocol" is quite abrupt, would it help to say, instead of "Assertions may apply to protocols or processes.", something like: "An assertions can be about the use of a protocol or process." (or "is about".)

I'm still concerned that some "protocols" are very wide ranging (e.g. the BBC Gel), and aren't really something you can say you've followed as so much will be BBC specific.

I think the processes seem more straightforward to apply (e.g. "we did usability testing and updated the site").

More details on responses

  • E.A. Draffan: last responded on 16, January 2023 at 13:18 (UTC)
  • Wilco Fiers: last responded on 17, January 2023 at 12:42 (UTC)
  • Jaunita George: last responded on 17, January 2023 at 15:16 (UTC)
  • Mary Jo Mueller: last responded on 17, January 2023 at 15:29 (UTC)
  • Gregg Vanderheiden: last responded on 23, January 2023 at 02:31 (UTC)
  • Jeanne F Spellman: last responded on 24, January 2023 at 14:07 (UTC)
  • Stefan Schnabel: last responded on 24, January 2023 at 14:31 (UTC)
  • Gundula Niemann: last responded on 24, January 2023 at 14:42 (UTC)
  • Bruce Bailey: last responded on 24, January 2023 at 15:12 (UTC)
  • Alastair Campbell: last responded on 24, January 2023 at 15:50 (UTC)

Non-responders

The following persons have not answered the questionnaire:

  1. Chris Wilson
  2. Lisa Seeman-Horwitz
  3. Janina Sajka
  4. Shawn Lawton Henry
  5. Katie Haritos-Shea
  6. Shadi Abou-Zahra
  7. Chus Garcia
  8. Steve Faulkner
  9. Patrick Lauke
  10. David MacDonald
  11. Gez Lemon
  12. Makoto Ueki
  13. Peter Korn
  14. Preety Kumar
  15. Georgios Grigoriadis
  16. Romain Deltour
  17. Laura Carlson
  18. Chris Blouch
  19. Jedi Lin
  20. Kimberly Patch
  21. Glenda Sims
  22. Ian Pouncey
  23. Léonie Watson
  24. David Sloan
  25. John Kirkwood
  26. Detlev Fischer
  27. Reinaldo Ferraz
  28. Matt Garrish
  29. Mike Gifford
  30. Loïc Martínez Normand
  31. Mike Pluke
  32. Justine Pascalides
  33. Chris Loiselle
  34. Tzviya Siegman
  35. Jan McSorley
  36. Sailesh Panchang
  37. Cristina Mussinelli
  38. Jonathan Avila
  39. John Rochford
  40. Sarah Horton
  41. Sujasree Kurapati
  42. Jatin Vaishnav
  43. Sam Ogami
  44. Kevin White
  45. Paul Bohman
  46. JaEun Jemma Ku
  47. 骅 杨
  48. Victoria Clark
  49. Avneesh Singh
  50. Mitchell Evan
  51. Michael Gower
  52. biao liu
  53. Scott McCormack
  54. Denis Boudreau
  55. Rachael Bradley Montgomery
  56. Francis Storr
  57. Rick Johnson
  58. David Swallow
  59. Aparna Pasi
  60. Gregorio Pellegrino
  61. Melanie Philipp
  62. Jake Abma
  63. Nicole Windmann
  64. Oliver Keim
  65. Ruoxi Ran
  66. Wendy Reid
  67. Scott O'Hara
  68. Charles Adams
  69. Muhammad Saleem
  70. Amani Ali
  71. Trevor Bostic
  72. Jamie Herrera
  73. Shinya Takami
  74. Karen Herr
  75. Kathy Eng
  76. Cybele Sack
  77. Audrey Maniez
  78. Jennifer Delisi
  79. Arthur Soroken
  80. Daniel Bjorge
  81. Kai Recke
  82. David Fazio
  83. Daniel Montalvo
  84. Mario Chacón-Rivas
  85. Michael Gilbert
  86. Caryn Pagel
  87. Achraf Othman
  88. Fernanda Bonnin
  89. Jared Batterman
  90. Raja Kushalnagar
  91. Jan Williams
  92. Todd Libby
  93. Isabel Holdsworth
  94. Julia Chen
  95. Marcos Franco Murillo
  96. Yutaka Suzuki
  97. Azlan Cuttilan
  98. Jennifer Strickland
  99. Joe Humbert
  100. Ben Tillyer
  101. Charu Pandhi
  102. Poornima Badhan Subramanian
  103. Alain Vagner
  104. Roberto Scano
  105. Rain Breaw Michaels
  106. Kun Zhang
  107. Regina Sanchez
  108. Shawn Thompson
  109. Thomas Brunet
  110. Kenny Dunsin
  111. Jen Goulden
  112. Mike Beganyi
  113. Ronny Hendriks
  114. Breixo Pastoriza Barcia
  115. Olivia Hogan-Stark
  116. Rashmi Katakwar
  117. Julie Rawe
  118. Duff Johnson
  119. Laura Miller
  120. Will Creedle
  121. Shikha Nikhil Dwivedi
  122. Marie Csanady
  123. Meenakshi Das
  124. Perrin Anto
  125. Stephanie Louraine
  126. Rachele DiTullio
  127. Jan Jaap de Groot
  128. Rebecca Monteleone
  129. Ian Kersey
  130. Peter Bossley
  131. Anastasia Lanz
  132. Michael Keane
  133. Chiara De Martin
  134. Giacomo Petri
  135. Andrew Barakat
  136. Devanshu Chandra
  137. Helen Zhou
  138. Bryan Trogdon
  139. Mary Ann (MJ) Jawili
  140. 禹佳 陶
  141. 锦澄 王
  142. Stephen James
  143. Jay Mullen
  144. Thorsten Katzmann
  145. Tony Holland
  146. Kent Boucher
  147. Abbey Davis
  148. Phil Day
  149. Julia Kim
  150. Michelle Lana
  151. David Williams
  152. Mikayla Thompson
  153. Catherine Droege
  154. James Edwards
  155. Eric Hind
  156. Quintin Balsdon
  157. Mario Batušić
  158. David Cox
  159. Sazzad Mahamud
  160. Katy Brickley
  161. Kimberly Sarabia
  162. Corey Hinshaw
  163. Ashley Firth
  164. Daniel Harper-Wain
  165. Kiara Stewart
  166. DJ Chase
  167. Suji Sreerama
  168. Lori Oakley
  169. David Middleton
  170. Alyssa Priddy
  171. Young Choi
  172. Nichole Bui
  173. Julie Romanowski
  174. Eloisa Guerrero
  175. Daniel Henderson-Ede
  176. George Kuan
  177. YAPING LIN
  178. Justin Wilson
  179. Tiffany Burtin
  180. Shane Dittmar
  181. Nayan Padrai
  182. Niamh Kelly
  183. Matt Argomaniz Matthew Argomaniz
  184. Frankie Wolf
  185. Kimberly McGee
  186. Ahson Rana
  187. Carolina Crespo
  188. humor927 humor927
  189. Samantha McDaniel
  190. Matthäus Rojek
  191. Phong Tony Le
  192. Bram Janssens
  193. Graham Ritchie
  194. Aleksandar Cindrikj
  195. Jeroen Hulscher
  196. Alina Vayntrub
  197. Marco Sabidussi
  198. John Toles
  199. Jeanne Erickson Cooley
  200. Theo Hale
  201. Gert-Jan Vercauteren
  202. Karla Rubiano
  203. Aashutosh K
  204. Hidde de Vries
  205. Julian Kittelson-Aldred
  206. Roland Buss
  207. Aditya Surendranath
  208. Avon Kuo
  209. Elizabeth Patrick
  210. Nat Tarnoff
  211. Filippo Zorzi
  212. Mike Pedersen
  213. Rachael Yomtoob

Send an email to all the non-responders.

Compact view of the results / list of email addresses of the responders

WBS home / Questionnaires / WG questionnaires / Answer this questionnaire

Report issues on GitHub project w3c/wbs-design or by mail to sysreq.