W3C

Site Navigation

Nearby

Results of Questionnaire Protocols Odds and Ends

The results of this questionnaire are available to anybody.

This questionnaire was open from 2022-04-13 to 2022-04-16.

7 answers have been received.

Jump to results for question:

  1. What do you consider the intent of protocols in relation to conformance?
  2. How do you think protocols should assess success?
  3. Should we alternate between 8am and noon (Eastern Time) to allow for greater participation?
  4. Would you support creating a group Slack channel in the A11y Slack to help facilitate real-time communication on issues, rather than just relying on email? We could share out a summary of the discussions weekly for the rest of the group/public record.

1. What do you consider the intent of protocols in relation to conformance?

A. Protocols are above or beyond any type of conformance, aka “extra credit”

B. Protocols are a way of meeting the guidelines, and how they fit into conformance is TBD

C. Something else (please provide an explanation in comments)

Summary

ChoiceAll responders
Results
Option A. 2
Option B. 3
Option C (please explain) 2

Details

Responder What do you consider the intent of protocols in relation to conformance?
Rachael Bradley Montgomery Option B.
John Foliot Option C (please explain) Structurally, Protocols do not sit "above" Requirements, they sit "beside" them. Protocols contribute in parallel to Requirements at content creation time. (Requirement: all non-text elements require a text alternative; Protocol: use the alt text decision tree at https://www.w3.org/WAI/tutorials/images/decision-tree/ to guide the author towards a "good" text alternative)

Protocols *MAY* provide methods or advice on how to meet a specific Guideline, but they may also NOT: the guidance found in a Protocol may not have any measurable requirements (eg. Plain Language: "Choose your words carefully" - https://www.plainlanguage.gov/guidelines/words/), but still provide important guidance in the content creation phase:

"Words matter. They are the most basic building blocks of written and spoken communication. Don’t complicate things by using jargon, technical terms, or abbreviations that people won’t understand. Choose your words carefully and be consistent in your writing style.

* Prefer the familiar word to the far-fetched.
* Prefer the concrete word to the abstraction.
* Prefer the single word to the circumlocution. (JF: even Plainlanguage.gov uses complex terms)
* Prefer the short word to the long.
* Prefer the Saxon word to the Romance word.

-- end --

My concern here is that it appears that in this context "conformance" is still envisioned as a binary pass/fail state, whereas the current proposed structure of WCAG 3 suggests that "conformance" is (will be) stratified (Bronze, Silver, Gold), and that meeting "Bronze" is a tacit admission that you are less than "perfect", and thus less than "fully conforming".

In my mental model of Protocols, adopting a Protocol accrues "points" towards a conformance score, and there will be a minimum score value for Bronze, a higher score for Silver, and a better-again score for Gold.
Jennifer Strickland Option A. Protocols would not be a way of meeting minimum level of conformance.
Michael Cooper Option B.
Jeanne F Spellman Option A. I think they should be above a minimum conformance. I don't think that Protocols should be a way of making a minimum bar because they have too much potential for "gaming' the system.
Charles Adams Option C (please explain) Both A and B can be true.

For me a protocol is a set of (to be) defined principles that allow for individuals or organizations to construct their own dynamic approaches that are not otherwise documented by W3C to achieving any goal that may or may not be documented by W3C. The W3C would define the "principles" that represent the building blocks for crafting protocols, and individuals/organizations are free to build anything they can imagine using those "principles". A protocol could be in service of "conformance" to a guideline or set of guidelines, or could be a well documented statement of advocacy that does not have any influence over conformance at all.

One company's use of a "protocol" could be to demonstrate that they use native AT testers to validate their conformance to X number of guidelines which impact functional needs of the AT testers (hypothetically the "text alternatives" and "structured content" guidelines and any number of additional guidelines yet to be created), and another company's use of a "protocol" could be to demonstrate use of a new "method" (which was not defined by W3C) that helps them achieve conformance to a specific guideline, such as the "text alternatives" guideline; and a 3rd company's use of a protocol could be to demonstrate their investment in advanced research which does not directly addresses any guidelines, but otherwise demonstrates a commitment to accessibility. The first two examples would influence "conformance", and the third example would be "extra credit".

My formal definition: A protocol is a set of (yet to be) W3C defined principles that can be used to build non-W3C documented approaches for advancing a goal related to accessibility.
JaEun Jemma Ku Option B.

2. How do you think protocols should assess success?

NOTE: “Inputs” may be documentation of steps, actions taken, date completed, conformance claims, etc. that WCAG would require as proof of doing the protocol. These are TBD. “Outcomes” are the results of a given protocol.

  • A. Protocols provide a way to evaluate whether subjective WCAG requirements have been met. (In other words, protocols evaluate the inputs and the outcomes.)

    B. Protocols test the inputs against what WCAG has required

    C. Something else (please explain in comments)

    • Summary

    ChoiceAll responders
    Results
    Option A.
    Option B. 3
    Option C (please comment). 4

    Details

    Responder How do you think protocols should assess success?Comments
    Rachael Bradley Montgomery Option B.
    John Foliot Option C (please comment). Closer to Option A but not stated as such - when you attempt to evaluate outcomes the results will always be subjective, so with Protocols you cannot do both - at best you can evaluate the 'inputs' and seek evidence of the application in the outputs.

    "Conforming" to a Protocol (accruing points) is done by declaring that a Protocol has been adopted via a formal declaration, WITHOUT attempting to assess how well the Protocol has been applied (the subjective part).

    This is because, from a "legal compliance" perspective, the declaring party will always make the subjective assessment declaration in the most glowing of terms, the litigant will always make their subjective assessment claim in the absolute worst of terms, and because the outcomes of Protocols will always be subjective, when attempting to evaluate outcomes it will be impossible to arrive at conclusions with any consistency. (Evidence: read any VPAT out there and specifically "Partially Supports")

    Even evaluating "proof points" cannot ensure that the application of the protocol has been done to a 3rd party's satisfaction. Example:

    * Declaration: This response has been written using the principles of plain language
    * Proof Point(s): I have read the requirements for plain language as outlined at plainlanguage.gov; I have taken an online course on how to write content using plain language principles; I had a third-party review this content.

    Prove me wrong. Prove that the principles of plain language HAVE NOT been applied here.

    This is why you cannot accurately or consistently evaluate outcomes.
    Jennifer Strickland Option C (please comment). "WCAG would require as proof" — afaik WCAG doesn't require proof. WCAG documents guidelines. Other entities might use the guidelines and turn them into requirements, and those entities might have a "prove" mechanism, although I have yet to see any actually evaluate it. The only evaluation I've seen are organizations that review their own work, and then user feedback and complaints.

    Protocols provide a way to evaluate subjective WCAG success criteria (we don't have WCAG requirements) and they also evaluate accessibility / inclusion / equity of an outcome, so can exceed what WCAG provides guidelines for.
    Michael Cooper Option B.
    Jeanne F Spellman Option C (please comment). Protocols can evaluate guidance where the outcomes can not be measured. We can measure subjective tests without using protocols. Where protocols are useful are in measuring and evaluating processes, such as Inclusive Design. The protocols says that to get credit for using Inclusive Design (for example) you must document that the designers are trained in Inclusive Design, follow the principles of inclusive design, and receive input on their designs from users with various (specified) types of disabilities and improve the design based on feedback. We don't measure the output, we measure the inputs (training, follow the training, take actions to get feedback, improve the design based on feedback). Each of those steps can be documented to assess success. The protocol specifies the inputs and how they are documented or measured.
    Charles Adams Option C (please comment). If we define a set of "principles" that individuals/organizations can use to build their own protocols, then a protocol can do any, all or none of the elements in A and B. An individual or organization can construct a protocol that evaluates ONLY inputs, another protocol could be built that evaluates ONLY outputs, a third can be built that does both, and a fourth does neither. As long as the creator of the protocol utilizes the (yet to be) set of pre-defined "principles" to craft their unique protocol, the individual/organization has liberty to create anything they can imagine.
    JaEun Jemma Ku Option B.

    3. Should we alternate between 8am and noon (Eastern Time) to allow for greater participation?

    We surveyed possible times several weeks ago and the two possible times were 8am and 12 noon Eastern. 8am is too early for US participants on the west coast and noon is too late for our European participants.

    Yes: We should alternate between 8am and noon (Eastern Time)

    No: We should not alternate between 8am and noon (Eastern Time)

    Other: Please provide an explanation in comments

    Summary

    ChoiceAll responders
    Results
    Yes 5
    No
    Other (Please provide an explanation in comments) 2

    Details

    Responder Should we alternate between 8am and noon (Eastern Time) to allow for greater participation?Comments
    Rachael Bradley Montgomery Yes
    John Foliot Other (Please provide an explanation in comments) Whatever the group decides. 9:00 AM was working fine for me.
    Jennifer Strickland Yes 9am was conducive for both Europe and the Pacific time zone. Could we go back to that? Alternatively, I do think we should alternate to ensure attendance from the Pacific and Asia time zones. The W3C creates guidelines for the globe, so how might we be more inclusive and welcome the globe?
    Michael Cooper Other (Please provide an explanation in comments) It might be hard for groups meeting at different times to develop overall consensus unless there is enough overlap in participation. But I wouldn't oppose a majority wish to try it.
    Jeanne F Spellman Yes We need to be able to include both groups. It would be better to find an 11 ET slot, but if we can't, then we should try alternating.
    Charles Adams Yes
    JaEun Jemma Ku Yes

    4. Would you support creating a group Slack channel in the A11y Slack to help facilitate real-time communication on issues, rather than just relying on email? We could share out a summary of the discussions weekly for the rest of the group/public record.

    Yes, I support creating a group Slack channel.

    No, I do not support creating a group Slack channel.

    Other (Please provide an explanation in comments)

    Summary

    ChoiceAll responders
    Results
    Yes 4
    No 3
    Other (Please provide an explanation in comments)

    Details

    Responder Would you support creating a group Slack channel in the A11y Slack to help facilitate real-time communication on issues, rather than just relying on email? We could share out a summary of the discussions weekly for the rest of the group/public record.
    Rachael Bradley Montgomery No My preference is not to. We have a lot of communication channels already and I am concerned with adding another. That said, I will support and engage with whatever the group decides.
    John Foliot No THE STONGEST OF OBJECTIONS!

    Splintering the discussion across multiple channels does not foster better communication, it frustrates it because now you have to monitor even more channels of information. We currently already have a mailing list and GitHub issues, which can already cause 'forking' in discussions.

    If we decide to use Slack, will we also have a Microsoft Teams channel (for those participants who do not or CAN NOT use Slack at work)?
    Jennifer Strickland Yes If the A11y Slack channel was Private and kept to WCAG members that would be of interest, although I know the WCAG leadership resisted the idea in the past due to some concerns (I don't recall what they were, as it was about a year ago — when I also suggested it). I do not want the conversation open more broadly at this time and there are people in the A11y Slack that would be problematic.
    Michael Cooper Yes
    Jeanne F Spellman No Slack is not as accessible as we would like. We have a wcag3 Slack channel that gets no use. If you want to try it out, then use that channel.
    Charles Adams Yes Yes(!!!), but I think that my company might restrict access to external channels. I've not been able to successfully connect to a non company channel. This could be user error.
    JaEun Jemma Ku Yes

    More details on responses

    • Rachael Bradley Montgomery: last responded on 14, April 2022 at 11:38 (UTC)
    • John Foliot: last responded on 14, April 2022 at 12:17 (UTC)
    • Jennifer Strickland: last responded on 14, April 2022 at 13:52 (UTC)
    • Michael Cooper: last responded on 14, April 2022 at 17:06 (UTC)
    • Jeanne F Spellman: last responded on 14, April 2022 at 22:42 (UTC)
    • Charles Adams: last responded on 14, April 2022 at 23:26 (UTC)
    • JaEun Jemma Ku: last responded on 15, April 2022 at 13:48 (UTC)

    Non-responders

    The following persons have not answered the questionnaire:

    1. Gregg Vanderheiden
    2. Chris Wilson
    3. Lisa Seeman-Horwitz
    4. Janina Sajka
    5. Shawn Lawton Henry
    6. Katie Haritos-Shea
    7. Shadi Abou-Zahra
    8. Chus Garcia
    9. Steve Faulkner
    10. Patrick Lauke
    11. David MacDonald
    12. Gez Lemon
    13. Makoto Ueki
    14. Peter Korn
    15. Preety Kumar
    16. Bruce Bailey
    17. Georgios Grigoriadis
    18. Stefan Schnabel
    19. Romain Deltour
    20. Laura Carlson
    21. Chris Blouch
    22. Jedi Lin
    23. Wilco Fiers
    24. Kimberly Patch
    25. Glenda Sims
    26. Ian Pouncey
    27. Alastair Campbell
    28. Léonie Watson
    29. David Sloan
    30. Mary Jo Mueller
    31. John Kirkwood
    32. Detlev Fischer
    33. Reinaldo Ferraz
    34. Matt Garrish
    35. Mike Gifford
    36. Loïc Martínez Normand
    37. Mike Pluke
    38. Justine Pascalides
    39. Chris Loiselle
    40. Tzviya Siegman
    41. Jan McSorley
    42. Sailesh Panchang
    43. Cristina Mussinelli
    44. Jonathan Avila
    45. John Rochford
    46. Sarah Horton
    47. Sujasree Kurapati
    48. Jatin Vaishnav
    49. Sam Ogami
    50. Kevin White
    51. E.A. Draffan
    52. Paul Bohman
    53. 骅 杨
    54. Victoria Clark
    55. Avneesh Singh
    56. Mitchell Evan
    57. Michael Gower
    58. biao liu
    59. Scott McCormack
    60. Denis Boudreau
    61. Francis Storr
    62. Rick Johnson
    63. David Swallow
    64. Aparna Pasi
    65. Gregorio Pellegrino
    66. Melanie Philipp
    67. Jake Abma
    68. Nicole Windmann
    69. Oliver Keim
    70. Gundula Niemann
    71. Ruoxi Ran
    72. Richard Boardman
    73. Wendy Reid
    74. Scott O'Hara
    75. Muhammad Saleem
    76. Amani Ali
    77. Trevor Bostic
    78. Jamie Herrera
    79. Shinya Takami
    80. Karen Herr
    81. Kathy Eng
    82. Cybele Sack
    83. Audrey Maniez
    84. Jennifer Delisi
    85. Arthur Soroken
    86. Daniel Bjorge
    87. Kai Recke
    88. David Fazio
    89. Daniel Montalvo
    90. Mario Chacón-Rivas
    91. Michael Gilbert
    92. Caryn Pagel
    93. Achraf Othman
    94. Fernanda Bonnin
    95. Jared Batterman
    96. Raja Kushalnagar
    97. Jan Williams
    98. Todd Libby
    99. Isabel Holdsworth
    100. Julia Chen
    101. Marcos Franco Murillo
    102. Yutaka Suzuki
    103. Azlan Cuttilan
    104. Joe Humbert
    105. Ben Tillyer
    106. Charu Pandhi
    107. Poornima Badhan Subramanian
    108. Alain Vagner
    109. Roberto Scano
    110. Rain Breaw Michaels
    111. Kun Zhang
    112. Jaunita George
    113. Regina Sanchez
    114. Shawn Thompson
    115. Thomas Brunet
    116. Kenny Dunsin
    117. Jen Goulden
    118. Mike Beganyi
    119. Ronny Hendriks
    120. Breixo Pastoriza Barcia
    121. Olivia Hogan-Stark
    122. Rashmi Katakwar
    123. Julie Rawe
    124. Duff Johnson
    125. Laura Miller
    126. Will Creedle
    127. Shikha Nikhil Dwivedi
    128. Marie Csanady
    129. Meenakshi Das
    130. Perrin Anto
    131. Stephanie Louraine
    132. Rachele DiTullio
    133. Jan Jaap de Groot
    134. Rebecca Monteleone
    135. Ian Kersey
    136. Peter Bossley
    137. Anastasia Lanz
    138. Michael Keane
    139. Chiara De Martin
    140. Giacomo Petri
    141. Andrew Barakat
    142. Devanshu Chandra
    143. Helen Zhou
    144. Bryan Trogdon
    145. Mary Ann (MJ) Jawili
    146. 禹佳 陶
    147. 锦澄 王
    148. Stephen James
    149. Jay Mullen
    150. Thorsten Katzmann
    151. Tony Holland
    152. Kent Boucher
    153. Abbey Davis
    154. Phil Day
    155. Julia Kim
    156. Michelle Lana
    157. David Williams
    158. Mikayla Thompson
    159. Catherine Droege
    160. James Edwards
    161. Eric Hind
    162. Quintin Balsdon
    163. Mario Batušić
    164. David Cox
    165. Sazzad Mahamud
    166. Katy Brickley
    167. Kimberly Sarabia
    168. Corey Hinshaw
    169. Ashley Firth
    170. Daniel Harper-Wain
    171. Kiara Stewart
    172. DJ Chase
    173. Suji Sreerama
    174. Lori Oakley
    175. David Middleton
    176. Alyssa Priddy
    177. Young Choi
    178. Nichole Bui
    179. Julie Romanowski
    180. Eloisa Guerrero
    181. Daniel Henderson-Ede
    182. George Kuan
    183. YAPING LIN
    184. Justin Wilson
    185. Tiffany Burtin
    186. Shane Dittmar
    187. Nayan Padrai
    188. Niamh Kelly
    189. Matt Argomaniz Matthew Argomaniz
    190. Frankie Wolf
    191. Kimberly McGee
    192. Ahson Rana
    193. Carolina Crespo
    194. humor927 humor927
    195. Samantha McDaniel
    196. Matthäus Rojek
    197. Phong Tony Le
    198. Bram Janssens
    199. Graham Ritchie
    200. Aleksandar Cindrikj
    201. Jeroen Hulscher
    202. Alina Vayntrub
    203. Marco Sabidussi
    204. John Toles
    205. Jeanne Erickson Cooley
    206. Theo Hale
    207. Gert-Jan Vercauteren
    208. Karla Rubiano
    209. Aashutosh K
    210. Hidde de Vries
    211. Julian Kittelson-Aldred
    212. Roland Buss
    213. Aditya Surendranath
    214. Avon Kuo
    215. Elizabeth Patrick
    216. Nat Tarnoff
    217. Filippo Zorzi

    Send an email to all the non-responders.

    Compact view of the results / list of email addresses of the responders

    WBS home / Questionnaires / WG questionnaires / Answer this questionnaire

    Report issues on GitHub project w3c/wbs-design or by mail to sysreq.