– DRAFT –
Protocols Sub Team - Jan 21, 2022

<JF> https://www.w3.org/events/meetings/bfc72cd9-fdfc-4847-826a-01afb9e3f5e7/20211105T090000

RESOLUTION: sub-team to present to AG WG on Feb. 8th

<Zakim> MichaelC, you wanted to say protocols define procedural steps, in the same way methods define actions

<Zakim> MichaelC, you wanted to say protocols provide way to test adherence to steps, quantitively and qualitatively, to evaluate

<Chuck> exactly, I'm in queue to move that we formally start the meeting

<Zakim> Chuck, you wanted to ask Rachael if she has updated our agenda page, and to suggest we scribe this (formally start call)

Continue brainstorming the protocols that we believe would be applicable

mc: protocols define procedural steps, in the same way methods define actions

protocols provide way to test adherence to steps, quantitively and qualitatively, to evaluate

jf: looking at examples, not sure they´re steps

<Chuck> jf: good summary for where we were, with addition of... I've been attempting to find examples of guidance in wild, and to see if that fits our concepts.

<Chuck> jf: "making content usable" is one candidate for me.

<Zakim> MichaelC, you wanted to disentangle definitions

<Chuck> mc: I don't consider content usable a protocol. It's an outcome. we need a protocol to determine how we achieve that outcome. I don't want to get tangled in specific words.

<Chuck> mc: protocols in wild that aren't exactly what we are looking for, we need to define the thing we are creating. I keep coming back to "steps", maybe including following guidance.

<Chuck> mc scribe to you

jg: +1; having part of the protocols able to measure adherence to the protocols allows showing you´ve achieved outcome

<JF> https://www.w3.org/TR/coga-usable/#testing-each-pattern

jf: Content usable has testing steps

it has user stories, we tell them what the expectation is

but the outcome is hard to measure

<Chuck> mc: A protocol could say "follow the guidance of content usable, follow those procedures". I think it needs to say how much effort you put into it.

<Chuck> mc: It could say "try following...." the protocol for wcag 3 conformance is still the doc that says to meet these protocols, here are these outcomes. Closely coupled to something like content usable.

<Chuck> mc: successful protocols will be successful if they model how orgs adopt protocols.

<Chuck> mc: ...as you follow the steps of this protocol, here's what you should be looking at.

<JF> Capturing "Proof points"

ja: can be like maturity model, you have steps in the protocol, and ways of describing how you address them

<Zakim> Rachael, you wanted to say we are assuming a protocol is a document referenced at a high level vs a small subset

rbm: hear an assumption that protocols are documents, but I think they can be small sets of steps that fit under something else

<Rachael> I'm not sure I fully understand "proof points:

<Chuck> mc: I think I need a better description.

jf: proposed, protocols is a collection of ¨proof points¨

in content usable, there are ¨I need to¨ statements

those can translate to proof points

e.g., ¨does the web site do X¨

<Chuck> mc: My understanding of problem we are solving is that we need protocols because we can't measure some proof points. Protocols evaluation needs to have ways to evaluate that you are trying to achieve the outcomes.

<Chuck> mc: I don't think it can go to that level because it would be a method.

<Chuck> jennifers: proof point idea bothers me, not very plain language.

js: find ¨proof point¨ a difficult term

<JakeAbma> https://w3c.github.io/wcag-maturity-model/index.html#proof-points

want to use plain language terms

don´t think outcomes from protocols are repeatable

<JF> Capturing "Evaluating your work by"

protocol should be about your due diligence, and evaluate that

<Zakim> Chuck, you wanted to say that we are trying to follow queue, and see if Jake wants to respond.

ja: if we´re interested in proof points, we should look further, see if it fits or not

maturity model has been using multiple names

the concept of something to prove that you at least started, and how mature

orgs commonly adopt protocols for how they address things whose outcomes can´t be measured

the protocol demonstrates that you´re on the way to following whatever guidance it is such as content usable

jf: think there´s alignment on that, I liked ¨evaluate your work by¨

<Rachael> Are "proof points" = options for meeting a goal/outcome with evidence to show the option was applied?

for protocols to work, they require a formal statement

in a targeting scenario, methods aim right for the target, protocols aim for reasonable near it

whether a protocol is a set of steps or a mental checklist, you´re following steps towards the goal

jg: concerned WCAG 3 won´t have the level of repeatability as WCAG 2

if so, legal pressure to adopt could relax

we need something orgs can be held accountable to

if there isn´t consistent measurement, there will be weak claims

<Zakim> Chuck, you wanted to say and explain that we are trying to more formally follow the queue process.

<Zakim> MichaelC, you wanted to talk about conformance levels and to say quality of outcome is higher level of the protocol

<Chuck> mc: Some of the questions I heard JF and Jaunita bring up, the idea that protocols should feed into wcag 3 conformance levels. At bronze, it's about "are you following..."

<Chuck> mc: At gold there could be a quality component.

<Chuck> mc: At least at bronze I would agree with Jaunita is it needs to be measurable and repeatable. Measuring steps would be a way.

jf: +1 on adoption concern

but right now, US federal sites have to use plain language, but how is that measured?

plainlanguage.gov has 8 outcomes

<jenniferS> * Queue Jaunita, please

these outcomes are subjective, but we can evaluate to some level

jg: my understanding is protocols are for things that are not testable, measurable, repeatable

they provide an objective basis against which to show your homework

in your effort to meet the outcomes

re US plain language requirements, there is very low compliance right now

many don´t even know of it

a protocol gives us ¨something¨

rather than let litigation wind through to a precedent

<Zakim> MichaelC, you wanted to say if we can measure outcomes, it´s a method

<Chuck> mc: Attempting to be clear. If we measure at outcome, it's a method not a protocol.

<Chuck> mc: We have protocols for cases where we can't measure outcome. At higher conformance levels, protocol should include outcome measurement that is the fuzzy near target results

<Chuck> mc: That could be a good case for a higher conformance level.

<Chuck> mc: Still the case that protocol is distinct from method.

<Chuck> mc: It would define its measurement.

jg: the legal structure in the US for private companies is ADA

only recently was WCAG 2 adopted for ADA

if WCAG 3 includes measurable outcomes as part of protocols, it might be considered unadoptable by the courts

jf: we shouldn´t be talking about conformance and scoring yet

¨methods evaluate outcomes, protocols evaluate objectives¨ seems to be a distinction

there are other objectives we want to accomplish

we want actual outcomes, which we can approximately recognize

<JF> +1 Jake

ja: the idea of the measuring system in outcomes and methods was to allow reaching additional outcomes

I see WCAG 3 as its own protocol

We provide ways to measure inside it

proof points, methods, whatever, they mean we´re measuring something

<Chuck> mc: I understand that we shouldn't be talking conformance in this group, but the motivation is to address issues that don't fit in conformance.

<Zakim> Rachael, you wanted to suggest capturing 1. Protocols may define their own measurement and 2. Integrating protocols should be done in a way that encourages adoption by legal community

<jaunita_george> +1 to michael

<Chuck> rachael: encourage us to capture some important things.

<Chuck> rachael: Protocols may define their own measurements, and we should encourage adoption by legal community.

<jaunita_george> +1

jg: agree with RBM and MC

jf: +1 to RBM

any disagreement?

question: Protocols may define their own measurement

<JF> Draft Decision: Protocols may define their own measurement

ja: if WCAG 3 is protocol, shouldn´t it handle stuff like ref to content usable

jf: <something about big project>

js: protocol is tool to measure WCAG subjective outcomes

<Chuck> mc: Let's address next week.