Meeting minutes
Review and discuss our progress and key questions which are recorded here: Protocols - Google Docs
Showing a document from Rachael.
The document is Protocols and Scoping Proposal.
Please focus on the context and not so much on the terminology.
<Chuck_> https://
Rachael says to review, pick apart, etc.
<Jaunita_George> +1
Jaunita requested a 30k foot view.
Rachael said the document attempts to clarify what protocols attempt to do.
Jake expresses uncertainty understanding what the document covers, as protocols has yet to be defined.
Rachael says this is a document I put together to help me understand what I was struggling with.
This is an attempt to understand the concepts.
<mbgower> yes
Jake: in my work place, we think certain documents we've created might fit a protocol, but likely not align with how we in WCAG3 define a protocol.
Jake: we indicated a specific problem the moment you say the COGA document is a protocol, other issues covered are part of our default WCAG2/3 approach, then you can never transfer those same requirements to that other document. There needs to be a clear distinction.
Jake: Now there's a protocol or request from COGA, and we can take it out, and now its part of another protocol or vice-versa.
* wow, this is super hard to scribe for!
Mike: Jake, you're speaking at a level of abstraction I'm struggling to follow.
Mike: if you could provide a sample protocol, it might help me understand.
Rachael: having looked at this and thought about it, I think COGA is a bad example of a protocol. I think plain language.gov or ipso plain language are better examples.
<mbgower> +1 yes plain language is closer to what I'm thinking about
Rachael: it is a review that is done, but doesn't have results that are clear and measurable, but worth being done.
Rachael: if it is clear and measurable, then it can be automated; if it can't then it might be possible to use a protocol.
Rachael: when you get into a test case, then we would expect them to lift the conditions, run through them. In a protocol, we can't do that, we can only give credit for having done the test.
Jake: response to mgower, if you have an approach to testing, and once you say this criteria and this requirements belong to one of them, you get into trouble when you get into the other. now we say it's a protocol, and then we create a new guideline, then it is not consistent and reliable over a longer period of time.
Jake: response to each, that is something that might be more interesting, it isn't a protocol testing approach isn't that something you can extract from this work and make it applicable to the guidelines and methods for wcag3, instead of saying this whole doc, you say…
so we'd have 20k different protocols, every country has some, and then if you have a clear way of measuring a protocol then isn't that something we can use in wcag3?
then you don't have the burden of saying this or not protocol is what we use.
<Chuck_> Jennifer: The way I saw protocols was more abstract. I recognize we are in WCAG and everything is measurable (or maybe not)
<Chuck_> Jennifer: This was about doing what you can to show you made the effort on things that are subjective.
<Chuck_> +1
<Chuck_> Jennifer: It's a point in time, it may not last for a few years. One challenge we all have is we are at the beginning, and everything is evolving so quickly.
<Chuck_> Jennifer: We've picked a point in time for WCAG x, we can't make things that are infinite. I think protocols aims to ask individuals to make the effort to test the subjective things, and show the work.
<mbgower> +1 yes make the effort to test subjective and show the work. very similar to how i envision
<Chuck_> Jennifer: It can be, the output of a protocol following the protocol is "here's the thing I used, here's the thing I did, here's the results, here's the lessons learned".
<Chuck_> Jennifer: I see it as a one pager fill in the blanks and you can go. I know it's not science, but technology is thankfully and painfully changing rapidly.
<Chuck_> Jennifer: People who need for example meaningful alt text need people who are creating this content to at least make the effort.
<Zakim> mbgower, you wanted to say least to most prescriptive, and hopefully migrating with time
Mbgower: an extremely ill-formed protocol would be 'make sure the text makes sense' - but if that's what someone used it, then they check off they did it, and track yes we did it.
another way to think of this matrix is from most prescriptive to least prescriptive.
the matrix is in the linked document.
response to Jake: in my hopeful view of the world, I think in the future we'll have more ability to test, and I think as protocols advance and ppl come up with repeatable algorithms, i.e., Rachael mentioned plain language -- people are making better tests for measuring language complexity.
as those become more refined they might go from a protocol to a test case.
the challenge we have is that the further we go to the right (least prescriptive), the…[lost this]
in the two decades we haven't been able to document sufficient techniques for [something] but we lacked the resources to do it.
I think protocols allows an org to establish what they think is sufficient.
they document this is what we did, and how, and why we think is sufficient.
Chuck: we have measures for alt text, what protocols is about is measuring the good faith effort to make the meaningful — is the process for how they tried, in good faith, … that process is the protocol.
Jake: beautifully spoken, I think everyone in this group agreed from day 1, what I think is interesting, and what Mike Gower triggered, is we have discussed what is a protocol, what external products fit our definition, do we want to take those and within WCAG3 "these protocols" are the ones you can use?
Are we WCAG3 defining the list of protocols? Instead of allowing folks to look around the world and choose their protocols?
So, are we creating a list of acceptable protocols, Mike? Or opening up to the whole world.
<Chuck_> Jennifer: The way I envision it is that WCAG 3 might have a list of ones that the WG knows of to set as examples, but that teams are able to use something other than that list because WCAG 3 is a group of humans with blind spots.
<Chuck_> Jennifer: We may discover new and better protocols. We still won't know about them. They won't take their homework and submit to teacher WCAG. It would be great for us to have manpower and review each, but we don't.
<Chuck_> Jennifer: But the fact that they do the due diligence is great. It would be great for every website to have a page that states what they did to make their product(s) accessible.
Rachael: I'd like to see usability testing and design guide creation as protocol criteria, too.
Rachael: a protocol seems like some kind of review against something that can't be currently quantified.
usability testing, a creation of a design guide - those are protocols, intended to create consistency, and seems like user process protocol.
+1
MbGower: about 2 years ago I wrote a toolkit at IBM to help folks understand the wcag
we broke things down by what needed to happen in design, ux, etc.
one thing I had to apply restraint is to not put things in that I thought should be covered by designers but that aren't in wcag.
there are undoubtedly many things that are hard to fit into the wcag2x model that are things designers should do, like make sure ppl understand how to interact with this page, how ppl might do this with screen magnifier or reader, a way of understanding the ux, how I thought about how a user with AT would interact with this page -- and those are things I think might be useful as protocols. how to perform one's job.
will provide a link to that guide.
<mbgower> ibm.com/able/toolkit
Chuck: I had not envisioned a finite list of pre-approved protocols from w3c. I figured we would define the attributes that make up a good protocol, and then the world was free to craft their own and use it.
Jake: I would like to see this on the agenda, and have a mutual consensus on this.
<JakeAbma> https://
I have seen your work Mike and it's amazing, there's also material design, the work from the BBC, and …
basically what you say is, I would really like to discuss. let's pick one or four and see how they fit as a protocol.
like if I have links to the various protocols, and the team uses those protocols, but I am not convinced that all those ppl really create products that are whole.
<mbgower> yep, bbc work is another example of material that is more subjective and approaching a series of protocols
at my company we look at all those, suggest ppl look at those, and use those internally as guidelines. I wonder if we have consensus around pick and choose whatever you want.
maybe that's why Rachael created her document, is this is how you choose and this is what we suggest you use to measure the protocol. Is that what we are aiming at?
Jaunita: if someone said they were following a protocol, how would we know, and how many points would we give them? what would it mean?
Chuck: a) no we don't yet have consensus. we
we've been working on something to present to silver and something to present to agwg. I'm hoping we don't need agwg consensus to present up.
and I thought we were close, but we lost John for a bit and are trying to keep the momentum going.
to Jaunita, we don't know yet. here's how you look at a protocol, determine if it is good or not, and
what Rachael presented is one way to take apart a protocol to determine if it is good.
<Zakim> mbgower, you wanted to say thinking about achieving and reporting on conformance now and in the future can help
I don't yet know how a company determines if a protocols is good or shows they did it.
Mike: I've been trying to separate what we're trying to achieve and what is being examined.
wcag2x has nothing about reporting in it.
most ppl use the ITI templates, accessibility conformance report that ITI puts out.
<Chuck_> +1 to no such thing as a perfect product
no company will have a perfectly clean report.
if I'm procuring software and it doesn't perfectly meet wcag, then I have to assess how it does/doesn't.
then we have an org kinda doing testing, then someone has to assess the quality of their report -- that's what happens right now.
in the future, w the idea of protocols, the org has the regular things they do now to report on wcag2x stuff which is hopefully more determined by automated testing.
then we move over to the protocols stuff, how do they say they achieve clean language, they say they are using this tool to evaluate and is part of their report, can link out to their results.
they report how they achieve that. as an assessor, I can say their method isn't that great, and this other org's assessment is clearer.
last point, nobody puts out ACRs for webpage. it tends to happen on an application basis. maybe in the future there will be some way… but I primarily think this would happen for applications not websites.
Chuck: point of order, ending in 7 mins.
Rachael: I originally thought we had to work out the high level conformance and scoring, but now I think we can separate.
if we can define what we testing, then the scoring/not becomes a lot easier.
<mbgower> +1 totally agree; figure out what we're going to test first
then we can see what kind of results we're getting.
we set a set of criteria in guidelines, and then what regulators choose is a different set. I do think addressing these separately is a good strategy.
Jaunita: if we develop pass/fail criteria then we could limit the number of protocols we allow then we could limit the number of protocols. and if we treated the protocols as an extra credit. otherwise we get into a situation where protocols meet a requirement but we can't measure them or that anyone has used them.
or we limit the number of protocols that can be used.
Jake: I hear 3 different things
based on what mike mentioned and clear language, the tool, etc.
acceptance criteria for a protocol which we need to judge, the usage by an org
then how John started this whole protocols idea, with a maturity modeling approach
then a 3rd one, the whole idea for a protocol was to embed wishes and needs for people with functional needs that can't find their way in wcag3.
what is the maturity of the approach you say you do, and what is the acceptance criteria for a protocol.
<Zakim> Chuck_, you wanted to say our 3rd agenda item was going to be our effort to try and get consensus on a concept or definition
Chuck: the consensus was… [I got lost!]
<Chuck_> The third agenda item today was going to be our effort to get to a "consensus"
Rachael: I should have said at the beginning that we were aiming for pass/fail - did you conduct a review and document issues found.