25 February 2022


Chuck, Jaunita_George, jennifer, MichaelC, Rachael, ShawnT

Meeting minutes


ca: JF away for indeterminate time

have asked Jaunita George to step in as subgroup lead

needs a different time

what´s good for people?

for me, Friday afternoons Mountain are good

<ShawnT> I'm good at any time

mc: Friday after noon Eastern best

<Rachael> Friday afternoons often work for me.

js: later Friday afternoon Eastern better

ca: will start a schedule poll based on that input

<ShawnT> 4ish is tough for me

Review https://docs.google.com/presentation/d/1b5xHQWBzoYdKp7BfPgIUBCpz-yaDOx_kSq_HlQxcFh0/edit#slide=id.g115901c77f1_0_218

rbm: <shares screen to present, scribing only non-slide content>

want to present this to AG tuesday

make sure different voices represented

so want input here

on slide 5, we have the need to go beyond WCAG 2 scope yet ensure things testable at pass / fail level

note the terminology is placeholder, focus on the concepts

ca: I´ve heard that we want tests that are beyond pass / fail

use rubriks etc. to convert to test scores

so different evaluation methods can convert to pass / fail?

not contradictory?]

rbm: agree, not contradictory

one idea is to report on measurements, with a sliding scale

another is to @@

third is to move the sliding scale down to the rubrik level

ca: that may address some concerns we have

rbm: slide 6, there are 4 concepts of scope for tests

4 types of tests slides 7 - 10

constant tests and condition tests have similarities, but think it´s useful to differentiate them

ca: seems to describe what we´ve done

rbm: example: text alternatives has 2 types of test, the constant test about providing it, and the condition test about its quality

meaningful sequence is one that´s mostly just condition test

slide 9, test case as used here is a new concept to us

instead of defining conditions for test, it defines the type of conditions to set

this is similar to how requirements for tests are defined in software engineering

should address the ¨you know it when you see it but can´t define it¨ situation

there can be known failures

this can be use for e.g., content usable testing, where it´s a lot about consistency

st: definition of affordance?

rbm: it is the visual indication of what something does

<Chuck> MichaelC: Been told by usability that this is something that can cause concerns. The affordance is what the object can do, and the perceived affordance is a person's awareness of that.

<Chuck> MichaelC: We may get debate if we use that definition. I want to not use just visual indication. The indication of its ability can happen in any appropriate channel.

<Chuck> Rachael: Alternatives of what we can call it?

rbm: chose the common usage of that term, but know it´s debated, let´s sub

js: indicator, cue?

<Chuck> Jennifer: Indicator/cue of usage.

rbm: good enough for now

ca: yeah, no wordsmithing!

rbm: down the road we´ll need to look at editorial consistency

that will be a kind of wordsmithing project...

<Chuck> +1 with slide

<general acceptance of the swap for now>

rbm: slide 10, this is protocols, tests whether a process was completed

this doesn´t assess quality per se, it just assesses that the protocol was followed

I see these as broad reviews against a process that can´t be evaluated quantitatively

<Chuck> MichaelC: Not sure if my thought fits under this, way I've been thinking of it is that they do everything you described, and additionally it has procedures for evaluating or ensuring that you are doing well, not just going through the motions.

<Chuck> MichaelC: That would still have to be... there may be some quantitative tests. It would be a protocol because there would be a qualitative test.

<Chuck> MichaelC: At conformance levels you adopt the protocol and follow the words, at a higher level you evaluate that the protocol is yielding positive results.

<Chuck> MichaelC: Not sure if it needs to be looked at separately or be included.

<Chuck> MichaelC: Measurement procedures for both following the protocol and quality of results.

<Chuck> MichaelC: Protocol would define procedures. In sub-bullets, something about report on internal measurement, on qualitative measurements.

<jennifer> The report would indicate the passing quality of the protocol guidance.

<wording slide to incorporate>

mc: +1 to how it looks now

st: will there be a list of protocols?

mc: think they would be like methods and listed similarly

st: mention that here?

rbm: capture for future work

jg: would we define requirements for a protocol that organizations can take up, or define our own protocols?

the former is more complex, the latter requires us to maintain more

keeping in mind that lay people should be able to conform, meaning should be able to follow a protocol

need some objective standards for how that works

rbm: added

ca: I would like for protocols to be flexible enough that orgs can craft their own

<Jaunita_George> +1 it would be great if they could apply

mc: +1, we can´t not let orgs define protocols, we can´t manage

jg: lagniappe if org protocols are shareable for other orgs as well

<ShawnT> I'm pushing for my organization to create protocols to add to the list in the future

rbm: various edits, take a look

jg: also looking at repeatability

rbm: setting that aside for this presentation

if we can agree on these basics, it will be easier to address questions like that

js: until larger group has established conformance levels, we should defer that aspect

<Zakim> Chuck, you wanted to aks about "The report would indicate the passing quality of the protocol guidance."

ca: looking for ^ seems to have gone missing

mc: seems covered in rewording

ca, js: ok, seems there

slide 11, grid the test objects with the test types

effectively goes from most precise in origin (smallest unit, constant tests) to least precise in end (aggregate, protocol) as a continuum

js: looking at machine testable stuff

rbm: currently machine testable is close to the origin, over time we expect it to move further in the table

ca: this slide puts it all together for me

think we can look at machine testing as adding test types it can cover

rbm: edits bullet

js: +1

rbm: slide 12, the @@ group tried this breakdown

it helped me to see that the needs for a given outcome span various parts of that matrix

jg: should we leave it to orgs to sort out how to apply?

js: original vision was not to have to submit to wcag, just document what they did

<jennifer> We could design a process of putting a Github ticket in. to pick up later!

mc: if we define requirements for protocols, any protocol that meets requirements should be allowed

would like us to include a protocol for protocol results to allow conformance calculation and sharing

as we work on things, we can differentiate between identifying ¨protocols are needed here¨ vs ¨our guidance is not complete until we have a provided a protocol¨

Minutes manually created (not a transcript), formatted by scribe.perl version 185 (Thu Dec 2 18:51:55 2021 UTC).


Succeeded: s/example:/rbm: example:/

Maybe present: ca, jg, js, mc, rbm, st