Meeting minutes
announcements
ca: JF away for indeterminate time
have asked Jaunita George to step in as subgroup lead
needs a different time
what´s good for people?
for me, Friday afternoons Mountain are good
<ShawnT> I'm good at any time
mc: Friday after noon Eastern best
<Rachael> Friday afternoons often work for me.
js: later Friday afternoon Eastern better
ca: will start a schedule poll based on that input
<ShawnT> 4ish is tough for me
Review https://docs.google.com/presentation/d/1b5xHQWBzoYdKp7BfPgIUBCpz-yaDOx_kSq_HlQxcFh0/edit#slide=id.g115901c77f1_0_218
rbm: <shares screen to present, scribing only non-slide content>
want to present this to AG tuesday
make sure different voices represented
so want input here
on slide 5, we have the need to go beyond WCAG 2 scope yet ensure things testable at pass / fail level
note the terminology is placeholder, focus on the concepts
ca: I´ve heard that we want tests that are beyond pass / fail
use rubriks etc. to convert to test scores
so different evaluation methods can convert to pass / fail?
not contradictory?]
rbm: agree, not contradictory
one idea is to report on measurements, with a sliding scale
another is to @@
third is to move the sliding scale down to the rubrik level
ca: that may address some concerns we have
rbm: slide 6, there are 4 concepts of scope for tests
4 types of tests slides 7 - 10
constant tests and condition tests have similarities, but think it´s useful to differentiate them
ca: seems to describe what we´ve done
rbm: example: text alternatives has 2 types of test, the constant test about providing it, and the condition test about its quality
meaningful sequence is one that´s mostly just condition test
slide 9, test case as used here is a new concept to us
instead of defining conditions for test, it defines the type of conditions to set
this is similar to how requirements for tests are defined in software engineering
should address the ¨you know it when you see it but can´t define it¨ situation
there can be known failures
this can be use for e.g., content usable testing, where it´s a lot about consistency
st: definition of affordance?
rbm: it is the visual indication of what something does
<Chuck> MichaelC: Been told by usability that this is something that can cause concerns. The affordance is what the object can do, and the perceived affordance is a person's awareness of that.
<Chuck> MichaelC: We may get debate if we use that definition. I want to not use just visual indication. The indication of its ability can happen in any appropriate channel.
<Chuck> Rachael: Alternatives of what we can call it?
rbm: chose the common usage of that term, but know it´s debated, let´s sub
js: indicator, cue?
<Chuck> Jennifer: Indicator/cue of usage.
rbm: good enough for now
ca: yeah, no wordsmithing!
rbm: down the road we´ll need to look at editorial consistency
that will be a kind of wordsmithing project...
<Chuck> +1 with slide
<general acceptance of the swap for now>
rbm: slide 10, this is protocols, tests whether a process was completed
this doesn´t assess quality per se, it just assesses that the protocol was followed
I see these as broad reviews against a process that can´t be evaluated quantitatively
<Chuck> MichaelC: Not sure if my thought fits under this, way I've been thinking of it is that they do everything you described, and additionally it has procedures for evaluating or ensuring that you are doing well, not just going through the motions.
<Chuck> MichaelC: That would still have to be... there may be some quantitative tests. It would be a protocol because there would be a qualitative test.
<Chuck> MichaelC: At conformance levels you adopt the protocol and follow the words, at a higher level you evaluate that the protocol is yielding positive results.
<Chuck> MichaelC: Not sure if it needs to be looked at separately or be included.
<Chuck> MichaelC: Measurement procedures for both following the protocol and quality of results.
<Chuck> MichaelC: Protocol would define procedures. In sub-bullets, something about report on internal measurement, on qualitative measurements.
<jennifer> The report would indicate the passing quality of the protocol guidance.
<wording slide to incorporate>
mc: +1 to how it looks now
st: will there be a list of protocols?
mc: think they would be like methods and listed similarly
st: mention that here?
rbm: capture for future work
jg: would we define requirements for a protocol that organizations can take up, or define our own protocols?
the former is more complex, the latter requires us to maintain more
keeping in mind that lay people should be able to conform, meaning should be able to follow a protocol
need some objective standards for how that works
rbm: added
ca: I would like for protocols to be flexible enough that orgs can craft their own
<Jaunita_George> +1 it would be great if they could apply
mc: +1, we can´t not let orgs define protocols, we can´t manage
jg: lagniappe if org protocols are shareable for other orgs as well
<ShawnT> I'm pushing for my organization to create protocols to add to the list in the future
rbm: various edits, take a look
jg: also looking at repeatability
rbm: setting that aside for this presentation
if we can agree on these basics, it will be easier to address questions like that
js: until larger group has established conformance levels, we should defer that aspect
<Zakim> Chuck, you wanted to aks about "The report would indicate the passing quality of the protocol guidance."
ca: looking for ^ seems to have gone missing
mc: seems covered in rewording
ca, js: ok, seems there
slide 11, grid the test objects with the test types
effectively goes from most precise in origin (smallest unit, constant tests) to least precise in end (aggregate, protocol) as a continuum
js: looking at machine testable stuff
rbm: currently machine testable is close to the origin, over time we expect it to move further in the table
ca: this slide puts it all together for me
think we can look at machine testing as adding test types it can cover
rbm: edits bullet
js: +1
rbm: slide 12, the @@ group tried this breakdown
it helped me to see that the needs for a given outcome span various parts of that matrix
jg: should we leave it to orgs to sort out how to apply?
js: original vision was not to have to submit to wcag, just document what they did
<jennifer> We could design a process of putting a Github ticket in. to pick up later!
mc: if we define requirements for protocols, any protocol that meets requirements should be allowed
would like us to include a protocol for protocol results to allow conformance calculation and sharing
as we work on things, we can differentiate between identifying ¨protocols are needed here¨ vs ¨our guidance is not complete until we have a provided a protocol¨