Meeting minutes
<JF> MC: lowercase outcomes = a11y benefit, upper-case Outcomes = specific WCAG requirement
1. Content usable is a great example of a protocol but good to keep in mind that parts may become methods as well as a protocol 2. Key question: Should outcomes be scored or reported on? Pros and cons to both. 3. It is more important to identify key questions with pros and cons to bring to the group sooner rather than later than it is to spend a lot of time away from the group trying to get to a perfect solution
Michael: In past talked about tagging things by author, technology, etc. We could expand tagging to be more contextual. Context by language. May be controversial but needed.
… protocols are about procedures you use to meet an outcome that is not easily measurable
JF: If we work off content usable to be an example protocol, we provide focused guidance to content authors but can't measure.
<jenniferS> It's about showing one's homework.
<jenniferS> Showing one's thinking process for how they delivered what they delivered.
<Zakim> MichaelC, you wanted to define ¨define outcomes¨ :) and to evolution and to separate steps from support
Rachael: Pieces of Content usable vs. Content usable as a whole document
Michael: Steps you follow to achieve an outcome. Everything else is supporting documents.
… that is how to separate those.
… focus measuring on what we can measure nad make everything else protocols.
jenniferS: The way the protocols effort is show your homework. Show how you attempted to meet standards. Show what you did.
… a statement on how we approached meeting standard. Where we took a different road and why.
JF: Yes, the larger proposal did have as part of the conformance piece. The moment you make a public declaration, you create a risk.
… that is where accountability comes into play.
jenniferS: Showing your thinking. Can get different results.
… doesn't necessarily contribute to score.
Jaunita_George: If its not measurable, it is hard to be accountable. If we use it in a similar way to sufficient techniques then it can be accountable. Only adopt protocol that has measurable criteria. Alt text can't exceed 136 characters. Don't know if its descriptive. Would a reasonable person think its decriptive. There is a degree of objectivity you want to introduce.
… If someone says we used this protocol to make this accessible, there is no way to see into it. It means that everything has to go to court to prove. Also, this is not legal advice.
… No benefit of forcing litigation.
<Zakim> MichaelC, you wanted to about wcag conformance vs protocol reporting and to separate the work of following a protocol from the work of reporting it and to say IMO we are here to make it possible to include things that can´t be measured in wcag and to say protocol ~~ sufficient techniques and to say protocols *can* measure outcomes
Michael: In reporting, the way I'm thinking of things in the propsoal is that a protocol has its own test procedure to test how well the protocol is being implemented and that reports a result that comes into your conformance statement. If its useful to report on individual protocols then it is doable.
… when you define a protocol there is work to define protocol , measure adn report protocol. Each part is distinct and necessary.
… I think the point of protocols is to achieve outcomes that we don't know how to measure.
<Jaunita_George> +1 to Michael
+1
… someone mentioned protocols as sufficient techniques and they are likely something like that.
… at the Bronze level it might be yes/no question. At higher levels it may be user testing. That may make us feel more comfortable.
JF: I agree protocols are for use on things that are hard to measure. Why do we talk abotu measuring?
MichaelC: Each protocol has to define how to measure.
<cwadams> I will cover for Rachael scribing.
JF: I see protocols being part of all three levels. In my mind, protocols and outcomes contribute to a score. Its your final score that determines level.
… I'm not a fan of continuing to keep them in separate buckets.
JF: They need to contribute to score no matter
MichaelC: That is included in the proposal. Protocols define steps and how those steps are followed. As I understand it, most methods don't evaluate outcomes they evaluate whether the code is correct and we hope it supports the outcome.
… scoring is outside this groups scope. I see protocols returning some result. I do think we need to make a distinction between protocols and methods. I think there will be a distinction that is important regardless of how they work in the conformance model.
JF: discussion of experience with bypass blocks
… need to focus on outcomes. Ask question of whether there is another technique that meets requirement.
… if we all understand that whitespace is a requirement then we start to understand. The difference between methods and protocols is the level of subjectivity.
Michael: Definition of subjective is subjective. Always the case that there is a sufficient technique we missed. WE need to address that.
<Zakim> Rachael, you wanted to say for protocols its not measuring its definign success and to say where to integrate protocols and what levels is another key quesiton
<cwadams> Rachael: You wanted to say... I think we should take away measuring. We are talking about defining success. Maybe "evaluate".
<cwadams> JF: That one thing you said is a critical point.
<cwadams> JF: Last week we did some straw poll decisions. I suggest we do a straw poll now.
<cwadams> JF: The difference between method and protocol is...
straw poll: Methods are measured; Protocols define/evaluate success
<cwadams> Jaunita: Defining success of outcome or which?
<cwadams> Jake: Anything can be a success because it is subjective. We can't use "success".
Jake: I liked the way MIchael stated it "Evaluate how well you followed the protocol"
<cwadams> Jake: Michael used "evaluate" in his sentence. Maybe there's another word. "Success" can be almost everything.
<cwadams> Rachael: There's a lot. The other thing, capture "where to integrate protocols" is a key question. "Capture" is an alternative.
<cwadams> JF: Can we go back to the decision/straw poll? Tease out the wording?
<cwadams> JF: Difference between a method and protocol.
<cwadams> Rachael: "Evaluate how well protocol was followed?"
<cwadams> MC: <constructing potential definition and poll>
<MichaelC> Proposal: Methods test result of implementing the method; Protocols evaluate the process of following the protocol
<cwadams> Jennifer: Rachael's is more plain language.
<cwadams> MC: I would use hers if we can.
<cwadams> JF: Both of those seem circular.
<Rachael> I think you need measure in methods
<cwadams> MC: I am thinking of how we program.
<cwadams> JF: Let's use "evaluated".
<cwadams> MC: New wording coming up.
<cwadams> JF: Yes Jaunita.
<MichaelC> Alternate: Methods are tested for results of following the method; Protocols are evaluated for quality of following its procedures
<cwadams> Jake: If I may, I heard something about measuring training or other processes that are embedded in the org.
<cwadams> Jake: Those are exactly the discussions we had. We had a proposal over there.
<cwadams> Jake: Protocols and maturity modeling should be looked at together.
<cwadams> MC: That sounds interesting.
<cwadams> JF: I spoke with Fazio before, I agree it may be useful for him to join this call.
<cwadams> JF: short time left.
<cwadams> JF: Going back to MC's proposed language.
<cwadams> JF: Jaunita asked if protocols would include methods, I think we can circle back to that. But right now Michael's statement making a distinction is a useful concept to capture.
<cwadams> Chuck: It still feels too circular.
<Jaunita_George> +1 only if protocols include methods and would be sufficient for outcomes if all methods in protocol are met
<cwadams> Rachael: Needs a bit more word smithing.
<cwadams> JF: I think this is mostly wordsmithing. I think the attempt to say it... Methods are specific tests looking for specific outcomes. Protocols are about following a process or procedure.
<Rachael> Protocols evaluate process; Methods measure implementations
<cwadams> JF: Goes back to last week, protocols measure inputs and not outcomes.
<cwadams> Jake: Before call is over, I have an idea. Measuring the maturity of following the protocol is possible. We have a proposal for it.
<cwadams> Jake: It's not very objective, but it is not difficult.
<cwadams> Jake: A bit off, a couple of weeks ago protocols had to do with something with design and development, but training was easy to evaluate.
<cwadams> MC: Thinking of that as part of my proposal.
<Rachael> You have identified a lot of important questions and points which would be great to capture somewhere
<cwadams> JF: No decisions today, we are having a fruitful conversation. What we are tackling is very squishy. Thank you all for contributing today.
<cwadams> JF: The co-chairs need to dash off.
<Rachael> bye