wilco: pass and fail directly inherit from EARL
shadi: so technically WCAG does not define pass and fail, it talks about satisfying the requirements, or meeting the requirements
... I did put some editorial suggestions that I think will help this. We often use pass and fail. I think pass fail for rules, and satisfy/not satify for requirements
wilco: How would we then better address this confusion?
shadi: Instead of using satisfy and not satisfy for rules. In this way you could pass a rule, but not satisfy a requirement
wilco: when it comes to success criteria, most professionals do pass/fail/inapplicable, so your proposal is not the common language
kathy: So my understanding is that with the atomic rules you can pass or fail, and the composed rules and determine if it meets the requirement
wilco: technically speaking you can never pass a success criteria, you can just not satisfy. Its more towards making a best effort to show you do not have the listed problems
alistair: there are only a few instances where we have said a success criteria can be passed or met. why don't we just specify our language somewhere that a success criteria has been met/unmet
... i think just changing the language a bit will fix
shadi: Agreed, i think we just need to spell things out and make sure we are clear.
wilco: Assigning shadi to this issue
<shadi> https://lists.w3.org/Archives/Public/public-wcag-act-comments/2018Aug/0000.html
wilco: I will take a look and assign this to me. I will bring it back to the group after we have gotten a resolution
shadi: in my draft i did make a few changes to this section.
... take a look at my draft and then we can come back to the issue.
alistair: what they want is for every line in the test to be specified, i don't think there is much we can do about this
stein: I think we should remove/adjust the sentence. As it is, it would require significant changes in order to work. he has a point, but it is not very solvable
... wants empirical method, but I am not sure it would be as rigorous as he would want
wilco: i agree
shadi: I did also raise an issue on benchmarking and accuracy, and it says the accuracy is measured by what an expert would claim
... does not say what percentage that needs to be, it just says here is how to measure the expert
... and what constitutes an expert, who qualifies for that.
... we will have to think about the judgement behind an act rule
alistair: that is a very broad topic, for example you might want an industry quality mark that says a level of accuracy
shadi: we kinda have that implicitly in the review group. I think there needs to be some review process for rules
alistair: the problem that you have is that W3C is standards and not a quality review.
stein: isn't this what we are doing with our review process already.
wilco: I think the argument is that it is not part of the standards.
shadi: I think we need to reference from the standards that a review process is necessary for the rules
... i do think we make a claim we want to help harmonization, but i am not sure that we are doing that
alistair: if you say you need a group of experts to get together, say 20, you could still have groups in one location e.g. America, Europe, etc. that deals with these things
... and that may not really be harmonization
wilco: Publishing rules in a given format that we have all talked about, i think helps in harmonization by itself
shadi: I don't think we really provide a format, there are multiple ways people could go out and still follow the standard
alistair: its the "can" bit that is the problem
... the word can is just making the rule "allowed" to pass, but isn't actually passing it. It should be more direct
stein: In principle, it should be the outcome of the rule that passes
wilco: No, its that the test result is to pass
... I will leave a comment and assign it to myself
wilco: I think we are all good with that.
alistair: out of interest, what is a local law
wilco: state or regional
alistair: we don't have any local laws with that
<shadi> "local or regional law"?
stein: Is there possible confusion, or is it just not applicable to some countries
wilco: the proposal is to take out "local"
wilco: I have already been doing that in some of the stuff i am doing for auto-wcag.
<shadi> +1
wilco: anyone think this is not a good idea?
alistair: I feel like its fairly obvious between the two. The problem with adding extra metadata is that in can cause problems.
wilco: I think the suggestion is just to be explicit
alistair: Would prefer it to be optional?
anne: Wouldn't we have the same problem if it was optional? Have people misusing
... I don't see what it should be optional, should either be there always or never
wilco: I think its worth being explicit about it.
moe: Looking at some of shadi's suggestions, i think this falls in line with his changes and will help.
wilco: We don't have a way now, but i suppose we could give examples
anne: The could be examples and then a link to a page with items people could pick and choose form
wilco: thinking through practically, would that be like a note that we can add stuff to over time
shadi: we can not make it a normative requirement, it would have to be informal
... the requirement is either to have a detailed description or a reference to a common aspect
... If you want to specify a particular way, you have to make it normative, if you want to leave it open then you can just provide a list of informative references
wilco: seems editorial
... I can update it
wilco: This is the pass or fail question we discussed earlier coming back again
anne: We state that rules must be consistent with accessibility requirements. It should not list the rule, but the composed rule. The composed rule should list the accessibility requirement
... we referred to the section on aggregation for how passes should work.
wilco: Please put that in a comment for this issue.
wilco: going to set up a call with annika.
alistair: I don't understand her comment. Where she is finding that a higher number means that the approach performed worse.
stein: lets take a call and see if she can clarify
wilco: she is correct, I used aggregation in two different ways
... just need to do some editorial work to clarify this
maryjo: or clarify that there are two levels of aggregation
alistair: the composed rules are not really an aggregation
wilco: assigning to myself and shadi