See also: IRC log
wilco: I think that this time isn't working
for everyone. Please fill up the form
... looks like Thursday is currently the best option
wilco: right now, we've got this weird
thing called "undetermined" in section 8.2
... it's not part of EARL, we made it up ourselves
... first question is do we even need this concept?
... we answered that last week. next quesiton is should it be
"undetermined" or "canttell" as is used in EARL?
Stein Erik: in my perspective, OK to use EARL's technology
wilco: sounds right to me
Stein Erik: we said we needed a dinstinction between "inapplicable" and "undetermined"
Charu: I think we add a discussion earlier,
trying to remember why we didn't kept "canttell"
... we have "untested", and "not fully tested". isn't that the case when
a fully automated tool isn't able to figure out the resulsts?
wilco: there's a bunch of terms, we just
have to pick one
... another way we could use is to have an explicit mapping to EARL
... "canttell" may read a bit weird, which is why I think we used
"undetermined"
... but I do personally prefer sticking with EARL terminiology
Stein Erik: either way we need a definition
wilco: I think the reason the EARL group came up with "cannot tell" is that it's not technology specific
Charu: when we get the "undetermined" outcome it's that it's not fully tested. can we use "not fully tested"? we'd need to map that to "cannot tell"
wilco: are you OK sticking with "cannot tell" and put a definition in our document?
charu: EARL's definition is not very specific, so if we have our own definition it should be OK
wilco: ok, let's stick with "cannot tell", we'll work on a definition
anne: ??? [bad audio]
wilco: I think the reason we put it in
there is that it's not a result from a rule. An aggregation can be
"inapplicable"
... it's not described explicitly in 8.2, you can find it in appendix
anne: we can think of cases where we can have "inapplicable" for the rule itself. we have tests like that
wilco: example?
anne: one where we start out by searching a
constant and test if it's included in landmark. if it's not there, the
rule is inapplicable
... we do have tests where for us it would be necessary to have
inapplicable as an outcome
Stein Erik: [clarifies the use case]
anne: we don't start out from a CSS selector
Stein Erik: what you need to test is a parent element instead of searching all children elements
scribe: you pass a parent, but then figure it's not relevant when testing the children
wilco: are you saying one test could impact the result of another test?
Stein Erik: yes, which may already be possible. it depends on how you split tests into steps
wilco: maybe you can explore this further
and come back to the group
... hopefully the time change will allow the developer to attend our
calls as well
anne: this was one of the results from our internal testing
wilco: [musings about the format not
requiring CSS selectors]
... the conclusion stands, use "canttell" and add a definition for it,
better than the one in EARL
wilco: we may not be too explicit about
which rule may be automated and which may not, technology is evolving
... I kind of agree
... I had several people asking that explicitly, to make it easier to
undertsand one or the other might fall
... it's more "we think it's this one", not a necessity
stein erik: can you give an example?
wilco: this is mentioned in 5.2
stein erik: OK. I think it's a valid point
wilco: we could either remove it
altogether, that doesn't mean that the test mode can't be part of the
eventual result, it's left up to the implementor
... not even sure it's required by EARL
... the other option is to have a suggested test mode
charu: I like that
+1
charu: in my thinking, if you have some tests that used to be manual but somebody found a way to automate that, the rule cam be revised
wilco: you don't much care how you implement it as long as you get to the same results
stein erik: makes sense
<Wilco> Decision: Update 5.2 to say "Suggested test mode"
stein erik: is it even necessary
<MoeKraft> I prefer recommended too
rdeltour: maybe "suggested" mean "recommended"? so "indicative" or "informative" is more appropriate?
moe: I just think "recommended" carries more weight.
wilco: what about "indicative"
moe: somewhat vague. indicated by whom?
rdeltour: [clarifies]
<MoeKraft> defined maybe
stein erik: maybe "this rule has been designed or tested using this mode" ?
scribe: something like "known test mode"
wilco: what about "default test mode"?
stein erik: I like the idea of default.
moe: I was thinking "defined"
... you would need to define what the default is
... in the rules format, we have a specified mode
charu: I was more leaning to "indicative" or "specified"
stein erik: isn't it very decided in a way? very firm?
<MoeKraft> 5.2 states: Test case mode identifying whether test case steps are automated, semi-automated or require manual testing.
anne: to me "specified" suggests that you don't follow the spec if you don't do this.
stein erik: "proposed"?
wilco: +1
charu: +1
<MoeKraft> default: a preselected option adopted by a computer program or other mechanism when no alternative is specified by the user or programmer.
+1
<maryjom> +1
<MoeKraft> proposed: put forward (an idea or plan) for consideration or discussion by others.
wilco: pretty easy one I think
... are Rules a 1 to 1 relationship to SC?
stein erik: isn't it explcit in the text that they can be 1-1 or 1-many?
charu: yes, we have rules like that
anne: yes, in section 3.3 Accessibility Requirements
<Wilco> https://github.com/w3c/wcag-act/issues/135
rdeltour: [describes his comment on #135]
moe: we were trying to replicate WCAG verbage
all: [debating an appropriate rewording]
wilco: what about "An ACT Rule MUST be …"
rdeltour: what about then "An ACT Rule is a complete or partial test…"
wilco: sounds ok to me
<MoeKraft> An ACT Rule MAY test only part of an accessibility requirement, or MAY cover more than one requirement.
moe: there are several places where we use
MAY, should we replace all of them?
... ok, let's just focus on this occurrence for now
<MoeKraft> Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this spe[CUT]
<Wilco> https://github.com/w3c/wcag-act/issues/139
rdeltour: [rephrases what he put in #139]
wilco: kinda like the idea, helpful to get the full picture. Moe WDYT?
rdeltour: the idea is to have a section
dedicated to describe the logic of evaluating a rule
... rather than have it conflated with the Rule description
... like in the "Test cases" section, which starts with statements on
thte rules description itself, but ends with describing the logic on how
the rule is processed an resutls aggregated
moe: romain, you're self assigned, do yo want me to wokr with you?
romain: sure, happy to propose something, I can try before TPAC
wilco: sounds fine
<Wilco> https://github.com/w3c/wcag-act/issues/133
rdeltour: not even sure if we want each rule to be universally identified or not?
wilco: in auto-wcag we're using string IDs that we can put in URLs
rdeltour: for isntance, I'm developing a rule set, am I allowed to use "image-description" as a rule ID?
wilco: one solution is to mandate a URL
rdeltour: or just be permiissive and state that we allow anything
anne: in my experience, we'd have question on what is the fornat for this identifier
rdeltour: +1, we could have a clarification that defining the identifier is left to the implementor
wilco: we can add a note
rdeltour: +1
wilco: I think we're done!