IRC log of wcag-act on 2017-10-23

Timestamps are in UTC.

13:51:53 [RRSAgent]

RRSAgent has joined #wcag-act

13:51:53 [RRSAgent]

logging to http://www.w3.org/2017/10/23-wcag-act-irc

13:51:55 [trackbot]

RRSAgent, make logs public

13:51:58 [trackbot]

Meeting: Accessibility Conformance Testing Teleconference

13:51:58 [trackbot]

Date: 23 October 2017

13:52:00 [Wilco]

agenda?

13:52:05 [Wilco]

clear agenda

13:52:18 [Wilco]

agenda+ Doodle poll for new meeting time https://doodle.com/poll/668bk8xeyqyxpg4t

13:52:35 [Wilco]

agenda+ Consistency: Possible outcome values - suggestion to use EARL terminology https://github.com/w3c/wcag-act/issues/117

13:52:43 [Wilco]

agenda+ Avoid hard statements about rules not being automated https://github.com/w3c/wcag-act/issues/120

13:52:50 [Wilco]

agenda+ Relationship between ACT rule and an accessibility requirement - one to one only or is one to many allowed? https://github.com/w3c/wcag-act/issues/122

13:52:57 [Wilco]

agenda+ Section 4 - Test Subject Types (input) - would like to see more than just HTML types https://github.com/w3c/wcag-act/issues/123

13:59:49 [rdeltour]

rdeltour has joined #wcag-act

14:00:13 [maryjom]

maryjom has joined #wcag-act

14:02:29 [anne_thyme]

anne_thyme has joined #wcag-act

14:03:51 [cpandhi]

cpandhi has joined #wcag-act

14:04:07 [Wilco]

present+

14:04:08 [rdeltour]

present+

14:04:14 [rdeltour]

scribenick: rdeltour

14:04:14 [cpandhi]

present+

14:04:16 [maryjom]

present+

14:04:17 [rdeltour]

scribe: Romain

14:04:17 [anne_thyme]

present+

14:04:23 [rdeltour]

zakim, takeup next

14:04:23 [Zakim]

I don't understand 'takeup next', rdeltour

14:04:29 [rdeltour]

zakim, take up next

14:04:29 [Zakim]

agendum 1. "Doodle poll for new meeting time https://doodle.com/poll/668bk8xeyqyxpg4t" taken up [from Wilco]

14:04:34 [skotkjerra]

skotkjerra has joined #wcag-act

14:04:41 [skotkjerra]

present+

14:05:10 [rdeltour]

wilco: I think that this time isn't working for everyone. Please fill up the form

14:05:24 [rdeltour]

… looks like Thursday is currently the best option

14:06:03 [rdeltour]

zakim, take up next

14:06:03 [Zakim]

agendum 2. "Consistency: Possible outcome values - suggestion to use EARL terminology https://github.com/w3c/wcag-act/issues/117" taken up [from Wilco]

14:06:38 [rdeltour]

wilco: right now, we've got this weird thing called "undetermined" in section 8.2

14:06:48 [rdeltour]

… it's not part of EARL, we made it up ourselves

14:07:05 [rdeltour]

... first question is do we even need this concept?

14:07:28 [rdeltour]

… we answered that last week. next quesiton is should it be "undetermined" or "canttell" as is used in EARL?

14:07:43 [rdeltour]

Stein Erik: in my perspective, OK to use EARL's technology

14:07:50 [rdeltour]

wilco: sounds right to me

14:08:14 [rdeltour]

Stein Erik: we said we needed a dinstinction between "inapplicable" and "undetermined"

14:08:40 [rdeltour]

Charu: I think we add a discussion earlier, trying to remember why we didn't kept "canttell"

14:09:12 [rdeltour]

… we have "untested", and "not fully tested". isn't that the case when a fully automated tool isn't able to figure out the resulsts?

14:09:23 [rdeltour]

wilco: there's a bunch of terms, we just have to pick one

14:09:38 [rdeltour]

… another way we could use is to have an explicit mapping to EARL

14:10:02 [rdeltour]

... "canttell" may read a bit weird, which is why I think we used "undetermined"

14:10:13 [rdeltour]

… but I do personally prefer sticking with EARL terminiology

14:10:27 [rdeltour]

Stein Erik: either way we need a definition

14:10:39 [Manoj]

Manoj has joined #wcag-act

14:11:13 [rdeltour]

wilco: I think the reason the EARL group came up with "cannot tell" is that it's not technology specific

14:11:18 [MoeKraft]

MoeKraft has joined #wcag-act

14:11:51 [rdeltour]

Charu: when we get the "undetermined" outcome it's that it's not fully tested. can we use "not fully tested"? we'd need to map that to "cannot tell"

14:12:10 [rdeltour]

wilco: are you OK sticking with "cannot tell" and put a definition in our document?

14:12:37 [rdeltour]

charu: EARL's definition is not very specific, so if we have our own definition it should be OK

14:13:24 [rdeltour]

wilco: ok, let's stick with "cannot tell", we'll work on a definition

14:13:34 [rdeltour]

anne: ??? [bad audio]

14:14:43 [rdeltour]

wilco: I think the reason we put it in there is that it's not a result from a rule. An aggregation can be "inapplicable"

14:15:32 [rdeltour]

… it's not described explicitly in 8.2, you can find it in appendix

14:15:57 [rdeltour]

anne: we can think of cases where we can have "inapplicable" for the rule itself. we have tests like that

14:16:01 [rdeltour]

wilco: example?

14:16:36 [rdeltour]

anne: one where we start out by searching a constant and test if it's included in landmark. if it's not there, the rule is inapplicable

14:17:09 [rdeltour]

… we do have tests where for us it would be necessary to have inapplicable as an outcome

14:17:27 [rdeltour]

Stein Erik: [clarifies the use case]

14:18:06 [rdeltour]

anne: we don't start out from a CSS selector

14:18:33 [rdeltour]

Stein Erik: what you need to test is a parent element instead of searching all children elements

14:18:53 [rdeltour]

… you pass a parent, but then figure it's not relevant when testing the children

14:19:43 [rdeltour]

wilco: are you saying one test could impact the result of another test?

14:20:01 [rdeltour]

Stein Erik: yes, which may already be possible. it depends on how you split tests into steps

14:20:15 [rdeltour]

wilco: maybe you can explore this further and come back to the group

14:20:35 [rdeltour]

… hopefully the time change will allow the developer to attend our calls as well

14:20:46 [rdeltour]

anne: this was one of the results from our internal testing

14:21:28 [rdeltour]

wilco: [musings about the format not requiring CSS selectors]

14:21:58 [rdeltour]

wilco: the conclusion stands, use "canttell" and add a definition for it, better than the one in EARL

14:22:02 [rdeltour]

zakim, take up next

14:22:02 [Zakim]

agendum 3. "Avoid hard statements about rules not being automated https://github.com/w3c/wcag-act/issues/120" taken up [from Wilco]

14:22:58 [rdeltour]

wilco: we may not be too explicit about which rule may be automated and which may not, technology is evolving

14:23:05 [rdeltour]

… I kind of agree

14:23:24 [rdeltour]

... I had several people asking that explicitly, to make it easier to undertsand one or the other might fall

14:23:37 [rdeltour]

… it's more "we think it's this one", not a necessity

14:23:46 [rdeltour]

stein erik: can you give an example?

14:24:59 [rdeltour]

wilco: this is mentioned in 5.2

14:25:09 [rdeltour]

stein erik: OK. I think it's a valid point

14:25:50 [rdeltour]

wilco: we could either remove it altogether, that doesn't mean that the test mode can't be part of the eventual result, it's left up to the implementor

14:25:55 [rdeltour]

… not even sure it's required by EARL

14:26:03 [rdeltour]

... the other option is to have a suggested test mode

14:26:09 [rdeltour]

charu: I like that

14:26:15 [rdeltour]

+1

14:27:06 [rdeltour]

charu: in my thinking, if you have some tests that used to be manual but somebody found a way to automate that, the rule cam be revised

14:27:22 [rdeltour]

wilco: you don't much care how you implement it as long as you get to the same results

14:27:30 [rdeltour]

stein erik: makes sense

14:27:50 [Wilco]

Decision: Update 5.2 to say "Suggested test mode"

14:28:13 [rdeltour]

stein erik: is it even necessary

14:28:46 [MoeKraft]

I prefer recommended too

14:28:57 [rdeltour]

rdeltour: maybe "suggested" mean "recommended"? so "indicative" or "informative" is more appropriate?

14:29:44 [rdeltour]

moe: I just think "recommended" carries more weight.

14:29:53 [rdeltour]

wilco: what about "indicative"

14:30:35 [rdeltour]

moe: somewhat vague. indicated by whom?

14:30:45 [rdeltour]

rdeltour: [clarifies]

14:30:45 [MoeKraft]

defined maybe

14:31:06 [rdeltour]

stein erik: maybe "this rule has been designed or tested using this mode" ?

14:31:17 [rdeltour]

… something like "known test mode"

14:31:35 [rdeltour]

wilco: what about "default test mode"?

14:31:55 [rdeltour]

stein erik: I like the idea of default.

14:32:05 [rdeltour]

moe: I was thinking "defined"

14:32:19 [rdeltour]

… you would need to define what the default is

14:32:36 [rdeltour]

... in the rules format, we have a specified mode

14:32:50 [rdeltour]

charu: I was more leaning to "indicative" or "specified"

14:33:08 [rdeltour]

stein erik: isn't it very decided in a way? very firm?

14:33:21 [MoeKraft]

5.2 states: Test case mode identifying whether test case steps are automated, semi-automated or require manual testing.

14:33:34 [rdeltour]

anne: to me "specified" suggests that you don't follow the spec if you don't do this.

14:34:05 [rdeltour]

stein erik: "proposed"?

14:34:12 [rdeltour]

wilco: +1

14:34:15 [rdeltour]

charu: +1

14:34:17 [MoeKraft]

default: a preselected option adopted by a computer program or other mechanism when no alternative is specified by the user or programmer.

14:34:18 [rdeltour]

+1

14:34:23 [maryjom]

+1

14:34:44 [MoeKraft]

proposed: put forward (an idea or plan) for consideration or discussion by others.

14:34:45 [rdeltour]

zakim, take up next

14:34:45 [Zakim]

agendum 4. "Relationship between ACT rule and an accessibility requirement - one to one only or is one to many allowed? https://github.com/w3c/wcag-act/issues/122" taken up [from

14:34:48 [Zakim]

... Wilco]

14:36:06 [rdeltour]

wilco: pretty easy one I think

14:36:27 [rdeltour]

… are Rules a 1 to 1 relationship to SC?

14:36:48 [rdeltour]

stein erik: isn't it explcit in the text that they can be 1-1 or 1-many?

14:36:56 [rdeltour]

charu: yes, we have rules like that

14:37:28 [rdeltour]

anne: yes, in section 3.3 Accessibility Requirements

14:38:57 [Wilco]

https://github.com/w3c/wcag-act/issues/135

14:39:09 [rdeltour]

rdeltour: [describes his comment on #135]

14:40:19 [rdeltour]

moe: we were trying to replicate WCAG verbage

14:42:31 [rdeltour]

all: [debating an appropriate rewording]

14:43:31 [rdeltour]

wilco: what about "An ACT Rule MUST be …"

14:44:01 [rdeltour]

rdeltour: what about then "An ACT Rule is a complete or partial test…"

14:44:07 [rdeltour]

wilco: sounds ok to me

14:44:12 [MoeKraft]

An ACT Rule MAY test only part of an accessibility requirement, or MAY cover more than one requirement.

14:44:53 [rdeltour]

moe: there are several places where we use MAY, should we replace all of them?

14:45:32 [rdeltour]

… ok, let's just focus on this occurrence for now

14:45:50 [MoeKraft]

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this spe[CUT]

14:46:16 [rdeltour]

zakim, take up next

14:46:16 [Zakim]

I do not see any more non-closed or non-skipped agenda items, rdeltour

14:46:26 [Wilco]

agenda?

14:47:14 [Wilco]

https://github.com/w3c/wcag-act/issues/139

14:49:57 [rdeltour]

rdeltour: [rephrases what he put in #139]

14:50:13 [rdeltour]

wilco: kinda like the idea, helpful to get the full picture. Moe WDYT?

14:51:06 [rdeltour]

rdeltour: the idea is to have a section dedicated to describe the logic of evaluating a rule

14:51:24 [rdeltour]

… rather than have it conflated with the Rule description

14:52:02 [rdeltour]

... like in the "Test cases" section, which starts with statements on thte rules description itself, but ends with describing the logic on how the rule is processed an resutls aggregated

14:52:27 [rdeltour]

moe: romain, you're self assigned, do yo want me to wokr with you?

14:53:05 [rdeltour]

romain: sure, happy to propose something, I can try before TPAC

14:53:09 [rdeltour]

wilco: sounds fine

14:53:49 [Wilco]

https://github.com/w3c/wcag-act/issues/133

14:55:14 [rdeltour]

rdeltour: not even sure if we want each rule to be universally identified or not?

14:55:34 [rdeltour]

wilco: in auto-wcag we're using string IDs that we can put in URLs

14:57:11 [rdeltour]

rdeltour: for isntance, I'm developing a rule set, am I allowed to use "image-description" as a rule ID?

14:57:42 [rdeltour]

wilco: one solution is to mandate a URL

14:58:02 [rdeltour]

rdeltour: or just be permiissive and state that we allow anything

14:58:33 [rdeltour]

anne: in my experience, we'd have question on what is the fornat for this identifier

14:59:53 [rdeltour]

rdeltour: +1, we could have a clarification that defining the identifier is left to the implementor

14:59:57 [rdeltour]

wilco: we can add a note

15:00:03 [rdeltour]

rdeltour: +1

15:00:22 [rdeltour]

wilco: I think we're done!

15:08:26 [rdeltour]

trackbot, end meeting

15:08:26 [trackbot]

Zakim, list attendees

15:08:26 [Zakim]

As of this point the attendees have been anne_thyme, shadi, wilco, Mary_Jo_Mueller, tobias, rdeltour, cpandhi, MoeKraft, maryjom, skotkjerra

15:08:34 [trackbot]

RRSAgent, please draft minutes

15:08:34 [RRSAgent]

I have made the request to generate http://www.w3.org/2017/10/23-wcag-act-minutes.html trackbot

15:08:35 [trackbot]

RRSAgent, bye

15:08:35 [RRSAgent]

I see no action items