<Wilco> https://github.com/w3c/wcag-act/issues/224#issue-338581348
Wilco: This is an updated list of everything we may or may not want to put in our exit criteria
Anne: My comments are from a long time ago
Trevor: Not sure why we need more than two expectations
Wilco: I made this very broad and thought we would cut it down in places
... Unless objections, I agree
Can we summarize Trevor's resolution?
Shadi: Also have comments on cutting down. I think General can be cut down to say that all rules have to conform to the specification. At least two rules at beginning is illusionary. No way to meet all combinations. It will be difficult to determine a specific number .Rules need to conform is sufficient
... In other sections we repeat confirmation requirement. This is are duplicate and unnecessary.
... We need to expand on technology. HTML and something else, e.g. PDF.
... This could show application of the spec on other technology. PDF working group has agree to write 3 rule categories, e.g. color contrast, just need to change the names to have it apply to PDF. Headings could also apply but may require PDF specific format.
... 3rd category of rules only applies to PDF
... Hoping other technologies, e.g. SVG, will also be interested in contributing rules
<romain> we intend to publish ACT rules for EPUB, fwiw
Shadi: Test cases do not have any requirements in the exit criteria. Need something for test cases, e.g. each rule must have at least 2 test cases
Wilco: I agree with general that we can simplify things. I will gladly trim down
Shadi: I think explicit is good but repeating conformance requirements is not necessary
Wilco: Technology: Because we don't explicitly talk about technology in the rules format, aspects under test at least two rules are included.Forces us to do HTML plus something else.
... Is this specific or should we spell out what aspects we should include
Shadi: Aspects are in a technology, e.g. DOM is HTML based, Would be nice to have something totally different, e.g. PDF or SVG, to show that the Rules Format is technology agnostic
... Need to demonstrate that
Wilco: Could add, "at least two different web technologies"
Shadi: Yes. That could work. PDF, SVG, ePub
... Anyone looking at different technologies?
Wilco: Auto WCAG is looking at SVG
Romain: We definitely will publish some rules for ePub
Shadi: How about for the CR?
Romain: When will this run?
Shadi: October? to January
Wilco: Testcases: I didn't explicitly include test cases but rules format requires test cases. Do we want to write this out?
Shadi: Yes. Just wondering why we are not writing this out
Wilco: Expectation and Applicability each require at least 1 test case. Ok, actually 2, pass/fail
Alistair: May we also do applicability too? 3 test cases. Pass, Fail, N/A
Wilco: 2 for inapplicable, 1 pass, 1 fail
Alistair: Why 2? Only need 1 each
... Atomic rule is not actually an atomic rule if it requires at least 2 expectations. When we talk about atomic rule we only talk about 1 expectation.
Wilco: Yes. Every expectation needs 1 pass and 1 fail.
Alistair: If we have more than 1 test case in a rule, it's not atomic
Wilco: Ya, I suppose so. Atomic means it cannot be broken down further.
Alistair: E.g. any video element or audio element...3 elements in a test. This makes 3 types of tests and reporting.
Wilco: If we break it down for expectation, the true false for each expectation does not tell you anything about the accessibility of the object. Does that make sense?
Alistair: No.
Wilco: Feel free to open an issue
Steinerik: Difficult to revisit issues when we are so close to publishing. It's been a long time since we introduced atomic rules
... That's be efficient in our feedback. If we think it's critical, then we should address
Wilco: Let's address in an issue
Shadi: What if it was called something other than atomic?
... If it's just terminology, I don't think it's critical to fix before the CR. If it's critical than we should fix before CR.
... Let's open a Github issue to address if it's critical or just terminology?
Alistair: I just think it's weird that atomic rules are looking at different things in this case but not a showstopper
<scribe> ACTION: Open a Github issue to address the terminology/definition of atomic rule containing more than 1 test case
<trackbot> Error finding 'Open'. You can review and register nicknames at <https://www.w3.org/WAI/GL/task-forces/conformance-testing/track/users>.
Shadi: Last point is that we repeat conformance requirements. We don't need to repeat bullets
Wilco: At least two rules that are automated, at least two rules semi-automated and at least two rules that use manual steps
... Do we need more rules from manual test sources like Trusted Tester?
Shadi: The WAI Tools project should be producing at least 5. We should have manual rules to show
Wilco: Do they map to existing technologies?
Kathy: I want to get a better understanding of what you are asking. Is this for a specific success criteria?
Wilco: At least one rule should look like a WCAG SC. It can be broken down further.
... Not proposing that any of these methodologies have adopted ACT rules but could to show that they could be adopted as ACT rules.
Kathy: I hope so. I think that is possible.
Shadi: Keyboard and Headings rules should exist in Trusted Tester
Wilco: Do you think this vague exit criteria is sufficient?
Shadi: I think so. I think it is okay to keep it open.
... Under expectations, I think the bullets are conformance requirements.
Wilco: Ok, removing them
... Benchmark. Not included. Expected to be removed
... MaryJo, How are the edits coming along?
Wilco: Is it just editorial?
MaryJo: No, we have new topics
Wilco: What happens next?
... Exit criteria, we need to do a survey
... Are any issues something we need to resolve before CR?
There are some questions around definitions
E.g. Test aspect and Test subject
Mary Jo: What's the criteria?
Shadi: We need to address all substantive issues before CR
... For example, Atomic rule. If this is substantivwe issue we need to address
... We should not expect comments that a feature is broken when we exit CR. This should be viewed as Feature Freeze
Mary Jo: I don't think there is anything
Wilco: I think there are some questions around definitions but we can work on these in CR
Shadi: Is this a MUST or SHOULD. Need to clarify this
Wilco: I will work on this before the next leaders meeting
Romain: Each aspect must be discreet with no overlap. I have a concern with this requirement.
... We don't say what we mean by this. Would rather get rid of this requirement or make it informative
Wilco: Was there opposition to this idea?
Shadi: What are the options?
Wilco: Remove the MUST
... Each aspect Should be discreet and avoid overlap.
Romain: SHOULD does not work here since it is not normative
<shadi> "Aspects under Test are discrete with no overlap..."
Romain: What does it mean for an Aspect to be discreet?
Wilco: What if we do not make it capitalized?
Shadi: No you should not use these keywords
... Just leave it out all together
Wilco: Will this address the issue?
Alistair: What if we take this out all together? I don't think we have to say this
Romain: We have to think what we add for the reader with this statement. Practically will do the write thing. We cannot not enforce this. I think we should drop the sentence.
Steinerik: I agree we should remove the statement
<cpandhi> +1
Anne: My comments, Each Aspect must be discreet but we can just remove it
<shadi> +1
<romain> +1
<anne_thyme> +1
<agarrison> +1
<Jey> +1
<trevor> +1
<kathyeng> +1
<maryjom> +1
Wilco: Can we remove it?
+1
RESOLUTION: We will remove the statement
Anne: What about editorial suggestions?
... Lots of contradictions
... Let's switch the order. Which belong to composite rules
... Might be editorial but it also changes the requirements
Also note that I am making edits to this section based on Anne and Shadi's feedback.