Anne, what we talked about last time. outcome reporting changed from test target to test subject. Didn't realize change before the edits last week.
Anne, We report more granular data.
Wilco, don't understand, you can still report test target level
Anne, Finding section also added which maybe to some extent covers some of it, but no term currently for stating that this target passes a certain rule.
Wilco, Meets the expectations?
Anne, Many edits makes it hard to grasp
Anne, we wanted terms or language to use to agree on. All that is removed from the test target level and put into a definition of "finding", which is only used for composite rules, but not mandatory
Wilco, ACT format doesn't have requirements for how to report - implementing
Anne, we have definitions for reporting outcomes. We decided to take aggregation out of spec, and we are back to the concept of aggregation on a page level, I guess, without having all the framework for how to work on test targets level
Anne, I am asking for us to require reporting of the outcome of test targets, which I understand was the case until I read last weeks edits
Anne, there have been assumptions for how things were working or should work that affect interpretations
Trevor, fi you pass all test targets, wouldn't that mean you pass the rule?
Anne, currently you can't pass a test target. A test target can only meet expectations.
Trevor, missed by scribe
Anne, I would like to be able to say first video on this page passed this rule, rather than whole page passed this rule
Shadi, I see what you mean. We discussed this when we discussed reporting, and we agreed to move it to informative.
Shadi, in WCAG conformance is defined on a page level
Shadi, what does it mean if one video fails the rule?
Shadi, it is a design issue. How do we break it down? What is stopping you from doing the detailed reporting in your implementation?
Shadi, In the reporting we tried to find a consistent way of reporting.
Shadi, the video in your example, would you be pointing to the video element or the button or which part of that? That is why we decided this would be a reporting/implementation thing.
Shadi, If we have these test cases and implementations report the same, we don't really care.
Skotkjerra, in our definition we say a test subject can be a collection - how do we handle this? E.g. in the bypass block SC
Shadi, the most granular subject in WCAG is pages.
Shadi, What are we trying to achieve that is not on implementation level
Anne, clearer definitions.
Anne, terms are more loosely defined or not defined.
SHadi, there is an issue of clarity vs. general terms. We have tried to work on this through the document.
Shadi, the other question is the level of reporting
Wilco, I am concerned to. It was unfortunate that we lost this as a consequence of the decision we have made
Anne, Unsure which decision lead to this
Wilco, it is unfortunate that we had to rewrite expectations to meet test subjects rather than test targets. That is the result of a discussion we had at TPAC, and that this was the way to go.
Wilco, The discussion at TPAC in Lyon: Discussed how do you do the accessibility mapping. Initial proposal was to leave this fairly open so there were more options to mapping.
Wilco, Objections to that.
Wilco, we would have to redefine expectations to contain expectations of the atomic rules...
Shadi, I recall the discussions, but recalling a different discussion about reporting and pointing to the different test targets.
Shadi,m which proposals are on the table?
Shadi, several people indicate that they would wish for a change
Anne, I have made an example, and we could say that this rule applies to all images on the page and 85% of them satisfies this certain condition. We should be able to do that within the old wording with test targets
Wilco, so this wouldn't be a composite rule?
<anne_thyme> https://github.com/w3c/wcag-act/pull/322#issuecomment-457185354
Wilco, it doesn't seem like you can compose a rule like this
Wilco, we would have to reevaluate the accessibility mapping because multiple pass and fail outcomes for each rules
Anen, that was what we thought we had before. Might be because we weren't grasping consequences of using test subjects
Trevor, if you have a mixture of passed and fail targets, would that mean that the atomic rules fails?
Anne, if you have anything failing in your test subject, then your test subject is failing
Anne, if you only have passed or inapplicable test targets within your test subject, the test will be passing.
Wilco, So your proposal here, I am guessing, is revert and if you want some sort of complex aggregation you have to rewrite the rule to accomodate for it?
Anne, yes…..maybe……or we could allow for composite rules that could say 80% of test targets in this atomic rule is passed for this rule to pass
Wilco, I don't think I fully understand the consequences of the proposal
Wilco, I am ok with the current solution, although I do not think it is ideal.
Anne, I could live with this solution with improved definitions
Wilco, proposal is to strengthen the definition of findings
Anne, what I liked before I thought that there was a term for meets all expectations that was just passed. That is kind of what I am missing now
Anne, especially when using results in composite rule
Anne, I really like passed and failed, but they have other meanings, so I don't really know
Wilco, not sure what we could change
<MoeKraft> Here's the current definition of Outcome:
<MoeKraft> Outcome One of three types of conclusions that come from evaluating an ACT Rule on a test subject. An outcome can be: Inapplicable: No part of the test subject matches the applicability Passed: The test subject meets all expectations Failed: The test subject does not meet all expectations
Wilco, meets all expectations, is that enough?
Anne, a lot of word instead of one term. Gets vague and conversational.
Anne, if we put in meets all expectations, that should b enough, but could use a better word
Wilco, "meets all expectations from the following rules"
Wilco, how about I create a definition "meets all expectations"?
Anne, let's go for it
<MoeKraft> So are we throwing out the definition of passed?
Wilco, the glossary section replaces the requirement for a definition of outcomes
Wilco, can we do a CFC?
Wilco, I'll send out a new version later, and everybody wil have until Monday
Wilco, if no comments before Monday, we go to CFC
Conclussion: CFC on Monday if no comments