15:43:44 <RRSAgent> RRSAgent has joined #silver-conf
15:43:44 <RRSAgent> logging to https://www.w3.org/2021/03/25-silver-conf-irc
15:44:00 <sajkaj> Meeting: Silver Conformance Options Subgroup
15:44:05 <sajkaj> Chair: Sajkaj
15:44:12 <sajkaj> Date: 25 Mar 2021
15:44:15 <sajkaj> agenda?
15:44:44 <sajkaj> Agenda+ Agenda Review & Administrative Items
15:44:44 <sajkaj> agenda+ Continue work on March Deliverable -- Report on Use Cases https://www.w3.org/WAI/GL/task-forces/silver/wiki/March_Report_to_the_Silver_TF
15:44:47 <sajkaj> agenda+ Assigned github issues https://github.com/w3c/silver/labels/Subgroup%3A%20Conformance%20Options
15:44:50 <sajkaj> agenda+ Use Cases Discussion (Continued)
15:44:52 <sajkaj> agenda+ Other Business
15:44:55 <sajkaj> agenda+ Be Done
15:45:09 <sajkaj> rrsagent, make log public
15:45:16 <sajkaj> rrsagent, make minutes
15:45:16 <RRSAgent> I have made the request to generate https://www.w3.org/2021/03/25-silver-conf-minutes.html sajkaj
15:56:41 <sajkaj> regrets: John_Northup
15:58:45 <Wilco> Wilco has joined #silver-conf
16:00:11 <sajkaj> Thanks, Wilco. We'll wait for you before closing on the draft report to the TF!
16:01:17 <JF> JF has joined #silver-conf
16:01:24 <JF> Present+
16:01:31 <JF> agenda?
16:02:52 <bruce_bailey> bruce_bailey has joined #silver-conf
16:03:07 <bruce_bailey> present+
16:03:31 <sajkaj> regrets+ Peter_Korn
16:03:38 <sarahhorton> sarahhorton has joined #silver-conf
16:03:43 <sarahhorton> present+
16:03:51 <KimD> KimD has joined #silver-conf
16:03:56 <KimD> Present+
16:04:32 <KimD> scribe: KimbD
16:04:40 <KimD> Scribe: KimD
16:05:32 <sajkaj> akim, next item
16:05:42 <sajkaj> scribe: KimD:
16:06:03 <ToddLibby> ToddLibby has joined #silver-conf
16:06:10 <ToddLibby> present+
16:06:47 <sarahhorton> zakim, next item
16:06:47 <Zakim> agendum 1 -- Agenda Review & Administrative Items -- taken up [from sajkaj]
16:07:07 <KimD> Janina: announcement: to EU participants: reminder about summertime/daylight time
16:07:30 <Wilco> present+
16:07:37 <KimD> Janina: we should be in synch now again.
16:07:47 <sajkaj> zakim, next item
16:07:47 <Zakim> agendum 2 -- Continue work on March Deliverable -- Report on Use Cases https://www.w3.org/WAI/GL/task-forces/silver/wiki/March_Report_to_the_Silver_TF -- taken up [from sajkaj]
16:08:33 <sajkaj> https://www.w3.org/WAI/GL/task-forces/silver/wiki/March_Report_to_the_Silver_TF#Report_of_the_Conformance_Options_Subgroup
16:09:01 <KimD> Janina: Report to Silver group (link above)
16:09:55 <KimD> JF: Reads report
16:12:18 <sajkaj> q?
16:12:21 <KimD> Janina: Comments?
16:12:48 <sarahhorton> q+
16:13:52 <KimD> JF: Use case B: "almost a layout table" is making an assumption
16:14:07 <KimD> JF: we should account for both a real table and a layout table
16:14:46 <KimD> Janina: We thought of both ways - point is it's simple enough to avoid being an impediment.
16:15:10 <bruce_bailey> q+ to ask to strike almost layout table
16:15:12 <KimD> Janina: Compare to complex, large table
16:15:14 <sajkaj> q?
16:15:21 <sajkaj> q?
16:15:26 <JF> Q+
16:15:45 <sajkaj> ack s
16:16:21 <KimD> Sarah: Can we talk through use case B and how WCAG 3 would address.
16:16:59 <sajkaj> q?
16:17:12 <KimD> Sarah: If 2 col table and no header labels, what are my outcomes?
16:17:27 <sajkaj> ack b
16:17:27 <Zakim> bruce_bailey, you wanted to ask to strike almost layout table
16:17:40 <JF> ack me
16:18:22 <KimD> Bruce: We may not be able to work this use case through, even though it's a failure, it's unlikely to be a blocker
16:18:27 <JF> Q+
16:18:43 <KimD> Bruce: using 'layout table' isn't helpful
16:18:57 <KimD> Sarah: trying to be able to test the framework
16:19:26 <KimD> Bruce: example: table is name, phone number; name, phone number
16:19:39 <JF> <th>Janina</th><td>123-456-7890</td>
16:19:41 <sajkaj> q?
16:19:46 <sajkaj> ack j
16:19:49 <KimD> Sarah: Would be we able to say this isn't a critical error?
16:20:26 <KimD> JF: Simple table w/ Bruce's table: still needs HTML associations. JF would fail it.
16:20:26 <sajkaj> q?
16:20:49 <KimD> JF: it could be a layout table, should still account for both tables
16:20:50 <sajkaj> q?
16:21:40 <KimD> Jeanne: Sarah's summary is helpful. Given current Silver structure, this simple table could pass because not a barrier.
16:21:43 <JF> Q+ to ask Jeanne about my example
16:21:55 <KimD> Jeanne: This would be addressed by silver structure
16:22:22 <KimD> Sarah: That's helpful. How will evaluator come to that conclusion?
16:22:36 <KimD> Jeanne: it would be in how the test was written
16:22:53 <KimD> Jeanne: the test would have to take that into account.
16:23:05 <KimD> JF: who decides if it's a critical error?
16:23:30 <KimD> Jeanne: test author, and assuming all the process around approval
16:24:07 <KimD> Janina: we'll need a metric that isn't so hard and fast as 2x2 grid, because it could still fail. For example, more than one line of content.
16:24:26 <KimD> Janina: like a list inside of a cell
16:24:27 <sajkaj> q?
16:24:30 <sajkaj> ack jf
16:24:30 <Zakim> JF, you wanted to ask Jeanne about my example
16:25:14 <sajkaj> q?
16:25:24 <KimD> JF: It's a fine line in what would pass and what wouldn't.
16:25:36 <KimD> Janina: how to resolve without asking for perfection
16:25:45 <sajkaj> q?
16:26:02 <KimD> Jeanne: we agree, doing it correctly gets more points. It can still be workable and thus NOT fail
16:26:32 <KimD> Jeanne: also, we haven't written the guideline yet. Point of use case is to see if this is something that could pass.
16:26:43 <KimD> Jeanne: with low points, but still could pass.
16:26:43 <bruce_bailey> @JF the point with my 2x name/phone number isn't that it's easy to fix, it's that a 3.0 review could allow a pass as-is
16:27:09 <sarahhorton_> sarahhorton_ has joined #silver-conf
16:27:13 <KimD> Jeanne: back ground of use case would be helpful in the doc
16:27:13 <bruce_bailey> i am hearing Jeanne now saying something very similar !
16:27:19 <sarahhorton_> q+
16:27:59 <KimD> Janina: add a sentence that shows how we might let something pass without it being perfect
16:28:01 <sajkaj> q?
16:28:05 <JF> @bruce - sure, but I'm more interested in squeezing out, or limiting subjectivity: pretty much anything can be answered with "it depends", but that's hard to standardize
16:28:05 <sajkaj> ack s
16:28:30 <KimD> Sarah: Todd and I talked about use case, wondering if it belongs here.
16:29:02 <KimD> Sarah: login form, when it doesn't work and comes back without an error.
16:29:40 <JF> Bingo!
16:29:45 <KimD> Sarah: WCAG 3 approach is missing related to severity and exceptions
16:30:02 <JF> Q+
16:30:10 <KimD> Sarah: i.e., a test that fails, but will it cause a block a person?
16:30:29 <Wilco> +!
16:30:32 <Wilco> +1
16:30:36 <KimD> Sarah: whether an error is critical or not isn't addressed by current framework.
16:30:38 <sajkaj> q?
16:30:57 <sajkaj> ack jf
16:31:40 <KimD> JF: agree with Sarah. Severity and criticality is unique for each person.
16:31:45 <sajkaj> q?
16:31:50 <KimD> JF: we need to find "break points"
16:32:18 <KimD> Janina: reminder, the use cases are meant to an illustration to show the grey areas
16:33:10 <KimD> Janina: "why did my login fail" question is one we've discussed. We can put it in this doc, it's in our Google doc.
16:33:12 <sajkaj> q?
16:33:28 <KimD> Todd: Not sure off the top of my head
16:33:50 <KimD> Janina: It's in the Google doc, and tried to put everything in a bucket.
16:34:20 <KimD> Janina: Not everything is covered, we're looking at edge cases.
16:34:37 <KimD> Janina: We can add the login use case.
16:35:00 <sajkaj> q?
16:35:12 <JF> +1 to Sarah
16:35:16 <KimD> Sarah: Seems all the current 3 use cases share the same issue: how do we clearly deal with failed tests?
16:35:49 <KimD> Sarah: Silver framework doesn't address
16:36:01 <KimD> Janina: do we want to keep this report?
16:36:12 <sajkaj> q?
16:36:35 <KimD> JF: Headings use case: it's both headings and properly nested headings
16:36:53 <KimD> Janina: issue is that the Headings are not there at all.
16:37:28 <KimD> JF: We need to address the Heading as an relationship/structure issue.
16:38:01 <KimD> JF: Details are needed because it goes to severity
16:38:08 <sajkaj> q?
16:38:48 <sajkaj> q?
16:38:55 <KimD> JF: Compares to zoom, where we have 200% and 400% - very specific
16:39:33 <KimD> Janina: We could write that use case with an arbitrary number.
16:40:16 <KimD> JF: Talks about data tables with multiple lines. Maybe word count is only one factor.
16:40:37 <sajkaj> q?
16:41:03 <KimD> Jeanne: Headings use case: we have Headings written. Can we work with ACT and see how they would address.
16:41:16 <sajkaj> q?
16:41:18 <KimD> Wilco: Agree, interested in what others have to say
16:41:22 <sarahhorton_> q+
16:42:40 <sajkaj> q?
16:42:44 <sajkaj> ack s
16:42:51 <KimD> Sarah: Good idea, let's talk through that. In Methods, there are a couple tests.
16:43:30 <KimD> Sarah: passes one, fails one of the tests. How do I know the severity of the failure? How would I know whether it's a show-stopper?
16:43:54 <KimD> Sarah: That's the gap I'm seeing. How do you decide whether it's a blocker?
16:44:09 <KimD> Janina: Is issue where to score on the scale?
16:44:37 <KimD> Sarah: Tests don't indicate anything about score; they're just T/F.
16:45:01 <KimD> Sarah: Scoring has info, methods have info, but there's no bridge.
16:45:10 <KimD> Jeanne: Example, please
16:45:43 <sajkaj> q?
16:45:53 <KimD> Sarah: Say I've done the tests, one fails, I record that. Next step is determining criticality.
16:46:41 <KimD> Sarah: Missing heading, but then have to determine if missing heading is needed to complete a process.
16:47:08 <KimD> Sarah: Does that identify if that failure translates to a significant impediment.
16:47:22 <KimD> Sarah: Needs to be more fleshed out because there is a gap.
16:47:29 <JF> A HUGE +1 to Sarah's points
16:47:35 <sajkaj> q?
16:47:51 <JF> Q+
16:48:04 <Wilco> q+
16:48:16 <KimD> Jeanne: Agree, so decision of "critical failure" needs more detail
16:48:58 <KimD> Janine: You don't get the info and the structure isn't there via headings, but it can be figured out.
16:49:23 <KimD> Sarah: Also need to look at frequency. That goes to severity.
16:49:41 <sajkaj> q?
16:49:47 <KimD> Janina: "the spoons problem"
16:49:53 <sajkaj> ack w
16:50:21 <KimD> Wilco: Are the test cases getting in the way? We know the problems, maybe the test cases need to be clearer?
16:51:03 <KimD> Wilco: Headings use case seems to be about severity/criticality.
16:51:05 <JF> Functional Performance Statements
16:51:42 <KimD> Wilco: should we express that rather than the use/test case. Maybe we report missing pieces, drop test cases.
16:51:47 <sajkaj> ack j
16:51:52 <sajkaj> q?
16:52:20 <KimD> JF: Criticality/Severity? FPC might address; deals with impact on individual users.
16:53:08 <KimD> JF: Blend of pass/fail and severity. Depends on user though.
16:53:22 <bruce_bailey> q+ to suggest that conversation about critical errors is not really helpful to this use case challeng
16:53:59 <KimD> Jeanne: That example isn't relevant (blinking/flashing not an issue if user has no vision). I think we've addressed that.
16:54:14 <sajkaj> ack b
16:54:14 <Zakim> bruce_bailey, you wanted to suggest that conversation about critical errors is not really helpful to this use case challeng
16:54:16 <KimD> Jeanne: Can you tell me if we haven't addressed?
16:54:48 <KimD> Bruce: Use cases are helpful and important; but "criticality" isn't relevant to this conversation.
16:55:05 <KimD> Bruce: We're talking about edge cases.
16:55:07 <Rachael> q+
16:55:10 <JF> I think the point is Bruce that for any individual, an error may or may-not be "critical"
16:55:35 <KimD> Bruce: They way they're written, it's clear they are not critical errors.
16:55:47 <sajkaj> q?
16:55:53 <sajkaj> ack r
16:56:26 <KimD> Rachael: "Critical error" - capture it as a question to be answered later.
16:56:55 <KimD> Janina: Do you recommend presenting and say that's why criticality is important?
16:57:20 <KimD> Rachael: Yes, we have ambiguity and need to work it out. Can be tied to examples or not.
16:57:40 <KimD> Jeanne: Examples help clarify and make it more understandable.
16:57:53 <KimD> Janina: Any objections?
16:58:06 <KimD> Wilco: Have issues and have examples go with them
16:58:32 <KimD> Janina: We know we need more work on authoring.
16:58:49 <KimD> Jeanne: Agree, point is to flesh out areas that need more work.
16:59:13 <Rachael> present+
16:59:15 <KimD> Janina: We can flesh out more.
16:59:56 <KimD> Janina: We need to report in March. Anyone uncomfortable with our approach? Should we report out?
16:59:58 <bruce_bailey> +1 to reporting
17:00:13 <KimD> Wilco, Sarah: ok with that
17:00:29 <KimD> Janina: Still need to work on GitHub issues.
17:00:52 <ToddLibby> ToddLibby has left #silver-conf
17:01:08 <KimD> KimD has left #silver-conf
17:02:05 <sajkaj> zakim, bye
17:02:05 <Zakim> leaving.  As of this point the attendees have been JF, bruce_bailey, sarahhorton, KimD, ToddLibby, Wilco, !, Rachael
17:02:05 <Zakim> Zakim has left #silver-conf
17:02:11 <sajkaj> rrsagent, make minutes
17:02:11 <RRSAgent> I have made the request to generate https://www.w3.org/2021/03/25-silver-conf-minutes.html sajkaj
19:08:35 <Jemma> Jemma has joined #silver-conf