16:42:39 <RRSAgent> RRSAgent has joined #silver-conf
16:42:39 <RRSAgent> logging to https://www.w3.org/2021/01/28-silver-conf-irc
16:42:48 <sajkaj> agenda?
16:42:53 <sajkaj> Agenda+             Agenda Review & Administrative Items
16:42:53 <sajkaj> agenda+               Use Cases Review & Continued Discussion
16:42:53 <sajkaj> agenda+             Other Business
16:42:53 <sajkaj> agenda+             Be Done
16:43:05 <sajkaj>  agenda?
16:43:41 <sajkaj> regrets: John_Northup
16:43:47 <sajkaj> Chair: sajkaj
16:44:01 <sajkaj> Meeting: Silver Conformance Options Subgroup
16:44:11 <sajkaj> Date: 28 January 2021
16:50:17 <sajkaj> present+
17:00:33 <Wilco> Wilco has joined #silver-conf
17:01:00 <PeterKorn> PeterKorn has joined #silver-conf
17:01:03 <PeterKorn> present+
17:02:05 <bruce_bailey> bruce_bailey has joined #silver-conf
17:05:15 <bruce_bailey> present+
17:05:26 <Wilco> present+
17:06:44 <sarahhorton> sarahhorton has joined #silver-conf
17:06:54 <sarahhorton> present+
17:07:21 <Jemma> Jemma has joined #silver-conf
17:09:08 <sarahhorton> zakim, take up next
17:09:08 <Zakim> agendum 1 -- Agenda Review & Administrative Items -- taken up [from sajkaj]
17:09:29 <sarahhorton> agenda?
17:09:54 <sarahhorton> zakim, take up next
17:09:54 <Zakim> agendum 1 was just opened, sarahhorton
17:10:02 <sarahhorton> zakim, close this item
17:10:02 <Zakim> agendum 1 closed
17:10:03 <Zakim> I see 3 items remaining on the agenda; the next one is
17:10:03 <Zakim> 2. Use Cases Review & Continued Discussion [from sajkaj]
17:10:07 <sarahhorton> zakim, take up next
17:10:07 <Zakim> agendum 2 -- Use Cases Review & Continued Discussion -- taken up [from sajkaj]
17:10:43 <sarahhorton> scribe: sarahhorton
17:11:58 <jeanne> https://docs.google.com/document/d/1GyUYTnZp0HIMdsKqCiISCSCvL0su692dnW34P81kbbw/
17:12:58 <sarahhorton> Peter: Reads through use case
17:13:41 <sajkaj> ~~~~q?
17:14:03 <sajkaj> ~~~~q?
17:14:27 <sarahhorton> Peter: Want to engage Detlev on the case study
17:15:30 <sarahhorton> Janina: Started email with Detlev, he'll try to expand on it
17:16:08 <sarahhorton> …started draft
17:16:33 <sarahhorton> ...can't you tell if there's a programmatically associated label, isn't that testable
17:16:46 <sarahhorton> ...can automatically test it but don't know if labels are correct
17:17:07 <sarahhorton> ...provide guidance about what's in process or what's out of scope
17:17:25 <sarahhorton> ...ratings are important to buying decisions, sharing information about products
17:17:39 <sarahhorton> ...going to have to provide guidance on how to reach that decision
17:17:59 <sajkaj> q?
17:18:00 <sarahhorton> ...obvious ones that are out, ones that are in, and gray area
17:18:31 <sarahhorton> Peter: Make comments to Detlev in doc to capture thinking, can't resolve without him here
17:19:27 <sarahhorton> ...adds questions to Google doc
17:20:57 <Wilco> +10
17:21:13 <sajkaj> q?
17:21:14 <sarahhorton> Jeanne: What can be detected by test is moving target, haven't thought of way to codify that's flexible
17:22:36 <sarahhorton> Wilco: ACT Taskforce decided to move away from automated and manual, depends on context, how many assumptions, anything can be automated if you know the environment really well
17:23:38 <sarahhorton> Peter: Wondering about after time has passed with introduction of code, testing for things, changes what's required to hit levels, don't know how to work into structure
17:24:24 <sarahhorton> ...grace period reasonable amount of time to adopt techniques to when we expect they will be used
17:24:43 <sarahhorton> Jeanne: Should only be thinking bronze level, bronze is regulatory
17:25:39 <sarahhorton> ...new category of critical error, e.g., error identified by automated test
17:26:44 <sarahhorton> ...error discovered by automated test, but then issue of false positives, complex sites where portions are not accessed by user, but trigger in automated audit, should add to use case list
17:27:12 <sarahhorton> ...automated test, user accessible area, impact of accessibility, could be easily fixed
17:28:06 <sarahhorton> Peter: Vendors developing tests, vendor A finds it, vendor B doesn't, need some notion of managing new technique, allowing some period of time
17:28:48 <sarahhorton> Jeanne: Could include here with caveats, make note in document to add use case
17:29:04 <sajkaj> q?
17:32:08 <sarahhorton> Janina: Working on response to issues in B, needs to think through different types of interactions
17:33:16 <sarahhorton> Peter: Reads through #5
17:33:31 <sarahhorton> ...break down use case into issues
17:33:57 <sarahhorton> ...Read through #6
17:36:10 <Wilco> q+
17:36:17 <sajkaj> q?
17:36:52 <sarahhorton> Bruce: Wants to keep conformance compatible with gov space, okay with use case
17:37:20 <sarahhorton> Wilco: Not telling use case, what is it trying to get at
17:37:51 <sajkaj> q?
17:38:13 <sarahhorton> Peter: Separate from critical error, must be fixed, separate from primary and secondary areas, will always encounter bugs that impact users, sometime severe
17:38:49 <sarahhorton> ...should accessibility be treated differently than issues that impact users generally
17:39:37 <sarahhorton> ...e.g., train station, 2 entries, ticketing machine out affects everyone, elevator affects wheelchair users
17:40:28 <sarahhorton> ...nothing of sufficient size/cmplexity will be free of bugs, if standard says need to be free of severe bugs, will never have site that passes bronze
17:40:59 <sarahhorton> Jeanne: Giving range of what's acceptable, alt text, full rating if you meet X%
17:41:47 <sarahhorton> Peter: One of two biggest concerns, need to figure it out
17:42:08 <sarahhorton> Jeanne: Alt text, in critical flow and block user from accomplishing task
17:42:41 <sarahhorton> ...if 95% passes and there are no critical errors it will pass
17:43:19 <sarahhorton> Peter: Temporal concern, any moment in time may be bug, hours or days, will find error, fail site
17:43:48 <sajkaj> q?
17:43:58 <sarahhorton> ...large sites some page will have critical error, esp if discovering error needs manual testing, no different from errors that affect everyone
17:44:04 <Wilco> ack w
17:44:16 <sajkaj> ack w
17:44:22 <Jemma> present+
17:44:38 <sarahhorton> Janina: Luck of draw because sampling expectations
17:45:09 <sarahhorton> Peter: Accurate sampling, might find it — most concerning when bug that can't be found with automated tests, still critical error
17:45:57 <sarahhorton> Wilco: QualityMark, did audits, even ones that had mark year before, always found something new
17:46:15 <sarahhorton> ...agree that it needs to be taken into account
17:46:26 <Jemma> rrsagent, make minutes
17:46:26 <RRSAgent> I have made the request to generate https://www.w3.org/2021/01/28-silver-conf-minutes.html Jemma
17:46:41 <sarahhorton> Peter: No idea how to take it into account
17:47:11 <Jemma> rrsagent, make logs public
17:47:19 <sajkaj> q?
17:47:20 <sarahhorton> Wilco: Critical errors works well page by page, very large — even moderate — will have errors, process question
17:48:00 <sarahhorton> Jeanne: QA has normal testing procedures for everything, does the button work, does it work on mobile, 1000s of tests that QA runs
17:48:22 <sarahhorton> ...small projects don't go out door until no critical bugs
17:48:39 <Wilco> q+
17:48:58 <sarahhorton> ...when testing on way out door, will do what org does for testing, accessibility bugs can't be worse than other bugs of other types
17:49:51 <sarahhorton> ...lots of testing in QA, automated, manual — accessibility bugs are proportional
17:50:10 <sarahhorton> ...maybe belongs in maturity model, nothing going out with critical bugs
17:50:32 <sarahhorton> Peter: In agreement, not coming to solution about codifying in Silver
17:51:00 <sarahhorton> ...blocker bugs discovered late, go to publish, fix quickly
17:51:28 <sarahhorton> ...blocker bugs launch, add fix to next launch
17:51:33 <Wilco> q-
17:51:54 <sarahhorton> ...massive site/software, going to have some number of critical bugs always
17:52:12 <sajkaj> q?
17:52:40 <sarahhorton> ...if not treating accessibility differently, critical things are critical, not treating disproportionately, should that be okay? How do we capture
17:53:49 <sarahhorton> ...going to be period of time between when flashing is noticed and when it's fixed, could be 2 hours, will be flashing
17:54:14 <sarahhorton> ...if test finds flash, is it noncompliant, penalties?
17:54:53 <sarahhorton> Janina: User agent could check for flashing, doesn't work in real time, would catch this critical error
17:55:45 <sajkaj> q?
17:55:46 <sarahhorton> Jeanne: Don't want to solve individual examples, we have categories, could add category for automated testing
17:56:18 <Jemma> I agree with Jeanne.
17:57:12 <sarahhorton> ...even with temporal problem, critical error that needs to be fixed, site overall has good accessibility, still only need 3.5 out of 4 to get bronze level, even with critical errors
17:57:46 <sarahhorton> Peter: Few critical errors could bring down below 4 and could get bronze
17:58:28 <sarahhorton> Jeanne: Guideline, write in different breakpoints of accessible and acceptable, overall score could address temporal, averaged and normalized over all guidelines
17:58:36 <sarahhorton> Peter: That works
17:58:56 <sarahhorton> ...will be lots of arguing about numbers, but it works
17:59:04 <sarahhorton> ...will clean up example
17:59:18 <sarahhorton> ...will report on extent to which use case is already addressed
17:59:51 <sarahhorton> Jeanne: Write up article explaining to people, good to have to explain how things are working
18:00:28 <sarahhorton> Peter: Yes but not until March when use cases are addressed
18:01:15 <sarahhorton> Jemma: Is the information about scoring somewhere?
18:01:47 <sarahhorton> Jeanne: It's in the FPWD, been looking at comments, questions, clearly didn't explain this well
18:01:58 <sarahhorton> ...figured out last minute
18:03:01 <sarahhorton> ...trying to address this important and valuable questions
18:03:42 <sarahhorton> ...need to make it more understandable
18:11:57 <sajkaj> zakim, bye
18:11:57 <Zakim> leaving.  As of this point the attendees have been sajkaj, PeterKorn, bruce_bailey, Wilco, sarahhorton, Jemma
18:11:57 <Zakim> Zakim has left #silver-conf
18:12:05 <sajkaj> rrsagent, make log public
18:12:22 <sajkaj> rrsagent, make minutes
18:12:22 <RRSAgent> I have made the request to generate https://www.w3.org/2021/01/28-silver-conf-minutes.html sajkaj
19:24:12 <jeanne> jeanne has joined #silver-conf