18:16:35 RRSAgent has joined #aria-at 18:16:35 logging to https://www.w3.org/2021/04/22-aria-at-irc 18:17:37 MEETING: ARIA and Assistive Technologies Community Group 18:17:40 CHAIR: Matt King 18:17:46 rrsagent, make log public 18:17:51 present+ 18:17:55 alflennik has joined #aria-at 18:18:25 Marie_ has joined #aria-at 18:23:22 TOPIC: App Development - Merge conflicts in test format? 18:25:22 scribe: s3ththompson 18:25:34 s3ththompson: Did we discuss using a dev branch to present merge conflicts? 18:25:46 James: - Not yet 18:25:56 James: Open to alternatives, but part of the reasoning was that status bit was not a test writing concern, it was a community concern. 18:26:07 s3ththompson: Will we reach a state where there are so many tests that we don’t want to change the status bits individual? 18:26:31 James: For a longer non-APG test plan, or a plan where we’re waiting for a more compositional test approach, there might be a test plan with lots and lots of tests 18:27:09 s3ththompson: what about merge-driver that understands our csv format? what about moving to a directory of files? (instead of an array in a csv format) 18:27:34 James: both could work... we definitely need some improvement 18:28:09 James: any short term mitigation would be good. current output from generation script is not very accessible... lots of scrolling to get back to the actual error 18:29:58 James: one big win would be to only generate files for test plans that have changed 18:30:13 s3ththompson: do we need even to have generated files checked in next to source files? 18:30:36 James: not necessarily, could be a feature of the app... whatever it is, we just need a simple way to preview 18:33:45 s3ththompson: given that we're working on both app & test repo infrastructure... we are open to possibilites that shift work from test repo *to* app 18:35:05 James: i agree with the idea to identify the "right" solution (the "real" fix, so to speak), but i think we also want to make sure that we identify short term mitigations to work towards in the process 18:35:44 James: merge conflict, test workflow topic is higher priority than test format for us 18:37:31 s3ththompson: I think broadly speaking we could definitely look at infrastructure stuff before the test format 18:37:46 s3ththompson: we mainly want to make sure that we are aware of upcoming test format changes when we do our database refactor 18:38:18 TOPIC: App Development - Sprint 4 18:38:23 scribe: alflennik 18:39:54 Seth: we've been working on changes to the app data model to support either working mode changes or generally to support greater flexibility 18:41:36 Seth: Right now the biggest constraint on the app is, first, the git sha is an underlying parameter that affects almost all of the views in the app 18:42:47 Seth: RE versioning test plans independently, we are thinking about CRUD operations and views to no longer assume a single version for all test plans 18:43:38 Seth: Another thing that has gradually become a limitation is the fact we currently save runs to a test table, and grouping them was a presentational question only, not specified in a model 18:45:40 How admins determine which tests are active needs to occur as a derived state based on the active test plans and the active AT/Browser configurations and test versions. The entries on the test queue, then, are entirely derived. This, initially, saved us from having to generate a huge matrix of combinations at the outset. However, this is tricky at this point, since it blocks us from changing the global version used by the app' 18:47:10 Seth: to summarize, we are moving towards a model where the test queue exposes normal CRUD operations, allowing for direct control over what the test queue displays. 18:49:24 James: to clarify, let's imagine a test like Combobox is finalized and active. Let's say it's version 1. Then we decide in 18 months to add support for touch-based screen readers. This is going to have implications for the tests, so we create version 2. Can this be a draft, tested with a different group of testers. They're not testing primarily for results, but as a review process, a QA process. Occurring within the app. Could it be 18:49:24 possible to have combobox v1 and combobox v2 handled, assigned or shared separately. 18:49:44 Seth: yes. That is a use case we want to support but don't currently, which is why we're making this change. 18:49:49 James: What would happen now? 18:51:32 Seth: since the only parameter you can change is the global version number. But when you change the version number the old runs will be, not deleted, but inaccessible. Currently. The context for that limitation being that we expected the test results to be published separately - this was the test cycle concept - but the issue there is that it assumes a model where we test, review and publish to completion and then go back to a blank 18:52:01 Seth: ...slate, and it's clear that this requirement to wait is something we'd like to revisit 18:52:42 James: The 60-day review process will require us from distinguishing changes made to tests during that window 18:52:48 Seth: Yes, exactly 18:53:34 Seth: in the past you had to immediately replace version 1 with 2, but now you can keep both around 18:54:15 Seth: it won't automatically group the results across 1 and 2 18:54:24 James: what about adding a period 18:55:00 Seth: so far we haven't proposed a shortcut for small changes, but it's something we could consider 18:55:15 James: a changelog or semantic versioning might work 18:55:20 Seth: right, that's a good point 18:55:33 Seth: What do you think of the term test cycle, James? 18:56:23 James: I think it's ended up less useful. I recall Matt being quite clear this is no longer an important consideration. We will have to ask him about that/ 18:57:35 James: we need to be fluid in how we assign things 18:57:43 Matt_King has joined #aria-at 18:59:38 Seth: the upcoming sprint is mostly about infrastructure and testing. We will spin up a new dev environment hosted by Bocoup servers, in addition to the current staging and production environments hosted by the W3C. The purpose is to API tests our routes based on our desired behavior, to set the stage for the implementation work 19:01:01 Matt: exciting to have GraphQL! 19:02:01 jongund has joined #aria-at 19:02:08 TOPIC: Test writing meeting agenda starts here 19:02:12 present+ jongund 19:02:28 scribe: s3ththompson 19:02:48 present+ 19:02:55 jesdaigle has joined #aria-at 19:03:07 Alyssa_G has joined #aria-at 19:03:42 present+ 19:07:14 group discusses the impact of sleep on memory and working... 19:07:16 joke/ should we update working mode to require minimum sleep? 19:09:13 TOPIC: Wording of assertions that use terminology for screen reader functions 19:10:54 Matt_King: for JAWS and VO command differences... I thought of these commands as reading the element itself... but including the words "information about", (esp. if we later add assertions for reading descriptions) feels like it's unnecessarily verbose 19:11:34 Matt_King: "read Combobox" vs. "read info about Combobox" 19:12:01 James: some elements sounded clunky e.g. "read unchecked radio button" 19:12:13 James: you're not really reading the unchecked radio button, but everything about that button 19:12:35 Matt_King: I think "name", "role", and "state" is basically implied by the word "read" 19:13:14 James: but if the name is "read a grid cell" it's really ambiguous since it's not reading the content... its reading info like coordinates 19:16:02 Sina: screen readers make some of these distinctions ambiguous, but we should be more pendantic about the different intentions to properly disambiguate between the different ways of reading an element and its metadata 19:17:25 Sina: analogy, for a letter... what do you say when you want to read stuff about how the letter was to, how it was delivered, and what the package is? 19:19:12 Matt_King: tricky part is that grid cell is both letter and envelope, while radio button is just letter 19:23:05 Sina: i think the executive summary is separate from the raw data... are some of your concerns Matt that the executive summary needs to be different than the raw data? 19:25:08 Could you use the word "present" or "presents" instead of "read" or "reads?" 19:27:43 Matt_King: I want to be clear that the terminology is framed in terms of the user's intent... 19:28:12 Sina: why don't we just have a table of definitions that we write alongside the test instructions and defer these conversations to a discussion about what the definitions should say 19:28:33 Matt_King: yeah, that's what I'd like to do... and i'd like the verb teminology to be concise 19:28:49 Matt_King: the other one i want to bring up is "read sequentially" which feels unnecessarily verbose 19:32:32 James: when someone is expected to press "down arrow" to get to result... should they be expected to know how many times they should press the down arrow? or should we encode the tests so that they press the arrow key *until* they find the right element and then we would record how many times down they had to press (and we could track analytics about that data point) 19:44:09 Sina: in my mind if tabbing to something vs arrowing to it creates a different result, we should really be fixing that, rather than testing it and catering to it 19:44:24 Matt_King: i care about being able to test with granularity 19:46:32 Matt_King: coming back to issue https://github.com/w3c/aria-at/issues/419, we have at least "read," "navigate," and "operate" 19:46:45 James: to be clear, we do have different tests for arrowing to an element vs tabbing to it… 19:48:13 Matt_King: so what's your take on how we distinguish multiple ways of "navigation"? just grouping them both under the same verb? we don't need different verbs, since the commands have the distinction. I'll reread and think about this again... 19:49:44 Sina: I think we might get pushback, but i think it's a conversation we should have with vendors... make them justify why different ways of navigating to an element should have extra or less info 19:57:34 s3ththompson has joined #aria-at 20:02:15 Marie_ has left #aria-at 20:05:43 scribe: mck 20:06:37 Resolution: will consider changing "read information about" to "read" after we have a glossary of test terms. Then, we would script the change if we make it. 20:06:47 rrsagent, make minutes 20:06:47 I have made the request to generate https://www.w3.org/2021/04/22-aria-at-minutes.html Matt_King