17:04:41 <RRSAgent> RRSAgent has joined #aria-at
17:04:45 <RRSAgent> logging to https://www.w3.org/2025/06/18-aria-at-irc
17:04:45 <Zakim> RRSAgent, make logs Public
17:04:46 <Zakim> please title this meeting ("meeting: ..."), jugglinmike
17:05:01 <jugglinmike> meeting: ARIA and Assistive Technologies Community Group Weekly Teleconference
17:07:31 <jugglinmike> present+ jugglinmike
17:07:33 <ChrisCuellar> present+
17:07:33 <jugglinmike> scribe+ jugglinmike
17:07:37 <jugglinmike> present+ Matt_King
17:07:37 <howard-e> present+
17:07:43 <jugglinmike> present+ Isa
17:07:45 <jugglinmike> present+ james
17:07:54 <jugglinmike> present+ elizabeth
17:07:57 <jugglinmike> present+ louis
17:08:01 <jugglinmike> present+ dean
17:08:05 <jugglinmike> Topic: Review agenda and next meeting dates
17:08:11 <jugglinmike> https://github.com/w3c/aria-at/wiki/June-18%2C-2025-Agenda
17:08:17 <jugglinmike> Matt_King: Requests for changes to agenda?
17:08:25 <jugglinmike> Matt_King: Hearing none, we'll keep the agenda as planned
17:08:31 <jugglinmike> Matt_King: No meeting Thursday June 26
17:08:35 <jugglinmike> Matt_King: Next meeting: Wednesday July 2
17:08:40 <jugglinmike> Topic: Next AT Driver Subgroup meeting: Monday July 14
17:08:57 <jugglinmike> Matt_King: Next AT Driver Subgroup meeting: Monday July 14
17:09:02 <jugglinmike> Topic: Current status
17:09:04 <jugglinmike> Matt
17:09:15 <jugglinmike> Matt_King: No major changes to the status this week, though we did have some important updates
17:09:29 <jugglinmike> Matt_King: Still 15 in "candidate review" and 4 almost rady
17:09:56 <jugglinmike> Matt_King: Isa and louis have made good progress
17:10:05 <jugglinmike> Matt_King: The JAWS report for "roving tab index" is now up-to-date
17:10:24 <jugglinmike> Isa: there is still one conflict with slider
17:10:30 <jugglinmike> Matt_King: Yup; we'll discuss that later
17:10:50 <jugglinmike> Matt_King: We have tabs coming up, but as usual, some of the test plan turned out to be more complicated than Isa and I anticipated, so we're still getting that ready for people
17:11:23 <jugglinmike> Matt_King: I forgot to check with Isa about "vertical temperature slider". We have the app changes and APG changes in place, so it's now just a matter of making the small changes to the arrow commands
17:11:36 <jugglinmike> Isa: I think I can prioritize that so we can complete the report by the end of Friday
17:11:54 <jugglinmike> Matt_King: Great! Even though we're not having a meeting next week, we can still try to move things forward asynchronously
17:12:08 <jugglinmike> Matt_King: It was just a couple tests where we need to circle back with people
17:12:18 <jugglinmike> Isa: PAC will be closed the entire first week of July
17:12:33 <jugglinmike> Matt_King: Ah, okay. If we have enough people available, we can still make progress
17:12:52 <jugglinmike> Matt_King: Hopefully we can get everything we need from Isa and James lined up asynchronously so we can still make progress that week
17:13:00 <jugglinmike> Matt_King: Thanks for calling that out, Isa
17:13:19 <jugglinmike> Isa: We can also work on the other disclosure--the FAQ one
17:13:29 <jugglinmike> Matt_King: Oh, right. We can adjust our plan accordinly
17:13:49 <jugglinmike> Matt_King: In early July, we could end up with a lot of things ready for testing
17:14:19 <jugglinmike> Topic: App issue 1430 - Improvements to new experience for marking commands untestable
17:14:27 <jugglinmike> github: https://github.com/w3c/aria-at-app/issues/1430
17:14:31 <jugglinmike> Matt_King: This is a pretty big change!
17:14:40 <jugglinmike> Matt_King: I want to talk about what's changing in the testing experience
17:14:51 <jugglinmike> Matt_King: In every single command, you're now going to see another checkbox
17:15:21 <jugglinmike> Matt_King: Right after the "output" field, there is a new checkbox that you check if, for some reason, when you execute it that the screen reader behaved in such a way that you can't answer the assertions
17:15:44 <jugglinmike> Matt_King: For example, if the screen reader was supposed to move to a button, but it didn't move to that button, you can't answer questions about the behavior of the button
17:15:59 <jugglinmike> Matt_King: If you make it as "untestable", then you have to answer a question about "why" it was untestable.
17:16:38 <jugglinmike> Matt_King: We automatically check the box which says that "there were negative side effects", and you need to designate why it was untestable, mark that reason as "severe", and then add a description with additional context
17:17:01 <jugglinmike> Matt_King: Isa and I did some testing before it rolled out, shared some feedback regarding three issues that we would like resolved
17:17:09 <jugglinmike> Matt_King: This issue is capturing one part of that feedback
17:17:40 <jugglinmike> Matt_King: Today, I want to talk through the details of issue #1430 and make sure that people are in agreement to the solutions to the problems that are raised by issue #1430
17:18:12 <jugglinmike> Matt_King: In my issue, I first tried to make sure that we would be aligned on exactly what the problems are
17:19:04 <jugglinmike> Matt_King: There were three problems. First, we have a label on the checkbox that is kind of hard for screen reader users to understand (and in fact, Isa pointed out that if the command was, like, the letter "b", you could read it and understand whether the letter "b" was the letter "b" or the word "b" just because of the way the label was phrased)
17:19:22 <jugglinmike> Matt_King: Then, after you check the box, we didn't have clear instructions about what you have to do next (which is to record what was untestable)
17:20:00 <jugglinmike> Matt_King: And third, if you submit the form without explaining what was untestable, there wasn't a clear message, and the focus didn't jump back to a location that was as helpful as it could be
17:20:04 <jugglinmike> Isa: I agree with all of that
17:20:49 <jugglinmike> Matt_King: On the first two issues (understanding what the label is and what it means), I feel like this is one of those situations where we ended up with a really long label because we wanted to be clear with testers about what it means. Once people understand it, they don't need such a long label
17:20:57 <jugglinmike> Isa: Right, it's overly verbose at that point
17:21:27 <jugglinmike> Matt_King: So I'm proposing that we use a shorter label along with a description. For the label, I simply wrote, "command is not testable", and for the description, it says exactly what that means
17:21:35 <jugglinmike> Matt_King: I'll copy that into the minutes, if it's helpful...
17:21:54 <Matt_King> description: Description: Executing 'COMMAND_NAME' affected behavior that made assertions untestable. If checked,
17:21:55 <Matt_King> then at least one severe negative side effect must be recorded below.
17:23:06 <jugglinmike> Matt_King: Is this good?
17:23:12 <jugglinmike> Isa: Sounds good to me
17:24:16 <jugglinmike> Isa: Testers who are more familiar with the platform, or the ones who are going to be running the test: is it clear why we are checking the checkbox? Is that instruction clear?
17:24:45 <jugglinmike> louis: from the verbal discussion, it does make sense
17:24:49 <jugglinmike> Dean: Agreed
17:25:07 <jugglinmike> louis: of course, when we see it in person, we may have a different opinion. But conceptually, it makes sense
17:25:32 <jugglinmike> Matt_King: If you try to submit a form with a side effect with a side-effect recorded, but you didn't put a description of the side effect, does that result in an error? I didn't check that
17:26:02 <jugglinmike> Matt_King: There are two error conditions: one is that they didn't input any side effect, and another is that they designated a side effect but they didn't give it a description
17:28:01 <jugglinmike> Matt_King: Are we all aligned: when you press "submit" and there are three errors on the form, focus should return to the first one, right?
17:28:07 <jugglinmike> Isa: Yes, that sounds right to me
17:28:15 <jugglinmike> Matt_King: Then I think we can give this one the green light!
17:28:27 <jugglinmike> jugglinmike: Carmen is out today, but she will see the minutes
17:28:59 <jugglinmike> Topic: Re-run JAWS report for color viewer slider
17:29:13 <jugglinmike> Isa: louis did the heavy lifting here
17:29:28 <jugglinmike> louis: "shift+f" didn't give the right output
17:29:42 <jugglinmike> louis: so my results were still the same as Hadi's on that one
17:29:55 <jugglinmike> Isa: Joe_Humbert is the other tester
17:30:04 <jugglinmike> Matt_King: Joe is not here today
17:30:36 <jugglinmike> Matt_King: So louis's output matched Hadi's when he was going backwards
17:30:45 <jugglinmike> Matt_King: This is similar to what we saw in radio group
17:31:41 <jugglinmike> louis: Now, the interesting thing is that if I sort of go off-script and just tab back-and-forth, then it will start reading
17:32:08 <jugglinmike> Matt_King: We noticed "shift+f" (I believe) in the "roving tab index" radio group
17:32:22 <jugglinmike> Matt_King: Just in some weird edge cases, "shift+f" works differently from "f"
17:32:47 <jugglinmike> louis: It's the oddest thing. I'm not sure what's going on. Eventually, you can get it to read if you mess around with it enough, but that would still be considered a failure
17:33:04 <jugglinmike> James: I cannot reproduce that. It announces the min and max value every time for me
17:33:24 <jugglinmike> James: The wording and lack of pause is somewhat questionable, but that aside, it isn't giving me any issues
17:33:35 <jugglinmike> Matt_King: I wonder, for this, because Joe isn't here...
17:33:42 <jugglinmike> Isa: I wonder if this is another Windows 10 versus 11
17:34:31 <jugglinmike> louis: I'm on Windows 11
17:35:03 <jugglinmike> Matt_King: There is no issue number for this
17:35:15 <jugglinmike> Isa: I don't get min and max here on my machine
17:35:24 <jugglinmike> Isa: I do get the "slider" role
17:36:14 <jugglinmike> Isa: If I maximize, I still don't get the min and max
17:36:36 <jugglinmike> Matt_King: I'm getting the "min 0 max 255" with "shift+f" on Windows 11. I'm running a JAWS beta that I think is just after the May update
17:36:46 <jugglinmike> James: I am not running a JAWS beta, and I'm running Windows 10
17:37:08 <jugglinmike> louis: I got mine to work. I hit the "run test setup" button and then I toggle PC cursor off and on
17:37:45 <jugglinmike> Matt_King: I refreshed the page, the focus is on the "run test setup" button. I press "enter", and JAWS announces the link, and then I press "shift+f", and it actually says "color viewer group left right slider 128 min 0 max 255"
17:38:01 <jugglinmike> louis: I did exactly what you did, and I got "color view group left right slider 128"
17:38:29 <jugglinmike> louis: But if I refresh it, press "run test setup", toggle virtual PC cursor off and on, then press "shift+f", I get the expected output
17:38:43 <jugglinmike> louis: But it seems every time I look at this, I observe something different
17:38:49 <jugglinmike> Isa: It's iconsistent
17:39:01 <jugglinmike> Matt_King: Are we all on Chrome 137 point something?
17:39:04 <jugglinmike> louis: I am
17:39:47 <jugglinmike> Matt_King: I don't know what to think of the extra step that Louis is inserting in order to get it to work
17:39:56 <jugglinmike> Matt_King: I don't have to take that extra step, and neither does James
17:40:08 <jugglinmike> Isa: But I do need to insert the extra step
17:40:12 <jugglinmike> Isa: And I'm on default settings
17:40:40 <jugglinmike> louis: It depends. If I reload it, and I run it five times, then on the fifth try, I may get it to read
17:40:52 <jugglinmike> Matt_King: By the way, I did maximize the window with the example, but that didn't change the behavior
17:40:59 <jugglinmike> louis: I have my Chrome default to maximize
17:41:27 <jugglinmike> James: For ARIA-AT, the popup does not open maximized by default even if Chrome maximizes by default
17:42:34 <jugglinmike> louis: For what it's worth, I ran it both on default settings and my configuration. That didn't make a difference
17:42:58 <jugglinmike> Matt_King: I don't know why it says "group"
17:43:07 <jugglinmike> James: Because there are multiple sliders, so they're in a group
17:44:13 <jugglinmike> James: I just reproduced the bug
17:44:22 <jugglinmike> James: Is there something wrong with this example?
17:44:26 <jugglinmike> James: I'm on Windows 10
17:44:41 <jugglinmike> Matt_King: The other people who are not getting the announcement (consistently), they are on Windows 11
17:44:53 <jugglinmike> Isa: I don't think there's anything wrong with the example
17:45:26 <jugglinmike> Matt_King: When I "shift+tab", I don't hear "group"
17:45:48 <jugglinmike> Matt_King: I've done it now many times, and I'm getting the "min"/"max" announcement every time (on the test page specifically)
17:47:02 <jugglinmike> Matt_King: I want to figure out a path out of this hole
17:47:14 <jugglinmike> Matt_King: We have something that's flaky, but we don't know the conditions for the flakiness...
17:48:24 <jugglinmike> James: Is this in a pull request or on "main"?
17:48:29 <jugglinmike> Isa: It's on "main". It's a conflict
17:48:44 <jugglinmike> Isa: For the record, it's only with JAWS. For the others, the behaviors are met
17:49:00 <jugglinmike> James: NVDA doesn't read "min" and "max", though, so it fails
17:49:04 <jugglinmike> Isa: Yes, that's right
17:49:38 <jugglinmike> Matt_King: In the ARIA-AT test case, I can't get it to fail no matter what I do. I am on a slightly later build of JAWS, so that could be a factor. I'm on Windows 11 and the same version of Chrome
17:49:49 <jugglinmike> Isa: Now I got the min and max, just by re-opening the test page
17:51:19 <jugglinmike> jugglinmike: Is this untestable? A room full of people can't agree on the behavior, and that seems like a precondition to testing to me
17:51:41 <jugglinmike> Matt_King: I've got to figure out how to move us off of this topic, but I'm honestly feeling a bit stuck
17:52:13 <jugglinmike> Matt_King: Do we wait for the next version of JAWS? The next beta release is in July, which isn't that far away now. We could hang out for a while and see if this gets better
17:52:31 <jugglinmike> Matt_King: James is right to question the integrity of the test case itself, but we're not finding any problems, there
17:53:06 <jugglinmike> Matt_King: I guess, with lack of a better option at this point in time, I'm kind of feeling like we should put this on ice until the July release
17:53:19 <jugglinmike> Matt_King: If there was a bug, Vispero would want the bug to be associated with the July release, anyway
17:53:32 <jugglinmike> Matt_King: I appreciate everyone's energy and enthusiasm in geeking out over a specific test case!
17:53:40 <jugglinmike> Matt_King: We're very close
17:54:48 <jugglinmike> Topic: Run of accordion test plan
17:54:53 <jugglinmike> Matt_King: We'll skip this for now
17:55:01 <jugglinmike> Topic: Conflicting results in Rating Radio Group
17:55:18 <jugglinmike> Matt_King: Dean had output that was completely different than the other testers, and I'm wondering if that came from Dean or from the bot
17:55:28 <jugglinmike> Matt_King: Do you think you could re-run the test, Dean?
17:55:30 <jugglinmike> Dean: Sure
17:55:40 <ChrisCuellar> ChrisCuellar has joined #aria-at
17:55:42 <jugglinmike> Dean: This may be a version thing, too. I'll have to check
17:55:55 <jugglinmike> Isa: This is the issue with the laptop key
17:56:08 <jugglinmike> Dean: Ah, right, then I got a cheap external keyboard
17:56:27 <jugglinmike> Dean: That allowed me to do it, but I don't know why there was a conflict. That was a while ago, so I'll have to look again. I'll do that today
17:57:11 <jugglinmike> Matt_King: This might have been bot output and not your output, but if you could manually go to test 14 and 15 and re-run them and make sure that the output that is recorded is accurate, that will move us forward
17:57:15 <jugglinmike> Dean: I will do that
18:00:26 <jugglinmike> Zakim, end the meeting
18:00:26 <Zakim> As of this point the attendees have been jugglinmike, ChrisCuellar, Matt_King, howard-e, Isa, james, elizabeth, louis, dean
18:00:28 <Zakim> RRSAgent, please draft minutes
18:00:29 <RRSAgent> I have made the request to generate https://www.w3.org/2025/06/18-aria-at-minutes.html Zakim
18:00:36 <Zakim> I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent.  Goodbye
18:00:36 <Zakim> Zakim has left #aria-at
19:25:50 <jugglinmike> RRSAgent, leave
19:25:50 <RRSAgent> I see no action items