Meeting minutes
Review agenda and next meeting dates
https://
Matt_King_: Requests for changes to agenda?
jugglinmike: I'd like to ask about the version of NVDA that testers have been using
Matt_King_: Okay
Joe_Humbert: Is there a way to input a new version of the software? A lot of time, I have to choose the wrong version because the version I'm using is not listed
Matt_King_: Okay, we'll discuss those things together, right after the status update
Matt_King_: Next AT Driver Subgroup meeting: Monday March 24
Matt_King_: Next Community Group Meeting: Wednesday March 26
Matt_King_: No meeting Thursday April 3
Current status
Matt_King_: There aren't really any major updates to what's happening in Candidate Review (with one exception, but that's covered later when we talk about what's on hold)
Matt_King_: We still have the same 13 plans in Candidate Review, hopefully it will be 15 plans, soo
Matt_King_: We have three we're working on in Draft Review. One of those (a new version of "radio group") is already in Candidate Review
Matt_King_: In terms of updates that are in progress, the one that is really close is "vertical temperature slider" and IsaDC has another with a goal of the end of next week
IsaDC: The "rating slider"
Matt_King_: That will wrap up sliders for us.
Matt_King_: Right now, we have a brand-new version of the "disclosure navigation menu" test plan that's in Candidate Review
Matt_King_: We're going to re-run that again after the new JAWS is released. We should probably re-run at least the final test for all of the screen readers (since it may change how we report things). We're going to save that until May
Matt_King_: I met with Vispero yesterday. They were hoping that the fix for same-page links would be released in March. They had to roll back the fix, unfortunately, so it won't ship until May
Testing radio group with active descendant
Matt_King_: NVDA and VoiceOver complete!
Matt_King_: Zero open linked issues!
Matt_King_: We have IsaDC's results, and we have partial results from Hadi which are out of date
IsaDC: I still have not heard from Hadi, neither in my personal e-mail nor in the mailing list
Matt_King_: I think we'll have to get someone else to run this and delete Hadi's test plan run. There are 39 conflicts there because they are basically out of date
Matt_King_: Do we have a taker for JAWS?
Joe_Humbert: I logged an issue with JAWS producing inconsistent results. I'm a bit worried that I'll run into that same issue
Joe_Humbert: I was sometimes getting some information and sometimes not, using the same key strokes each time
Joe_Humbert: I don't think this was related to my use of the demo version of JAWS
Joe_Humbert: If we have a lot of testing that needs to be done, there is a three-month license that I can look into purchasing
Matt_King_: We're hoping to get more volunteers for JAWS
Joe_Humbert: I can re-run this one, it's just going to take me a little while. I ran this test plan with NVDA, so I know there's a lot to do there
IsaDC: Thank you, Joe_Humbert
Matt_King_: By the way, as we are anticipating onboarding some more people, a common question that we have been getting is: "how long does it take to run these test plans?"
Matt_King_: IsaDC typically tells people a long test plan takes 90 minutes
Matt_King_: I usually take more time than that
Joe_Humbert: Each of the radio test plans took me about 2 hours
Joe_Humbert: When we had to re-do the one when we already had responses, I got through those a lot quicker because I was just confirming what was already there (rather than entering the information manually)
IsaDC: I timed myself for fun, earlier. When running the "media seek slider" test plan with VoiceOver, it took me about 25 minutes (which included double-checking my results--I went back to each and review)
Matt_King_: That's fast! Though you are very good with the recorder
mmoss: I don't have an estimate for my time; I'm often distracted by pings while testing
IsaDC: I can delete Hadi's test plan and create one for Joe_Humbert
Testing Radio group with roving tab index
Matt_King_: We're in largely the same situation with this one, so I think we want to do the same thing
Matt_King_: Let's prioritize "Active descendant," first and then see how that goes
Matt_King_: Then if Joe_Humbert or mmoss or someone else has bandwidth next week, we can tackle "tab index"
Matt_King_: There are two issues here in the agenda
Matt_King_: Both of these are related to the VoiceOver Bot
Matt_King_: The first issue is 1171 w3c/
Matt_King_: This is related to extra spaces in the output
Matt_King_: If an issue related to the bot was raised in a separate repository, we could reference this issue there and close this issue
Joe_Humbert: I was consistently getting the same issue with the bot where I was observing a significant number of extra spaces
jugglinmike: We can create a separate issue for the bot, reference this issue there, and close this
howard-e: You can assign this issue to carmen
Matt
Matt_King_: Done
Matt_King_: The second issue is 1172, w3c/
Matt_King_: It's also about the bot. Should we handle this in the same way?
jugglinmike: We haven't investigated this, yet
Matt_King_: I'm going to assign this to carmen, as well
carmen: Sounds good
ChrisCuellar: We have an issue for this. I'll reference this issue from that and close this
Matt_King_: Joe_Humbert, when you're running a test, sometimes these things are related to what happens in a specific test. So reporting in this way creates some helpful trace-ability
howard-e: An issue label such as "bot" might be helpful, but I don't know whether a Tester should add such a label or if we should add it together during these triage discussions
ChrisCuellar: We can propose a process for this kind of categorization
Matt_King_: If anyone raises a test plan issue and the issue is related to the bot or the app or anything like that, then in general, that's totally okay as long as we have a fast way to close it (e.g. by linking to it from a app or bot issue)
Matt_King_: I have no problem with that because it's so convenient to raise from the app like this (and because you might not know whether the issue is specific to some other sub-system)
carmen: I can own this--it's already part of my process, informally
Matt_King_: Great!
Testing Media Seek Slider
Matt_King_: This has three issues open. I think two of them (1205 and 1206) are the same issue
Matt_King_: 1205 is related to decrementing by one step
Matt_King_: This is a rendering problem. But there are a lot of comments. I want to make sure this is fully resolved
Matt_King_: We're not going to take any of howard-e's suggestions because Joe_Humbert has already re-run it manually
IsaDC: So we can just close 1205, then
IsaDC: Although I caught an editorial error. I'm just scared to push any new version of that test plan!
Matt_King_: It should be safe
Matt_King_: Anyway, I will close issue 1205
Matt_King_: Can we close 1206?
carmen: I'm going to clean this up after this call
Matt_King_: Excellent
Matt_King_: And now for issue 1209.
Matt_King_: This is Joe_Humbert's issue about an inconsistency with the "f" key in JAWS
Matt_King_: You finished the plan, and you got the expected result some of the time
Joe_Humbert: I don't know if it was the expected result because I got both. The test plan allows for both to be there. It has questions for both
Joe_Humbert: Sometimes, it wouldn't say the minimum and maximum values. Other times, it would. I used the "f" key for all cases
Matt_King_: I don't remember it saying min and max when I was running the test plan, though I was using an older version of JAWS
Joe_Humbert: I think I ran it 7 or 8 times, and I think it said the min and max value only two of those times
IsaDC: I haven't run the test plan yet, but I'm planning on doing so today
Matt_King_: Okay, so let's leave this open for now. In cases where flaky behavior yields a bad result a reasonable amount of time, I would be tempted to have us record the bad behavior in the test results
Matt_King_: We don't have a side-effect called "sometimes works"
Joe_Humbert: And I don't think we should have that
Matt_King_: Yes, but unless it shows up in a report somehow, it's hard to track it
Matt_King_: Let's leave this open for now and see what IsaDC experiences
Screen reader versions
Matt_King_: Joe_Humbert mentioned that the version he is using is sometimes not available
Matt_King_: James, IsaDC, and myself all have the ability to add new versions to the system. The first step would be to ask us to add a new version
Joe_Humbert: I experienced this with JAWS most recently, but I've also experienced it with VoiceOver and NVDA
Matt_King_: When I look at the list of all NVDA versions on the "Data Management" page, I'm not seeing 2024.4.2. I see 2024.4.1.
IsaDC: I will re-add it
James: Before you re-add it. If there is an issue with versions not being listed, I think we need to know that
James: If I assign myself to NVDA and then click "Start Testing", the latest is 2024.4.1
James: are we saying that we added that and you can see it in your test queue?
Matt_King_: When you are in the test queue, IsaDC, are you saying that you can see 2024.4.2?
howard-e: They appear in order of release date, descending
James: can we track when each version was added to the app and display that?
jugglinmike: That could be addressed by our larger plan to surface all events (rather than extending this specific UI with still more information)