Meeting minutes
Review agenda and next meeting dates
Matt_King: Next AT Driver Subgroup meeting: Monday November 17, but this is expected to change.
Matt_King: Next CG meeting: Thursday October 30
Matt_King: Requests for changes to agenda?
IsaDC: Perhaps checkbox?
Matt_King: I didn't add checkbox because switch is already present
Matt_King: howard-e and carmen have previously requested topics, but we'll wait on those since they can't attend today's meeting
Matt_King: Hearing no further requests, we'll stick with the agenda as planned
Current status
Matt_King: There are six plans in "draft review" (though due to a clerical error on my part, it appears that there are only five)
Matt_King: The "Switch with HTML checkbox" and "two-state checkbox" plans are ready for people to work on
Matt_King: The spin-button example is coming up
Running test plan for Tabs with Automatic Activation
Matt_King: We'll skip this topic because Hadi isn't present today, so it's down to IsaDC or me to resolve the conflicts. It should probably be me
Running test plan for Tabs with Manual Activation
IsaDC: There is a new version. I fixed some VoiceOver commands
Matt_King: I took care of that
Matt_King: On the VoiceOver test plan, you merged a fix to test #2
Matt_King: So we have the run by Joe_Humbert--it's complete now, except for test #2.
Joe_Humbert: I'll get test #2 by next week
Matt_King: It is currently ready for Dean to tackle, though Dean isn't present today
Matt_King: That's it for VoiceOver
Matt_King: We need to let Dean know that it's ready for him
Matt_King: Let's look at the 5 conflicting NVDA results. These are conflicts between Joe_Humbert and Dean
Matt_King: Starting with test #4: this is one where you both recorded the same output. However, you had different assertion verdicts
Matt_King: So, "shift+tab". The mode switching one, dean was saying that it was working for him
Joe_Humbert: That could just be because I missed it. I'll have to double-check
Joe_Humbert: I'm guessing that I marked it wrong. I know that "shift+tab" generally changes the mode
Joe_Humbert: I'm guessing that I marked it "yes" on the previous test that was similar
Joe_Humbert: I'll change my answer after I double-check it
Joe_Humbert: It's weird that my browser version is listed as 99.0.4.8. I guess I'm just verifying the bot results. That's a very old version!
IsaDC: Is that what the bot is running?
Joe_Humbert: I don't know
Matt_King: Test #13 is another one where you both have the same output. Here, you reported a difference in negative side effects
Matt_King: "The tab panel boundary was ignored." And "the focus moved to the content of the tab panel"
Matt_King: I'm curious about why you recorded that as a side-effects. Don't we have an explicit assertion for that? If the assertion fails, then we shouldn't need to track this with a negative side-effect
Joe_Humbert: It has been a very long time; I can't recall why. Generally, I report failures in this way when I want to make sure we discuss it here on this call, but I can't say for sure
Matt_King: I think dean's should be changed, here.
Joe_Humbert: Based on this, I'm kind of confused why it was not announcing the tab panel (or property page, in this case). I think it did for the tabs key, so I was confused why it doesn't do it for the arrow key
Joe_Humbert: That might be why I put a note in for the previous test--just to make sure I wasn't missing anything
Joe_Humbert: The next two conflicts are the same
Matt_King: There's a part of me that is wondering... "Property page"--is that really...? I guess if you're an NVDA user...
mmoss: This reminds me of VoiceOver reading grids as labels
james: I think there's a fine line. We explicitly are not in the game of dictating what screen readers say. I would say that semantically, there is no such role within NVDA as "tab panel". If you search the source code for "tab panel", you won't find anything. They are treated as "property pages"
james: perhaps NVAccess assumed that if you are switching to NVDA to JAWS, you would be better served by familiar vocabulary (although JAWS has since changed their terminology)
Matt_King: Instead of bringing NVDA up to date, we'll bring dean back in time and get aligned with NVDA
Matt_King: So that's three of these conflicts
Joe_Humbert: The next two (for tab in both browse mode and focus mode), it's the same issue
Matt_King: Okay, so for the very first conflicts, you'll validate.
Joe_Humbert: For the second, I'll remove the side-effect
Matt_King: And for the next two, we'll have to talk with dean
Joe_Humbert: That just leaves test #15
Matt_King: So this is "activate a tab in a tab list" and it's related to the space
Joe_Humbert: The output we reported does not match
Joe_Humbert: If this is not an issue, I'm happy to go back and remove a negative side-effect
Matt_King: For JAWS, we put a moderate side effect
Joe_Humbert: I reported it as severe, but I can change that
Matt_King: Yes, this is definitely moderate. It's just redundant speech
Joe_Humbert: Okay. I'll also re-test just to verify that it's still an issue
[Dean joins the call]
Matt_King: In test #13, for "down arrow" and both of the "tab" commands, the word "property page" is in the output, and that is NVDA-speak for "tab panel". You marked it as failing for "conveying the 'tab panel' role"
dean: We both recorded the same thing. There's no "tab panel" in that output
Matt_King: But the term "property page" is.
dean: Ah, okay. So I need to change all three commands to passing. I can do that
Matt_King: Then, in test #15, we would like you to re-run and check the output for the space bar
Matt_King: Joe_Humbert's output is showing that "selected" is showing multiple times
dean: I've seen that before elsewhere, so I'll check it again
Matt_King: Now I'm wondering if we actually want to record the negative side-effect for this one
Matt_King: This is tabs with manual activation, so pressing the space bar generates a state-change event: the tab goes from "not selected" to "selected". I guess NVDA is reading the result.
Matt_King: James, I'm curious about your thought on this. What do we expect for a speech response in this case?
Matt_King: You're pressing the space bar on a tab that's not selected
James: As an NVDA user, I would expect to hear the word "selected" (The change in state), and that's it
Matt_King: Well, we can call it excessively verbose because after reading the state, it reads the name, the role, and the set again
Matt_King: We only have one assertion here, and that is that the change in state is conveyed
james: There is a persistent issue in NVDA where, when you activate a tab, it reads the tab again
Joe_Humbert: When I re-ran the setup for this test, it puts you into forms mode. You have to run the setup again
Matt_King: That's expected because it's a tab that receives focus
Joe_Humbert: I'm not hearing "select" a second time, so it might have just been the bot results
Joe_Humbert: I'll change my results
Matt_King: Okay, so Dean, you have nothing to do on test #15
Joe_Humbert: The left-arrow test still has all my results, but the shift+f keypress does not.
Matt_King: I don't know how that happened
dean: I changed my results for test #13, so I'm showing no conflicts, now
Matt_King: Okay, great
dean: I was seeing weird results, so I updated, and then, my results seemed accurate. But they still didn't match with the bot
Matt_King: If you unassign yourself, will that delete it?
Dean: That's how I did it before. I had completed only a couple tests, unassigned myself (wiping it), then started a bot run, and finally re-assigned the bot run to myself
Matt_King: You can do that again.
ChrisCuellar: The bot is still behind (we still need to update it), but that seems like it isn't the issue here
Matt_King: If we collect using the bot at version 15.6.1, then if the bot results are accurate then you can just leave it
Dean: sure. It doesn't matter if we have the same version as long as we get the same results
Joe_Humbert: I just need to re-do the one test, correct?
Matt_King: That's right
Joe_Humbert: I'll do that ASAP
IsaDC: That just leave me to finish the JAWS test run. I'll do that later this afternoon
Matt_King: Perfect
Running tests for Switch Example Using HTML Checkbox Input
Matt_King: We have three bot runs available, and we don't have any testers
Matt_King: We're looking for two volunteers for each screen reader
dean: I'll do VoiceOver
Joe_Humbert: You can sign me up for all three. I'll have time for that over the next two weeks
dean: I can do NVDA eventually, but I have three test plan runs assigned to me already, so it may be a bit
mmoss: I can run one of these for switch.
mmoss: The test queue isn't loading for me at all. I'm getting a 502 Bad Gateway
Matt_King: Same for me. It was loading correctly just a moment ago
ChrisCuellar: We saw a 502 last week. It's loading for me right now, for what it's worth, though it is a little slow
ChrisCuellar: I'll file an issue for my team to look into offline
mmoss: I have access to all three screen readers
Matt_King: Maybe we should assign you and Joe_Humbert to JAWS (because dean doesn't have access to JAWS)
Matt_King: Then we can have Joe_Humbert and dean on VoiceOver
Matt_King: Then, for NVDA, we can wait and see who is ready first
Matt_King: But we can't do that right now because the site is down
Matt_King: I think that by next week, we'll have both "tab" plans done and we'll be working on "switch." Hopefully we can start working on "checkbox"
Matt_King: I think that covers it. There is a lot more potential work in the automated-update space. Elizabeth finished one.
Matt_King: Thank you for completing the "action button" test plan with VoiceOver!
Matt_King: Now, we have these other plans in the test queue. Do you have availability to work on one in the next week?
Elizabeth: yes, I saw that the "action menu button" is available in the automated updates tab, so I assigned myself to that.
Matt_King: Right, you did the "active descendants" one. If you would like to take the other, that would help
Matt_King: The "command button" test plan for VoiceOver is showing up as a manual test run, but it was an automated test run...
ChrisCuellar: Please file an issue and we can take a look into it
ChrisCuellar: I'm filing an issue for the test queue performance
dean: There was a big lag when I tried to assign it to myself. It wouldn't work at all
ChrisCuellar: We have an issue for the lack of feedback when you reassign. That issue is still on our queue; we just haven't gotten to it, yet
Matt_King: Thank you everybody. We're making some great progress, and I appreciate all of it!