Meeting minutes
<Joe_Humbert> 3 of us got in
<Joe_Humbert> I clicked jion from the email that was sent out 20 minutes ago
Manual testing progress check-in
Matt_King: We have four plans that we're trying to get new versions into "candidate review"
Matt_King: Alert, command button, link, and toggle button
Joe_Humbert: I use VoiceOver completely up-to-date. I have NVDA on my personal machine, but I'll have to check if it's completely up to date
Joe_Humbert: I do not have JAWS
Matt_King: That's very helpful; we have folks who can test with JAWS
Matt_King: We don't have anyone assigned to Command Button with VoiceOver besides Isabel
Matt_King: Isabel completed a run for Command Button, so we only need one additional tester
Matt_King: It looks like Alert might be ready to advance to candidate, now
Joe_Humbert: It might be due to the testing I completed just before Christmas. I can't recall which test plans I ran, though.
Joe_Humbert: Out of curiousity, is there a way for me to see what I've already tested?
Matt_King: Not at the moment
Joe_Humbert: As soon as you change the phase, it leaves the Test Queue, and I no longer have a way to see what I've done
Joe_Humbert: It's not a big deal for me, though
Matt_King: It would also be useful for administration, if only to recognize the contributions of all the volunteers
Matt_King: I've made a note of this to myself; I'll raise an issue at least (though I don't know where it fits in the roadmap)
Matt_King: I want to prioritize VoiceOver and Safari for the remaining three: command button, link, and alert
Joe_Humbert: I can do all three no problem. I might not be able to complete them by next week, but I can certainly complete them by the next meeting on 2024-01-25
Matt_King: That's awesome! You can go ahead and use the "assign yourself" button on https://
Matt_King: Please try to complete them in alphabetical order
Matt_King: I have a meeting with Vispero on the 24th. If you could get a few done before then, that would be very helpful
Joe_Humbert: I think that's definitely possible
Matt_King: Great!
Upcoming app updates
Matt_King: The plan is to deploy into production on 2024-01-22 with the NVDA automation
howard-e: That's right
Matt_King: We also have some bug fixes. I haven't reviewed the entire list of everything that needs to be tested on the "sandbox" server yet, though
howard-e: I think "staging" is where the bug fixes are, now
howard-e: The "sandbox" server is running the automation work right now, which is currently on a distinct branch of development from the bug fixes
Matt_King: Ah, yes, I remember that, now
howard-e: That includes the "key up" work. Also, there was an issue with the test results not being properly migrated when there was a new R&D version pushed. The fix for that should be included, too
howard-e: I think there are some other details, as well.
howard-e: We should be able to support the v2 test report page, as well
howard-e: There are a couple things that would also be important, but I don't think they will be a part of "staging" necessarily
howard-e: For instances, supporting "level 0" assertions, and also capturing severity
howard-e: Those are pull requests that exist for ARIA-AT, not the App. But once those changes in ARIA-AT land, then we should be ready to move on the corresponding updates to ARIA-AT App
howard-e: Everything I'm mentioning here, I'll push to staging toward the end of the day
Matt_King: If you could send me an e-mail when you do, that would be great
Matt_King: I'm looking forward to announcing those changes
Matt_King: Joe_Humbert do you have any feedback about the new content in the data collection form?
Joe_Humbert: It seemed fine. I haven't come across any unexpected behavior, yet, though. That will be the true test
Matt_King: Ah, okay, though we don't have those specific changes deployed yet
Joe_Humbert: Having fewer tests overall was definitely nice
Targets for 2024 H1
Matt_King: I have a proposal here in the agenda. The first part is a breakdown for what I'd like to see from a testing perspective
Matt_King: It's "6+ test plans achieve recommended status," and we're well on our way for that
Matt_King: For the next set of testing, we'll go back to using our prioritization method
Matt_King: I'm hoping we have enough people to get all that testing done before the middle of April
Matt_King: Then we'll have a long tail of stuff that we can get into the "draft review" phase and ready for the "candidate" phase
Matt_King: I think we'll need more people present in this meeting before we can have a meaningful conversation about whether this plan is too aggressive or not aggressive enough
Matt_King: We still have a lot of talking to do about the details of the App development
Matt_King: But I've laid out some goals in broad strokes on today's agenda
Matt_King: 1. Total: refactoring and testing complete for at least 20 test plans.
Matt_King: 2. 12 to 20 test plans have AT support tables in APG.
Matt_King: 3. APG AT support tables are redesigned to show must/should/may counts.
Matt_King: 4. Automation is updating data for each new release of JAWS, NVDA, and VoiceOver.
Matt_King: 5. Report site shows trends for each AT.
Matt_King: When I meet with the folks at Vispero, I'll be speaking with them about their recent efforts to run the tests automatically
Matt_King: I believe they have some technical questions, so I'll be sure to connect them to howard-e and jugglinmike to discuss
Joe_Humbert: It'd be nice to get manual test results from Vispero in addition to the two volunteer-reported results
jugglinmike: And Z has started on the VoiceOver work. I don't have a sense for when that will be ready for production, though
Matt_King: As for the fifth goal: we want to be able to understand how interop data has changed over time, both at a high level and for any given test plan
Matt_King: That's another thing we're hoping to complete before June 30
Matt_King: We launched the first version of the support tables in time for Global Accessibility Awareness Day, so I'm thinking it could be nice to launch version two of those tables in time for the even this year
howard-e: Sounds good! I'm especially looking forward to being able to surface trends and progress