Meeting minutes
Meeting tine survey
Sam_Shaw: We got 10 responses. This slot on Thursdays works for 5, it's somewhat workable for one person, and not workable for four.
Sam_Shaw: The other slots are definitely workable for fewer people
Matt_King: There were 17 responses on the first version of this survey, which had more time slots on it
Alyssa_Gourley: It might be a little confusing, but we could alternate between two different meeting times every week
Matt_King: That sounds like a pretty good idea. Thank you
Sam_Shaw: I will send the survey results to Bocoup
testing schedule
Matt_King: I'd like to decide which tests we still want to perform in March, and from information we currently have available, pick the next three-to-five patterns we'd like to pick in the April-to-May timeframe
Matt_King: I want to figure out exactly where we're incomplete on any test plans
James_Scholes: Testing is totally complete, I think one tester just forgot to press the "submit" button
James_Scholes: We're just blocked on not having any prioritization in the spreadsheet
Matt_King: Then let's pick a candidate so we have an immediate plan
Matt_King: Here's the planning spreadsheet: https://
Matt_King: We didn't put a column for us to specify a sequence and then sort by that
James_Scholes: I was expecting just to re-order the rows
Matt_King: The only problem with that is that it prevents us from being able to sort the spreadsheet for other purposes--if we do that without a dedicated column for "priority," we won't be able to restore the priority order
James_Scholes: Understood
Matt_King: Okay, then I'll add a column to track priority
Matt_King: There are two different kinds of testing we can do at this point. One is we can do testing for more combinations of the first eight patterns. Another is we can get other patterns ready for the "candidate review" phase
Matt_King: My preference would be to have at least two more test plans ready for "candidate review"
Matt_King: I say this because I don't know how much more work we'll have to do with screen reader developers in terms of changes to the plans. My expectation is not much. I may be wrong, but I think we'll get about five plans to the "recommended" phase in the coming weeks. That would leave us room to get a few more plans in candidate review
James_Scholes: I agree with that--if we can make it happen, that'd be good.
James_Scholes: Hopefully, in the mean time, we can get the data we need to prioritize the plans we have in the sheet
Matt_King: We don't have VoiceOver with Chrome or Firefox with JAWS or Firefox with NVDA. That's not a great thing for APG. We should work on filling in those three gaps next, especially for the first five we have ready for recommended
James_Scholes: That's a lot of work. We can commit to that, or we can look at doing two more plans with the existing combinations.
Matt_King: My preference is to make sure we have at least two plans ready for candidate review first, and then go back and extend coverage for the others
James_Scholes: My preference would be to revisit the first eight with more combinations, and then to start prioritizing the spreadsheet once we have APG visitor data
Matt_King: If we did the first five with those three screen reader combinations, then that would be a significant improvement to what shows up in the APG. Essentially, we'd go from providing three data points to providing six data points for support
Matt_King: Maybe that's what we should do, then!
James_Scholes: Will we need two testers for each?
Matt_King: Yes, for quality-control purposes
Dylan_Sheffer: I volunteer to do some testing. I have access to JAWS, NVDA, and VoiceOver
Matt_King: Great. Which one we ask you to test will depend on what other people are able to test
jongund: I can take a few. I also have access to JAWS, NVDA, and VoiceOver
jongund: I'll take VoiceOver and Chrome
Dylan_Sheffer: I'll take Firefox and JAWS
Matt_King: Let's start with the "Command Button" and "Toggle Button" pattern
Matt_King: And then we'd go to "Alert", then "Link", and then "Radio"
Matt_King: But for now, let's try to get aligned on doing two at a time
Alyssa_Gourley: I can take NVDA and Firefox
Matt_King: Basically, for Windows screen readers, we're doing Firefox, and for macOS, we're doing Chrome
Matt_King: I said "VoiceOver and Firefox" is out-of-scope. I said that partly to conserve our bandwidth, but also because I would be surprised if any screen reader users use that combination
Alyssa_Gourley: I think JAWS and Firefox could be interesting, though I don't know if people actually use that combination, either
<jongund> i need to leave early
JoeHumbert: WebAIM has rough usage statistics
JoeHumbert: The most recent data I can find is from 2021
(discussion of the WebAIM survey data)
Matt_King: We have a plan for the next couple weeks. We'll revisit on a rolling basis
James_Scholes: I'll send out an e-mail with some guidance to the volunteers
Gathering broader community input
Matt_King: I've been thinking about creating a survey with a Google Form, maybe opening it up to the whole ARIA working group (I don't know about going broader than that), and make a short survey. Three questions: which of these test plans (and provide a list) do you think is most important? Second question: which is the next-most important? And third: which is the third-most-important?
Matt_King: Basically, just say, "pick you top three" and use that as one of our inputs.
Matt_King: As opposed to making this decision just within this group
James_Scholes: We've had this spreadsheet for just over a month, and we haven't made any inroads on it. If we're going to do this, we should do it sooner rather than later.
Matt_King: This doesn't have to be on the critical path. We could use input like this at any point to inform the decisions we're trying to make at that moment. It doesn't have to line up perfectly with today, for instance
James_Scholes: That makes sense
Matt_King: And maybe instead of plans, we just list the patterns. There's two different kinds of List Box, five different kinds of Slider. Maybe we don't need to break it down into so much detail. Does that seem reasonable?
James_Scholes: Someone may think that testing a certain type of slider is very important more than they think that testing sliders in general is important
Matt_King: That seems unlikely to me
Alyssa_Gourley: limiting the survey to the patterns would make the list much shorter
Matt_King: I'm also thinking about what we need from the data
James_Scholes: So if Slider was a "winner", would we prioritize all five of the slider plans?
Matt_King: I think we would just make sure to prioritize at least one representative example
Sam_Shaw: We can think about it, discuss it, and come back with a proposal
Matt_King: Okay, then lets give it a shot
AT Support Table Launch Plan
Matt_King: The Bocoup team is working on updating the app so we have the ability to publish the latest data to the reports
Matt_King: I'm hoping that there's only one bug in the current sandbox code that needs to be fixed, and then it will do the basics of what we need it to do so then we can push the latest data to the report site. Then, we'll be able to point the APG iframes to the latest "report" site
Matt_King: At the same time, we're going to bring Apple and Vispero into the discussion, showing them the preview and some early launch messaging
Matt_King: I sent an e-mail to this effect to Vispero, and I'm going to send one to Apple
Matt_King: I mentioned that this will be ready for them to look at next week. Which is a bit of a risk, since it's not ready yet
Matt_King: We'll finish preparation for the launch during the week of April 3.
Matt_King: We'll publish the APG on Tuesday, April 11.
Matt_King: We'll make sure everything looks good on April 12
Matt_King: We'll make the announcement on April 13
Matt_King: I didn't mention the APG preview in pr 2569 in my message to Vispero. Some content changes are not available, and some of the JAWS data is not the correct data. I'm going to follow up with Alex Flennikan about this in the pull request
Matt_King: Since we can't push the latest test results to production, it's hard to see what the latest test results are
Matt_King: For instance, I don't know what the scores for JAWS with the Toggle Button are. Right now, I'd have to manually walk through the plans
Matt_King: I'll be writing up that feedback. There's data that we need to be able to run the process that we still can't see
Matt_King: For Monday, we're looking for the bug fix that Howard is working on right now. I want to be able to test that and then deploy it to production
Matt_King: Then we can promote the test results that we currently have in the queue to the test site