18:51:33 <RRSAgent> RRSAgent has joined #aria-at
18:51:37 <RRSAgent> logging to https://www.w3.org/2023/10/05-aria-at-irc
18:51:37 <Zakim> RRSAgent, make logs Public
18:51:38 <Zakim> please title this meeting ("meeting: ..."), Matt_King
18:51:57 <Matt_King> MEETING: ARIA and Assistive Technologies Community Group
18:52:09 <Matt_King> CHAIR: Matt King
18:52:13 <Matt_King> present+
18:52:20 <Matt_King> rrsagent, make minutes
18:52:21 <RRSAgent> I have made the request to generate https://www.w3.org/2023/10/05-aria-at-minutes.html Matt_King
18:52:28 <jugglinmike> jugglinmike has joined #aria-at
19:02:44 <howard-e> howard-e has joined #aria-at
19:02:52 <howard-e> present+
19:03:00 <jugglinmike> present+ jugglinmike
19:04:23 <jugglinmike> scribe+ jugglinmike
19:06:47 <jugglinmike> Matt_King: Currently, I'm still planning on having a meeting on Wednesday of next week
19:06:53 <jugglinmike> Matt_King: Though the Bocoup team will be out
19:07:16 <jugglinmike> Matt_King: It depends on whether or not I come up with more blocking issues related to finishing out the specification of things that the Bocoup team is waiting on
19:07:27 <jugglinmike> Matt_King: If nothing comes up by Tuesday, I may cancel next Wednesday's meeting
19:07:36 <jugglinmike> Matt_King: But I want to keep it on the calendar for now
19:08:00 <jugglinmike> Matt_King: There is no automation meeting on October 9th--that is also cancelled. The next automation meeting will be October 23
19:09:06 <jugglinmike> Topic: Update on ARIA-AT site changes
19:09:21 <jugglinmike> Matt_King: On Tuesday, howard-e and the team at Bocoup pushed out a huge update to the website
19:09:30 <jugglinmike> Matt_King: I wanted to take some time to walk through the changes with this group
19:09:43 <jugglinmike> Matt_King: They're really important updates, and I think it's important for people to be aware of them
19:09:56 <jugglinmike> Matt_King: The agenda includes a link which leads directly to the new "data management" page
19:10:50 <jugglinmike> Matt_King: This page is available to anybody who comes to the website--there's a different form for "signed out" users and for "signed in" users
19:11:11 <jugglinmike> Matt_King: Specifically, the view changes for "signed in" users who are admins
19:11:23 <jugglinmike> Matt_King: At a high level, this page is to help with version control of test plans
19:11:40 <jugglinmike> Matt_King: As we move a Test Plan through the working mode, we want to be able to see which version of which test plan is in which phase of the working mode
19:11:57 <jugglinmike> Matt_King: And Admins want to see what needs to be done to move each to the next step in the process
19:12:18 <jugglinmike> Matt_King: Right now, you can see there are 35 test plans that exist in the repository
19:12:44 <jugglinmike> Matt_King: A test plan can have more than one version active at the same time. That was previously very hard to see; this interface makes it much more clear
19:13:03 <jugglinmike> Matt_King: The page allows you to filter Test Plan Versions and narrow down
19:15:10 <jugglinmike> Matt_King: Right now, all Test Plan Versions in either "R&D Complete" or "In Candidate Review". In the future, when we begin deprecating Test Plan Versions, that will be apparent, too
19:18:24 <jugglinmike> Matt_King: I'm really excited about this table. To me, it makes it really clear what's going on in the project.
19:18:56 <jugglinmike> Matt_King: The very first column, the "test plan" column, is a link. If you open one of those links, it opens a page that shows you information about all versions of that test plan that were ever created
19:19:07 <jugglinmike> Matt_King: ...Here, you can see information about the plan at a high level
19:19:50 <jugglinmike> Matt_King: ...Next, you have a version summary table.
19:20:00 <jugglinmike> Matt_King: ...Next, you have a list of all related GitHub issues.
19:20:23 <jugglinmike> Matt_King: ...Then there's a timeline for all versions. This shows the chronological order of how a Test Plan was developed
19:20:47 <jugglinmike> Matt_King: ...Then there's a series of sections which detail the history of each version
19:21:19 <jugglinmike> Matt_King: This is the "Test Plan Version History" page. It's an awesome piece of work. I love this page
19:22:01 <jugglinmike> Matt_King: Back on the "Test Plans Status Summary" table, the next column describes the "Covered AT"
19:24:10 <jugglinmike> Matt_King: Further along, in the "Candidate Review" column, there is a button labeled "Required Reports:". This opens a dialog which surfaces missing reports, if any
19:24:50 <jugglinmike> Matt_King: In that same cell, you can see the number of relevant open issues on GitHub (in the case of "Action Button Menu Example", there are three)
19:25:35 <jugglinmike> Matt_King: And it shows the target date for completing Candidate Review (for admins, this is rendered as a button which allows them to modify the target date)
19:26:36 <jugglinmike> Matt_King: The final column is for each Test Plan's "Recommended" version. We don't have any Test Plans with a version in the "recommended" phase right now, so there's nothing to see here at the moment
19:27:09 <jugglinmike> Matt_King: So that's the "Data Management" page!
19:27:18 <jugglinmike> Matt_King: The other changes in the app are largely related to version control
19:27:24 <jugglinmike> Matt_King: There are still more on the way
19:28:07 <jugglinmike> Matt_King: The goal is that the providence of any information in the system is obvious no matter where you are
19:28:27 <jugglinmike> Matt_King: It's about transparency for the big picture, and giving admins the ability to manage the working mode
19:28:44 <jugglinmike> Matt_King: Big thanks to the folks at Bocoup for making this happen. This is a big turn of the crank
19:28:54 <jugglinmike> present+ Michael_Fairchild
19:29:06 <jugglinmike> Michael_Fairchild: This is huge--thanks!
19:29:28 <jugglinmike> Matt_King: There are ten known bugs right now. I've put a link to those in the agenda. Some will be fixed in the next deployment
19:29:43 <jugglinmike> Matt_King: The next deployment is tentatively planned for Thursday of next week
19:29:56 <jugglinmike> Matt_King: One of the big changes coming in that deployment is a change to how we present reports
19:30:26 <jugglinmike> Matt_King: We'll spend some time walking through that in our meeting which follows that deployment, on October 19
19:30:41 <jugglinmike> Topic: Issue 809 - Define required AT versions for reporting on recommended test plans
19:30:58 <jugglinmike> github: https://github.com/w3c/aria-at-app/issues/809
19:31:18 <jugglinmike> Matt_King: Last week, we talked about how when a test plan becomes recommended, that's when we want to start making sure that the data for the test plan remains current
19:31:37 <jugglinmike> Matt_King: That way, visitors to APG can get information about how the latest version of their screen reader is working with that test plan
19:32:16 <jugglinmike> Matt_King: We don't have the ability to show reporting history and trends on the site (that's something we'll talk about next year)
19:32:34 <jugglinmike> Matt_King: Last week, we decided that we can't keep up with doing that for doing that for every new browser version and every new AT version
19:32:44 <jugglinmike> Matt_King: We decided we would only keep up with new releases of AT
19:33:22 <jugglinmike> Matt_King: So I'm thinking about how we determined when/if AT versions are missing
19:33:25 <jugglinmike> Matt_King: In this issue, I'm proposing three possiblities as to decisions that we could make
19:33:51 <jugglinmike> Matt_King: There may be more options, I just wanted to have a starting point for a decision framework
19:34:19 <jugglinmike> Matt_King: The three options that I came up with are...
19:34:46 <jugglinmike> Matt_King: Suppose we marked a test plan as recommended today, let's limit our discussion to NVDA.
19:35:21 <jugglinmike> Matt_King: There are three options for "which versions of NVDA should have required reports?"
19:36:09 <jugglinmike> Matt_King: One option: since we first started with NVDA in 2022 (let's say), we should generate data for every version since then. (The rational being that we started collecting for that version, and we don't want gaps in the data)
19:37:16 <jugglinmike> Another option: We use whatever the latest version of the test plan that we have
19:37:39 <jugglinmike> A third option: We only consider new versions released after the moment the report became recommended
19:38:33 <jugglinmike> In summary: (1) the version used to generate the earliest approved report, (2) the version used to generate the latest approved report, or (3) the first version released after the test plan becomes recommended
19:38:47 <jugglinmike> present+ Hadi
19:39:06 <jugglinmike> Hadi: I'm having trouble following; I think seeing it in writing would help
19:39:18 <jugglinmike> Matt_King: I wrote this up in issue gh-809
19:39:26 <jugglinmike> Hadi: Great, thank you
19:40:01 <jugglinmike> Matt_King: I don't necessarily expect to make a decision today, but I definitely didn't want to make the call on my own. This effects the folks here and various external stakeholders
19:40:26 <jugglinmike> s/Another option:/Matt_King: Another option:/
19:40:38 <jugglinmike> s/A third option/Matt_King: A third option/
19:40:57 <jugglinmike> s/In summary:/Matt_King: In summary:/
19:42:15 <jugglinmike> present+ James_Scholes
19:42:41 <jugglinmike> James_Scholes: I think we need to explicitly define what we mean by "recent" when it comes to browser releases and AT releases. We shouldn't ask Testers to make a judgement call about that
19:42:48 <jugglinmike> Matt_King: Sounds good
19:42:49 <jugglinmike> Matt_King:
19:44:06 <jugglinmike> James_Scholes: I think the word should be interpreted in terms of the time that the Tester is done
19:45:45 <jugglinmike> Matt_King: Knowing the current major version of stable browsers will take some work
19:46:20 <jugglinmike> Matt_King: If we have this requirement, can the app figure it out, or will it be a human job?
19:46:54 <jugglinmike> howard-e: I think it might have to be a human job because otherwise, we may make false assumptions
19:47:35 <jugglinmike> Matt_King: Can the app use an API (e.g. provided by a browser vendor) to discover new versions?
19:47:54 <jugglinmike> James_Scholes: It has to be possible to some extent because Playwright knows how to download the latest browser
19:48:14 <jugglinmike> Matt_King: Do we know if we can do that for Firefox, Chrome, Safari, and Edge?
19:48:31 <jugglinmike> howard-e: I don't know for all of those. Mostly likely yes for Firefox and Chrome
19:49:00 <jugglinmike> James_Scholes: Edge may be difficult because of the way Edge is pushed out
19:49:38 <jugglinmike> Matt_King: I'll open an issue to perform an investigation into the feasibility
19:50:19 <jugglinmike> James_Scholes: We could do this together on a weekly cadence
19:51:11 <jugglinmike> James_Scholes: But we shouldn't ask someone to do this more often, so we'll still be in a position where we don't know for sure that we've identified the latest version available
19:56:12 <jugglinmike> jugglinmike: Recall that last week, we talked about setting policies in terms of release date, e.g. "any version released in the past 6 months"
19:58:50 <jugglinmike> jugglinmike: So the data on release that we collect (whether via an automated process or a manual one), should probably include both the version numbers and date
20:01:19 <jugglinmike> jugglinmike: We don't have to only consider the release of software in the time window
20:03:21 <jugglinmike> jugglinmike: Instead of saying "the version released in the past 90 days, or, missing that, the next version released"...
20:03:45 <jugglinmike> jugglinmike: ...we could say "the newest version available in the past 90 days" (meaning that even if there was no release, we would accept the version that Testers would plausibly have installed during that time frame
20:03:50 <jugglinmike> Zakim, end the meeting
20:03:50 <Zakim> As of this point the attendees have been Matt_King, howard-e, jugglinmike, Michael_Fairchild, Hadi, James_Scholes
20:03:53 <Zakim> RRSAgent, please draft minutes
20:03:54 <RRSAgent> I have made the request to generate https://www.w3.org/2023/10/05-aria-at-minutes.html Zakim
20:04:00 <Zakim> I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent.  Goodbye
20:04:01 <Zakim> Zakim has left #aria-at
20:04:05 <jugglinmike> RRSAgent, leave
20:04:05 <RRSAgent> I see no action items