19:57:33 <RRSAgent> RRSAgent has joined #aria-at
19:57:37 <RRSAgent> logging to https://www.w3.org/2025/01/23-aria-at-irc
19:57:37 <Zakim> RRSAgent, make logs Public
19:57:38 <Zakim> please title this meeting ("meeting: ..."), Matt_King
19:58:03 <Matt_King> MEETING: ARIA and Assistive Technologies Community Group
20:00:00 <jugglinmike> jugglinmike has joined #aria-at
20:01:20 <jugglinmike> present+ jugglinmike
20:01:22 <jugglinmike> scribe+ jugglinmike
20:01:34 <Joe_Humbert> present+
20:01:46 <jugglinmike> meeting: ARIA and Assistive Technologies Community Group Weekly Teleconference
20:01:56 <mmoss> mmoss has joined #aria-at
20:02:00 <jugglinmike> present+ Matt_King
20:02:09 <mmoss> present+
20:02:09 <ChrisCuellar> ChrisCuellar has joined #aria-at
20:02:25 <jugglinmike> present+ Joe_Humbert
20:02:32 <jugglinmike> present+ ChrisCuellar
20:02:42 <jugglinmike> present+ dean
20:02:53 <jugglinmike> present+ IsaDC
20:02:58 <jugglinmike> present+ JamesScholes
20:03:14 <jugglinmike> present+ RichardSteinberg
20:03:47 <jugglinmike> Topic: Review agenda and next meeting dates
20:03:50 <howard-e> howard-e has joined #aria-at
20:05:14 <jugglinmike> Matt_King: Requests for changes to agenda?
20:05:30 <howard-e> present+
20:06:09 <jugglinmike> mmoss: There was an issue that I filed to capture the agenda item request from two weeks ago. The one related to some of the results for the navigation menu example with results pertinent to Vispero and Apple. Issue number 1174
20:06:36 <jugglinmike> Matt_King: Ah, yes. Let's add that
20:07:20 <jugglinmike> present+ MichaelFairchild
20:08:16 <jugglinmike> Matt_King: Any other changes?
20:08:24 <jugglinmike> Matt_King: Hearing none, we'll stick with what we've got
20:08:32 <jugglinmike> Matt_King: Next Community Group Meeting: Wednesday January 29
20:08:38 <jugglinmike> Matt_King: Next AT Driver Subgroup meeting: Monday February 10
20:08:44 <jugglinmike> Topic: Current status
20:09:21 <jugglinmike> Matt_King: We have not set a goal for mid-year, yet
20:09:42 <jugglinmike> Matt_King: I typically set goals for every six months internally here at Meta, and I tie these public goals to that
20:10:15 <jugglinmike> Matt_King: We had a really aggressive goal for last year. I think we still need to be aggressive this year, to show progress. But I want to set something more realistic at least for the first half of this year
20:10:34 <jugglinmike> Matt_King: Of course, we have a lot of dependencies on people beyond this group when it comes to reaching those goals, but that's the nature of the project
20:10:59 <jugglinmike> Matt_King: This week, we had a meeting with Vispero and reviewed some plans that they haven't approved, yet. A couple of those are blocked by us right now. One is blocked by them
20:11:31 <jugglinmike> Matt_King: On our side, we're making changes to the "radio group" plan, and we have some issues to close in the three "menu button" plans that are in there (two of those, I'm going to get on the agenda for next week)
20:12:29 <jugglinmike> Matt_King: A dependency we have on Vispero is related to automation. As you all know, we have automation facilities for NVDA and VoiceOver, but not for JAWS. That's made it hard for us to keep the plans for JAWS up to date
20:12:54 <jugglinmike> Matt_King: Vispero have made a lot of improvements recently, though, and they'd like to see those improvements reflected in the plan
20:13:18 <jugglinmike> Matt_King: They may approve the plans as-is if their support for automation gets delayed further than they expect
20:14:01 <jugglinmike> Matt_King: That's the current status on what's in Candidate Review. I don't have any updates from Apple at this time
20:14:33 <jugglinmike> Joe_Humbert: Have you been able to secure a commitment from Apple, NVAccess, Freedom Scientific, etc. about a more timely review?
20:14:53 <jugglinmike> Joe_Humbert: A lot of us are volunteering our time to get things done, and it sits idle while we wait for those folks to do their part
20:15:21 <jugglinmike> Matt_King: I haven't been very aggressive about that. I would say some of that is more about me than it is about them. I've been struggling to make sure we're always meeting our end of the bargain on time
20:15:46 <jugglinmike> Matt_King: When they have feedback on things, and we have to make changes to test plans and run them through the review process again--sometimes the delays in that part have been because of me
20:16:13 <jugglinmike> Matt_King: I want to get better expectations, process, and more discipline around this. Part of that has really been dependent on me being able to dedicate more time
20:16:26 <jugglinmike> Matt_King: We've been making staff changes around here, so I'm hopeful that I'll be able to do that
20:16:44 <jugglinmike> Matt_King: 2024 was really hectic, but I think we have most of the technical barriers behind us
20:16:56 <jugglinmike> Matt_King: I hope to finally establish a better rhythm this year
20:17:33 <jugglinmike> Joe_Humbert: Thanks. I understand that there are things that are on us. Often times, during the meetings that I'm able to attend, I hear that we're waiting on Vispero or on Apple. It would be nice if they had a commitment to get things done in a timely manner
20:17:50 <jugglinmike> Matt_King: I think we should revisit this conversation at least once a quarter, and I'll try to take more accountability for making that happen
20:18:13 <jugglinmike> Matt_King: If there are people (especially the vendors) that have thoughts on this, we can talk more offline
20:18:22 <jugglinmike> Topic: Testing for radio group
20:18:50 <jugglinmike> Matt_King: The issue Joe_Humbert raised related to the group label getting hidden--we now realize that has effected not just the current radio plan but also the other one
20:19:11 <jugglinmike> Matt_King: We made a list of changes that JamesScholes and IsaDC are working on. Hopefully we will get new versions of radio test plans ready by next week
20:19:16 <jugglinmike> Matt_King: From there, we can continue testing
20:19:44 <jugglinmike> Matt_King: Hopefully we'll be able to preserve a decent amount of test results of the plan that's already in Candidate Review, but due to the nature of the changes, we may lose quite a few results
20:20:32 <jugglinmike> Joe_Humbert: I skipped the ones where I could visually see the label was getting hidden. In those tests, especially in VoiceOver, it would ask you to do a key command twice, and would certainly put you in the wrong position. I skipped those during my testing. The rest of the tests, I did
20:20:50 <jugglinmike> Matt_King: That's it for radio. Hopefully we'll have more to say on this next week
20:20:57 <jugglinmike>  Topic: Testing for remaining link plans
20:21:07 <jugglinmike> Matt_King: I was shocked to find so many conflicts here
20:21:27 <jugglinmike> Matt_King: It turns out to be an NVDA behavior where it reads a whole bunch of elements as being on the same line
20:21:40 <jugglinmike> Matt_King: That's a default behavior which oddly did not impact link example 1
20:22:05 <jugglinmike> Matt_King: This is a side-effect of how the "navigate forward/backward from here" links are added in to the DOM
20:22:17 <jugglinmike> Matt_King: All of those side-effects would go away if we were to change the test case, so I filed an issue for that
20:22:46 <jugglinmike> Matt_King: My assumption is that, at minimum, if we put a paragraph tag or a "<BR>", that would separate them and prevent the side effect
20:23:27 <jugglinmike> IsaDC: I made the changes since December for those, but the app did not update because the changes were in the examples but not the CSV files
20:23:48 <jugglinmike> JamesScholes: In the APG, these are all under one page in a table. The test cases we are using are very different
20:24:15 <jugglinmike> JamesScholes: The first is a "span" element. The second is an "img"
20:24:53 <jugglinmike> JamesScholes: separately, the app doesn't register updated test plan versions when you change the example (as opposed to when you change the CSV files)
20:25:12 <jugglinmike> Joe_Humbert: They can see the changes uploaded into the system, but the system isn't recognizing that it needs to update the code
20:25:23 <jugglinmike> Matt_King: Ah, but the app did not import the new reference file
20:25:28 <jugglinmike> IsaDC: Yes, I reported it already
20:26:13 <jugglinmike> howard-e: The problem here is that with the reference files, they are really just pointed to. The contents are not actually "diffed." I suggested that if there is a change to the reference file, the reference folder's timestamp should be updated as well
20:26:52 <jugglinmike> howard-e: We can also track the file contents directly. I suggest creating a new issue to manage that feature addition
20:27:21 <jugglinmike> Matt_King: We don't have to modify the timestamp, right? We can just add a suffix because that name is referenced from references.csv
20:27:52 <jugglinmike> Matt_King: If that works for you, IsaDC, then I would rather do that then spend more engineering time on this
20:28:17 <jugglinmike> IsaDC: The issue here is that there is not consistent updates. Because I rebuilt the whole test plan, it should include the new files
20:28:51 <jugglinmike> JamesScholes: I would encourage us to be a bit more intentional. Matt_King is right that the date indicates when we pulled in the example. It doesn't say when the example in the APG was updated, though.
20:29:11 <jugglinmike> JamesScholes: Rather than adding something that's vague and easy to overlook, I would support updating the time stamp
20:29:22 <jugglinmike> Matt_King: Yeah, because that import date really isn't that important
20:29:45 <jugglinmike> JamesScholes: It allows us to say, "well, we last imported this on this date"
20:29:56 <jugglinmike> IsaDC: If we change it now, then we'd be breaking that pattern
20:30:07 <jugglinmike> JamesScholes: So it's up to us if we want to change the meaning of that date
20:30:43 <jugglinmike> Matt_King: I'm changing my thinking, I think. The Git history is keeping track of our modifications. We should rely on the Git history. And we do have a commit that has a hash which says, "hey, this file changed"
20:31:25 <jugglinmike> Matt_King: When you say we "reference" the files but we don't "import" them--is the URL pointing directly to a file in GitHub?
20:31:49 <jugglinmike> howard-e: Yes, it would be pointing to a file in GitHub. It would be pointing to a commit
20:32:00 <jugglinmike> JamesScholes: So in this case, the app doesn't know to change the hash
20:32:12 <jugglinmike> JamesScholes: how do you know to update the hash for CSV files?
20:32:25 <jugglinmike> howard-e: Pretty much any change to the test folder that isn't "reference folder only" will force an update
20:33:20 <jugglinmike> JamesScholes: Could you read every file under the reference directory and hash it combined together?
20:33:50 <jugglinmike> howard-e: Creating a hash of these reference files seems like the easiest way to go. I think there are a couple things to explore here, and it likely involves hashing
20:34:20 <jugglinmike> Matt_King: For this particular occurrence, if IsaDC makes a copy of the timestamped reference folder. Just copy it and put something in its name that indicates a change...
20:34:37 <jugglinmike> JamesScholes: Could we change something elsewhere in the test plan in a way that is not significant
20:34:47 <jugglinmike> Matt_King: Perhaps the reference row in the reference CSV
20:35:05 <jugglinmike> JamesScholes: could we change something in the setup script, perhaps?
20:35:35 <jugglinmike> Matt_King: A copy of the reference folder with a slightly different name would address our problem right now. You would change the path in the "reference" row or the references.csv file
20:35:48 <jugglinmike> Matt_King: We could do that this one time, and howard-e could investigate a better solution for the future
20:36:09 <jugglinmike> IsaDC: if we are going to make these changes, could howard-e make them? I think that would be faster
20:36:16 <jugglinmike> JamesScholes: I can make them
20:36:39 <jugglinmike> JamesScholes: could we rename the whole directly?
20:37:00 <jugglinmike> howard-e: It would think of it as a new plan
20:37:24 <jugglinmike> JamesScholes: Have we made these updates to all of these plans?
20:37:38 <jugglinmike> Matt_King: The first one doesn't need to be changed because it doesn't exhibit the problem
20:37:53 <jugglinmike> Matt_King: There are not changes required to the HTML. The HTML changes have already been made
20:39:24 <jugglinmike> Matt_King: [reiterates steps for JamesScholes]
20:40:22 <jugglinmike> Matt_King: The net effect for people who are running this (I think it was Joe and Dean), the bot will have to re-evaluate, and then Testers can go in and review the responses
20:40:35 <jugglinmike> JamesScholes: do you care if its a rename or a copy?
20:40:56 <jugglinmike> JamesScholes: Git will follow change history through renames, but it will consider copied files to be completely new
20:41:30 <jugglinmike> JamesScholes: Currently, they all say "link.html". What if I rename them to "link_css.html" and "link_whatever.html"?
20:41:41 <jugglinmike> Matt_King: I like that. Git will follow changes across renames well
20:41:49 <jugglinmike> JamesScholes: Then that's what I'll do
20:42:02 <jugglinmike> Topic: issue 1174 on disclosure nav testing
20:42:09 <jugglinmike> github: https://github.com/w3c/aria-at/issues/1174
20:42:26 <jugglinmike> mmoss: The results appear to be stale at this point
20:42:43 <jugglinmike> mmoss: With current macOS 15.1.1, we're no longer getting the expanded attribute state change when it happens
20:43:07 <jugglinmike> mmoss: What is the process? Would we let the current results move through the process and eventually make their way to the APG site and then do a new round of testing?
20:43:13 <jugglinmike> Matt_King: This is related to the next topic
20:43:35 <jugglinmike> Matt_King: The report that we have out there right now... What is the Candidate Review status of these? Has either company approved them?
20:43:45 <jugglinmike> Matt_King: Actually, we have a discussion with Vispero ongoing about this one
20:43:57 <jugglinmike> Matt_King: Vispero has recognized some bugs in their behavior, and they are planning on making some changes
20:44:13 <jugglinmike> Matt_King: However, the specific thing that you're talking about was a VoiceOver behavior related to ARIA-expanded
20:44:24 <jugglinmike> mmoss: Right, some of the core functionality--the "musts" in the test plan
20:44:38 <jugglinmike> mmoss: At the time I tested it, they were met, but now, they are not met
20:44:43 <jugglinmike> Matt_King: So VoiceOver has regressed
20:44:50 <jugglinmike> mmoss: That's correct
20:45:23 <jugglinmike> Matt_King: So we could re-run this test plan for VoiceOver and update that. I think that would be a good thing to do before having more discussion with Apple on this plan. I don't think we've even touched on this plan with Apple
20:45:44 <jugglinmike> Matt_King: I believe we can just re-add this to the test queue for just VoiceOver.
20:45:57 <jugglinmike> Matt_King: We could replace that report by running with a later version of VoiceOver
20:46:48 <jugglinmike> Matt_King: Ideally (related to the next agenda item), when a new version of VoiceOver comes out, we will automatically re-run all Candidate and Recommended test plans and re-generate those results. We shouldn't have that problem when we have that capability
20:47:00 <jugglinmike> mmoss: I'm happy to run through the test plan again
20:47:09 <jugglinmike> Matt_King: Okay. We can add this to the test queue right now
20:48:05 <jugglinmike> IsaDC: I'll re-add the "disclosure navigation menu" test plan to the test queue, just for VoiceOver
20:48:13 <jugglinmike> Matt_King: Then we will re-run it
20:48:34 <jugglinmike> Matt_King: That's a good observation, mmoss. It's a bummer that there's been a regression
20:48:44 <jugglinmike> Topic: System behavior after adding new AT versions
20:50:09 <jugglinmike> github: https://github.com/w3c/aria-at-app/issues/1162
20:50:19 <jugglinmike> present+ stalgia
20:50:42 <jugglinmike> stalgia: This feature has to do with refreshing reports for test plans that have a report completed for a previous AT version when a new AT version is added to the system
20:51:06 <jugglinmike> stalgia: We thought the best way to approach this is iteratively with a low-impact and minor extension to the interface
20:51:37 <jugglinmike> stalgia: We're thinking that when a new AT version is added to the app, there would be an additional element that appears whenever the AT is viewed in the "manage AT versions" disclosure
20:52:04 <jugglinmike> stalgia: that would allow the user to start testing with that version and tell them how many runs that would involve
20:52:23 <jugglinmike> stalgia: We wanted to give admins some control over when that whole slew of test plan runs will be created
20:53:01 <jugglinmike> Matt_King: I was thinking. In the "data management", we have "report status". The button that opens the dialog will say something like "required reports complete" or something of that nature. Or "not complete" or "in progress"
20:53:14 <jugglinmike> Matt_King: I'm wondering if that's where we might want to indicate that this is in the data manage
20:53:25 <jugglinmike> Matt_King: If you opened that dialog, you would see rows
20:53:30 <jugglinmike> s/manage/management/
20:53:57 <jugglinmike> Matt_King: (There is one thing about that table that I am finding a little difficult to use. That is: we don't group all the required rows together.)
20:54:16 <jugglinmike> Matt_King: I'm wondering if that might be more consistent with our current approach to managing data and reports
20:55:13 <jugglinmike> Matt_King: In the "Candidate Review" column, in the very first row, right now, for "action menu button", there's an "advance to recommended" button and then a "Required reports complete" button
20:55:55 <jugglinmike> Matt_King: I'm wondering if, when a new version is added, that could be changed to something like, "updates to required reports pending"
20:56:54 <jugglinmike> Matt_King: . And then when yo u open the dialog, we have a table. We could add rows to the table. It's candidate, so it says, "or later". We could add a row with a specific NVDA version in the "AT" column (e.g. NVDA 2024.4.4 or whatever), and then in the "report" column (the far right column), it could say "update pending"...
20:57:16 <jugglinmike> Matt_King: I guess the problem with this approach, though, is that you'd have to press a lot of buttons.
20:57:27 <jugglinmike> stalgia: Right. that was the thinking behind a centralized action
20:57:41 <jugglinmike> Matt_King: What if it appeared just above the data management table for the administrators?
20:58:09 <jugglinmike> Matt_King: It would be nice if it was a heading that reflected both the number of reports and the screen reader
20:58:18 <jugglinmike> Matt_King: If there's just one button to run them all, that would be greast
20:58:23 <jugglinmike> s/greast/great/
20:58:32 <jugglinmike> stalgia: we can get to work on a proposal
20:59:18 <jugglinmike> Matt_King: If some output is different than it was before, we'll need humans to intervene. I believe those should be automatically added to the test queue, and I think we have all of what we need in the test queue for that. My expectation is that anything requiring human review ends up in the test queue
20:59:35 <jugglinmike> stalgia: I think the system already has this notion of tracking historical verdicts like this
21:00:00 <jugglinmike> Matt_King: How would we know when it's complete? Would we have something else on the data management page? Should we have a log of what happened below that table?
21:00:19 <jugglinmike> stalgia: That's an interesting question. We can think on it and include a suggestion in our proposal
21:00:41 <jugglinmike> Zakim, end the meeting
21:00:41 <Zakim> As of this point the attendees have been jugglinmike, Joe_Humbert, Matt_King, mmoss, ChrisCuellar, dean, IsaDC, JamesScholes, RichardSteinberg, howard-e, MichaelFairchild, stalgia
21:00:45 <Zakim> RRSAgent, please draft minutes
21:00:46 <RRSAgent> I have made the request to generate https://www.w3.org/2025/01/23-aria-at-minutes.html Zakim
21:00:53 <Zakim> I am happy to have been of service, jugglinmike; please remember to excuse RRSAgent.  Goodbye
21:00:53 <Zakim> Zakim has left #aria-at
21:01:02 <jugglinmike> RRSAgent, leave
21:01:02 <RRSAgent> I see no action items