Meeting minutes
Closing out testing of plans for APG support tables
<Matt_King> Michael: Get email about Dillon helping with testing
<Matt_King> James: dillon has done some sample testing.
<Matt_King> MJ also expressed interest
<Matt_King> Michael: MJ moved jobs outside Deque but may investigate how to continue working with project
<Matt_King> James: PAC closes tomorrow, matt can you you do some admin next week?
Matt: Yes I can, Tuesday through Thursday of next week I can work on this project
Matt: Link and Command Button would be the two we can use JAWS 2022 results and put them in a APG table
Matt: Those should be ready to publish by EOD tomorrow
Matt: We will also update results for Toggle button and Alert, for Voiceover and NVDA
Matt: Alyassa and Myself, maybe Louis can work on testing Alert, Toggle Button, and Radio Group for Jaws 2022
Matt: We can test those three next week
James: The only testing that is incomplete is radio group on iOS, which could be that Louis hasn't gotten to it, I can send an email.
James: There are a couple of conflicts we will deal with tomorrow, then it should be in a clean state
Matt: Lets look at the Queue now
Matt: Are there any conflicts in link or command button?
James: No there are no conflicts in either of those
Matt: Alyssa thank for completing your testing, and there are no conflicts there
Matt: Alert for NVDA is also complete
Matt: For Voiceover, no conflicts for alert thats ready to go, for command button there is no conflicts and its complete, and link that has no conflicts.
Matt_King: When JAWS update comes out, Alyssa, Louis and I need to test Alert, Command Button, and Link
Matt_King: So for sure, we are well positioned for button and link, everything is done.
Matt_King: We could also do alert if we can get two people to test next week
Matt_King: Alyssa can you help testing?
Alyssa: No next week I am unavailable next week, but I am available the week of the 26th
James: Please be sure to use JAWS 2023
Matt_King: Seth can Louis help out testing JAWS next week?
Seth: I think he was waiting to see when the new release came out to scheudle the work. I think I just need to have that conversation with him now that we know when that will be
Matt_King: Link and Alert button don't require much testing, so if he could focus on those two first that would be great.
Matt_King: The JAWS December release should be early next week, probably Tuesday
James: For Louis's testing is he testing with 2023?
Matt_King: It would be great to test Alert Command button, link, and toggle button with JAWS so that we have results for all AT
James: There are some conflicts for Toggle Button, we should be able to resolve them
Matt_King: If Louis can test these, then Alyssa we wont need you to do any testing
Seth: James will you follow up with Louis regarding these notes?
James: Yes I will follow up
James: For the new year, we will have Dillion and a new person at PAC. They can start in January
Matt_King: The plan now is for Louis and I to do the testing with JAWS next week
Matt_King: As of this moment, if we did not more testing, we can send the Preview of the APG support tables for link and command button to vispero
Process for publishing re-tested plans
Matt_King: Lets start with the mechanics for the ARIA AT App
Matt_King: So we have these buttons in the table, labeled "Mark as in review" What does that button do?
Howard: In Prod now if you press Mark as in review, then that button becomes Mark as finalized, which when pressed would put the test plan it in final reports. But there are changes to this now that we added language for recommended
Matt_King: If I press that button now for link and command button what will happen?
Howard: It will just mark it as in review
James: I just fixed the conflicts on toggle button
Matt_King: I will try to press this button on NVDA and Chrome on the command button plan
Matt_King: Did it do anything?
Matt_King: It reloads the page, updating test plan status
James: I don't see any change yet
Matt_King: Now it says in review, mark as candidate
Matt_King: I'm a little confused if i pressed the wrong one
Matt_King: Now it says, mark as candidate
James: Lets establish what will happen when we press that
Matt_King: We changed it from draft to in review, did that just change visibility of the status on test queue page, but didn't change anything on the results page? Yes
Matt_King: Right now there is a NVDA Command button row in the candidate test queue already
Matt_King: If we go to the candidate tests, and go to NVDA
Matt_King: I dont see anything about the date of the test
James: No that page doesn't show any data about the test plan
Matt_King: So this currently says "Review status on candiate page is ready for review"
Matt_King: We need to discuss how we want this to work
James: Lets take command button, and say we want to publish the report for NVDA what do we want to happen? Right now it will add a row to the reports page, but they will both say Toggle button
Matt_King: We would ideally have access to all the test data
James: So we wont replace the data, we want to add the data with the new version to the run history. Is it then codified somewhere that the support level will be calculate only based on the latest version of a Screen reader?
James: If the support level is based on a amlagam of the versions than that will be inaccurate.
Howard: Yes confirming the report data will be based on the latest data
Howard: So for clarity, we are working on displaying the data in one row on the reports page
Howard: However, to capture the run history, we need to work on collecting and displaying this data
James: The run history is currently being scoped to the report, we want it on the test plan
Matt_King: Based on our website design, we want to surface the latest data only
Matt_King: We will need to change the website to be able to show test results from earlier versions of the screen reader or browser
Matt_King: So right now, add additional rows becomes problematic. We just want to surface the latest datea
Matt_King: Our website isn't designed to drill into previous results
Seth: There are two pieces in transition, that required to be revisited, one is how do browse reports across time, the other one is in a slightly differnet level of support depending on where in the app, is allow the review determinations to be made on the test plan, divorced from the res'ults
Seth: We are currently unbundling the test plan and results
Seth: Next year, alot of this will become natural when we shift of view of plans and data
Matt_King: So for now, if we press the mark as candidate it will add another row to the reports table, then Bocoup would manually change what is displayed onm the reports page and results page?
Matt_King: I am concerned about having multiple rows for JAWS, if we update alert and have two rows on the results page that will be confusing for Roxanna
Howard: So can we manual change the table? Yes we can edit to have a single slice of data on both reports
Seth: One thing I want to say is Howard has been working on the reports page to seem like there is no duplicate rows, Matt you are asking can we do the same thing o the candiate page, I think its possible but its a little more complicated
Seth: Howard and I can work on this
Matt_King: So we want to do this for Button and Link, sounds like the more we do, the more work there is to make it seem like things are the way they are
James: I'm also concerned that the publication dates are represented on all pages. I'm also concerned that if we make these changes while people are on holiday, we will all loose track of what has been published and not
Matt_King: Any manual changes we make should be in a github issue
Matt_King: Lets use Link, its in candidate test page for all 3 SR and in the reports page. It sounds like another option is to make it appear as there is no problems. If we do this we will have two rows on canidate page, I guess we can figure out what the consequences are in the next year
Matt_King: there will not be anyone reviewing this until sometime in January
Matt_King: So the risk of having duplicate data is zero
Seth: Also there is no pubic view of candiate test currently
James: Seth you mentioned you worked around this for the reports page, what will happen when you press that button now?
Howard: Its a change we have been working this week, we need to still deploy it
James: How would the feature work?
Howard: A new report would get added to candiate test, along with the recommended status, it will compare its self with other test plans, and if its at a later date it wil override it
James: What happens to that old data?
Howard: It still exists, its just not reachable on the webiste
James: Does it change the dates?
Howard: When it gets promoted it will have a new date
Matt_King: There will be two rows remember
James: So theoretically, when we publish a new report, the newer test result will display
Matt_King: There is some confusion here about versions
Matt: We have test plan version 1 t1 and screen reader version 1 v1, if they currently show 96% on the reports page, if the next step is we have Test 2 T2 and Screen reader V2, that the results of Test 2 and Version 2 will display.
Howard: Right now the version of the AT is not being considered, just the test plan date
Matt_King: How does the current feature you are working on work
Howard: It doesn't take the AT Version into account
Howard: It just takes the latest version
Howard: What ever the newest date is takes precedence
Matt_King: We might want to investigate that logic, it will work for now
James: If the logic is based on whenever you republish a test plan the candiate date will change, we don't want it to reset the beginning of the candidate phase
James: We also don't want to retest the plan when we republish it
Seth: Yeah this is just a quick work around, we can revisit it in the new year
Matt_King: when do we plan to have this change in prod?
Howard: Early next week
Matt_King: I wont push any buttons before then
Matt_King: Seth we will need tight communication
Matt_King: who will be available next week to work with me to make sure we get the support table out?
Seth: Alex is working next week, Howard is working Monday through Thursday, I think we should start a email thread and use that as a channel
Matt_King: Okay that sounds good to me
Seth: We are having some challenges what the status or issue of the deployment process
Seth: I asked for clarification on that threa
Seth: As we get the work done, you might want to try a different channel with Shawn and Michael
Seth: We could help, but we haven't been given any info on the issue, thats causing the delay
Seth: You could wait til next week after this deploy
Matt_King: I was thinking to get an update this afternoon
Matt_King: This is also a potential blocking issue to publishing