Meeting minutes
Merging in changes without throwing out data
Jes: have working prototype for a feature related to merging in new test plans
You can pull new version of test plan
and decide what to do with report.
Link to sandbox of new experience:
<jesdaigle> https://
Galk through experience of how new test plan versions are merged into existing report; it is in sandbox and in a prototype state
Jes: next step for Lewis and Isaac to review and test
then make adjustments.
Jes: The way the plans are structured, with a plan with tests inside of it, would it make sense to separate the test plans from the test plan run?
<jongunderson> https://
mk: Jes can you raise an issue where we document the logic used to determine which tests data is copied to the new run?
Jes: yes
mk: So, the way the works is the test que shows a run of a test plan for specific AT/Browser. We can update teh version of the plan for that run, which essentially makes a new test run for that browser/AT and copies data from old run to new run. Optionally, the test admin can delete the old run.
Jes: yes
mk: It is the logic for copying data from old run to new run that we need to dive into. We want to do it in a way where test admin can make decisions to override the default rules that might prevent copying results from test X in old run to text X in new run.
The default rules might say that text X haschanged in a breaking way, but the test admin may know that it is not a breaking change.
We need to consider whether that new run should be flagged with a different status if the admin overrode logic to make sure the decision is verified?
We don't want test admins to inadvertantly introduce errors into the results.
Jes: we also need to discuss whether or not we will truly delete or only soft delete
mk: very good question
Boaz: will discuss with Seth and bring some considerations to table
Current test runs
mk: looking at select-only combo for FF/NVDA. Alyssa and Louis results have 9 conflicts.
Walked through how to either correct results or raise issue to ask a different person to correct results.
Alyssa can start working on those conflicts after Mar 7
Jon: started working on test guide
https: //
https://
Discussed when to mark the unexpected output box.
Jon: we don't have criteria for when to check box
for unexpected output.
Joe marked it for when NVDA said clickable on combobox.
Joe: we previously discussed and agreed that is actually an expected, default behavior for NVDA in certain circumstances.
I'll change my results where they conflict for taht reason.
What about where NVDA says list instead of listbox?
mk: Let's cover in next meeting.