Meeting minutes
App Development Update: Candidate Review queue
Seth: Erica almost ready for review on the PR to add the CR queue
Howard now focusing fully on performance.
Identified a few issues, added comment on issue raised by James Criag.
App hitting a memory limit, which may opena conversation about the hosting environment.
Parsing error is a side effect due to timing/out of memory.
App is currently getting 1G of RAM.
Pretty low for a prod app.
Going to see if we can get limit raised.
We are also looking at mitigations in code.
Looking at performing some of the more intensive calculations off line.
Bocoup realizes it is preventing work.
James: Wonder if we want to host the app elsewhere.
Get cloud vendor sponsor.
Seth: Might be reaching point where w3c infra is potentially not sufficient.
Might be a good idea.
James: should we hold off on people running more tests.
Seth: I don't think there is a risk of losing data.
It still might be a good idea to hold off.
James: the only left in the queue is checkbox, but have not emailed Lewis about it yet.
Matt: how does this impact the QR queue deployment.
Seth: still expect new CR Review queue in sandbox at end of next week.
That is by October 7.
Matt: will need reviewers of new experience during week of October 10.
Mode switching tests
We are going to look at a Combo box example with Default NVDA settings
Discussing mode switching for this example: https://
To start we want to review navigating to a combo box
The issue we face is so far all of the asserted behavior has been the same. The assertions are all inserted on a per test basis, instead of per assertion
Matt: Instead of incorporating this into the test navigating forward or backward to a combo box, we could have another test, potentially called activate a combo box. I had an idea to separate this test from navigation test, and the only assertion was did the mode switch?
James: Before we go on to that, we should discuss as the tests are written, even if we remove c, f, and tab because we are going to remove those separately, would that change current tests?
Matt: No, the current tests would stay the same, you just wouldn't added new assertions to them. Then have a new test with a certain set of commands to change more or not
James: If we have tests that only test when the mode should change, that feels unbalanced. We should have tests with assertions for cases when the mode should stay the same, and tests with assertions for one the mode should change
Matt: The fundamental thing that would be helpful would be to have different assertions for different commands within the same test, which is not currently possible and would be a nightmare to implement
James: We would have to overhaul the experience and make it backward compatible
Matt: The other option is to have more tests for each example of mode switching
James: Already the test plans names are getting pretty long
Matt: We have to think about the fact that humans and machines will have read these longs strings
James: These test names appear in multiple places, so that increases the time to read and might eventually become lost within the context
If we remove the state from the test name, that would result in 12 tests with the same name
Matt: We could change the test plan names to include "with automated change to interaction mode" or "in reading mode" or "Without interaction mode change"
James: I would prefer to keep "in reading mode" suffix, with mode switching commands
James: To keep the test names following a similar format
James: we have some automation for creating these names, so from a work flow perspective to would easier to change the task name, than change the entire ending
We had a discussion of the scope of tests, testing for known bugs in mode changes vs testing unknown bugs. This will need more discussion
Matt: I feel this is too important of thing to not test
James: I will work on creating some examples for Combo box
Matt: Grid is another important one to tes