Meeting minutes
Review agenda and next meeting dates
Matt_King: any requests for changes to the agenda?
mmoss: I have a question about a specific test plan
mmoss: It's disclosure navigation menu
Matt_King: Got it; we can add it to the agenda
Matt_King: Next AT Driver Subgroup meeting: Monday January 13
Matt_King: No meeting Wednesday January 15
Matt_King: Next Community Group Meeting: Thursday January 23
Current status
Matt_King: Currently, in the app, in the test queue, we have some conflicts in the "link example 2" results
Matt_King: It looks like only one person has tested "link example 3" so far
Matt_King: I'd like to review those conflicts with "link example 2" and see if they're something simple that we can address today
Matt_King: We're done with VoiceOver and JAWS for "link example 2"; we have only NVDA remaining
Matt_King: We have two conflicts
Matt_King: In test 2, it looks like Dean's output is completely different from Luke's
Matt_King: Did this output come from the bot?
dean: It's been so long since I ran this that I'd have to do this again to see what went wrong
dean: It doesn't look like something I would write, though
Matt_King: If you could re-run this, that would be the best thing right now
dean: Will do
Matt_King: I think we may want to allow testers to run the bot
Matt_King: And further, I think we may want to be able to run for just a single test
howard-e: That's not currently possible, but it's something we could implement. Off-the-cuff, though, I don't know what the level of effort would be for such a change
Matt_King: In "link example 3", it looks like there's bot output, and the output hasn't been evaluated, yet
dean: I'm afraid I can't complete those tests right now because I don't have an "insert" key
michaelfairchild: There are a couple things you can do. You can plug in a Windows keyboard, or you could remap keys using specialized software
dean: Got it!
Joe_Humbert: You should also be able to remap NVDA to not use "insert". I think you can use "Caps Lock", instead
Matt_King: That's true, but as much as possible, we're trying to use the default settings
Matt_King: Now, for the "radio" test plan
Matt_King: We have people assigned, but it doesn't look like any testing has occurred, yet
Joe_Humbert: I'm assigned to both NVDA and VoiceOver for that test plan
Joe_Humbert: I stopped because I ran into an issue where pressing the "run test setup" button causes an "h3" to disappear in test 1
Joe_Humbert: I wasn't sure if that was valid, so I paused because I didn't want the test results to be thrown out
Matt_King: I believe that is intentional. James did that on purpose to create a specific state
Joe_Humbert: Understood. I can continue on with the testing
Joe_Humbert: There might have been an issue, though, because I think something failed. I think it was removing part of a label... I'll double-check offline and report back later
Joe_Humbert: I think the heading was used for a group name, and since the heading is "display: none", it no longer announces it
Joe_Humbert: Like I said, I'd like to double check, but that's my recollection of the behavior I was observing
Matt_King: thank you!
Issue 1171 - VO Bot output has extra spaces
github: w3c/
jugglinmike: The issue Joe raise is about how the output he reported varies in subtle ways from the VO bot. Based on the use of spaces and new lines
jugglinmike: I wanted to bring this to the fuller group to understand how the bot should behave. I don't know what the significance of new lines and spaces are. Maybe we all could define some heuristic together
jugglinmike: For example, if you look at the example, should we consider 2 responses that look mostly the same as the same response. This might be an issue for 2 different human reporters too, so that might not be just a bot issue.
Matt_King: It's important to distinguish if outputs change over time and across AT version. The other time this matters is when we're display the output (for example in the test runner, etc). So if we're doing normalization, there's some interesting consequences if we normalize input into a text box in a certain way.
jugglinmike: I was thinking about displaying what people actually type. But also for the sake of transparency, but also surfacing a normalized version. So that testers can compare their raw output vs the normalized output.
Matt_King: Maybe anything that's a report, we always show the normalized output. But we can always make the raw human input also available. But for the runner, we shouldn't do normalization.
Matt_King: Is there a decision we need to make about what we ask testers to do? We might not to, if we agree on a normalization algorithm.
jugglinmike: I think so. This came up in internal conversations. Maybe we set the rule to match what's in the voiceover history. But I don't know what exactly the VO rules are for how they render text output. And I'm not sure Apple can commit to stable output either. So I wouldn't want to lead with that.
jugglinmike: Bocoup can put together a proposal for this. We'll take this as an action item
Joe_Humbert: I filed another issue about radio buttons. There's an issue with a disappearing header making the requested command incorrect
Matt_King: This is a problem. We can't have the tests mismatch
Joe_Humbert: Back to the extra space, it makes it difficult for me to edit the output if there's unnecessary new lines and trailing spaces. Like 50-60 trailing spaces between new lines.
Joe_Humbert: This wasn't visually rendered on Github
Joe_Humbert: The problem seems to be somewhat random
jugglinmike: I think we can still move forward with a normalization solution
Thanks jugglinmike !
Disclosure issue
Matt_King: Sorry, mmoss, we're out of time for today. Can you file an issue about this? Then we can follow up in the next CG meeting
mmoss: Absolutely