Meeting minutes
Review agenda and next meeting dates
Matt_King: Next Community Group Meeting: Wednesday November 6
Matt_King: Next AT Driver Subgroup meeting: Monday December 9
https://
<Joe_Humbert> I have a general question when time permits
Matt_King: Requests for changes to agenda?
IsaDC: I have a quick question: could we get the latest version of NVDA for the NVDA bot? It fixes some issues that we've had for disclosure
IsaDC: version 2024.4 (we have 2024.3 right now)
jugglinmike: Yes, that is generally a quick task, but we never know what we'll find with each never version. We'll prioritize this work next week
Matt_King: Okay, then beyond that, we'll stick with the agenda as planned
Current status
Matt_King: Now we have eight out of the ten plans that were in candidate review are up-to-date and current (it was seven last week)
Matt_King: Now, we just have "disclosure" and "navigation menu button"
Matt_King: Once that's done, all of the reports on the site will be in the new format. We're getting really close on that important milestone, and that will allow us to complete more of these conversations with AT Developers
Matt_King: Hopefully we'll be celebrating 10 out of 10 in the next couple weeks
Matt_King: And then beyond!
Matt_King: We are making progress in the conversations with both Apple and Vispero
Matt_King: I feel like we're going to be at the point where several of these plans are in the "recommended" phase, soon
action menu button state priority question
github: w3c/
Matt_King: I almost hate to be raising this issue so late in the process
Matt_King: I'm asking whether or not communicating the state "collapsed" should be a priority one or priority two
Matt_King: I'm starting to feel that the state is only a "should" because you could still operate this button if you didn't know the menu was collapsed
Matt_King: For a long time, we said to only put the "expanded"/"collapsed" state on the button when it's expanded because it seemed unnecessary when collapsed
Matt_King: It's sort of like what Apple talks about with "pressed" and "not pressed"
Matt_King: There were a couple reasons why we shouldn't be inconsistent in that way. That we should always have "aria-expanded" and it should be up to the screen reader to determine whether it was conveyed
Matt_King: One of the issues had to do with touch. When you are using these things on a touch device, you can focus the expanded or collapsed menu button regardless of the menu's state. It's really helpful to know the state of the menu in that case, especially when the menu doesn't appear adjacent to the button
Matt_King: So the question I'm asking is about the "collapsed" state itself (not the state change)
Matt_King: I suppose this would also apply to when you're navigating to...
IsaDC: Also when you close the menu. I think it should stay
James: I'm inclined to say that if aria-expanded was not conveyed when it was "False" on a button, I would be concerned
James: However, a menu button is different than a regular button
Matt_King: The disclosure is completely different. You wouldn't know that, otherwise. In this case, because it's a menu button which has its own roles...
Matt_King: With a "must" assertion, basically, people wouldn't be able to perceive the interface at all if the screen reader didn't perform it
Matt_King: I'm thinking about if a screen reader fails to convey a "Collapsed" state on a menu button, what is the impact on a screen reader's ability to use the menu button?
Hadi: They would miss it
Matt_King: Well, if it said "Actions menu button", you would still recognize the menu button--even though it didn't say "Collapsed"
IsaDC: We didn't have state before, when we first ran the test plan, it didn't assert state at all
James: Reading the definitions, my view is that it should be a "may". The "should" says that it is "likely to impede users". I don't think the lack of state information is likely to impede users
Hadi: sometimes the children are not adjacent to the parent. The menu must be exposed, but it might not be immediately after that button. I don't know that, so I have to explore back and forth. It might be a long distance to travel to those children. I'm more toward "must", but I'm open to "should." Not "may", though
Matt_King: The menu button is supposed to move focus into the menu when it is expanded
Matt_King: If the menu is collapsed and it just says "action menu button" and you press it, then the focus is supposed to go inside of the menu
Matt_King: Would that impede you from using the menu button effectively?
James: To clarify my previous comment: going solely on the assertion descriptions in isolation, I think this is a "may" because if a menu button is collapsed, it's true that focus does not necessarily move into the menu. The state being conveyed as "collapsed" does not necessarily help you avoid that situation
Matt_King: Since we're not asserting "expanded", if it didn't convey "Expanded", then you'd be in a bad place
James: Since we don't have a test for "Expanded" and any such test would be convoluted and contrived, I believe it should be a "should" for collapsed
Hadi: These things have been developed to improve the experience for everyone, including screen reader users. I am tired of meeting the minimum requirement. This minimum requirement means more work for people to get something done
Hadi: I know we have some protocol to follow, but I don't mind pushing to make it a requirement
Matt_King: Remember that we're not talking about the web page developer; we're talking about the screen reader developer
Hadi: Exactly, if the web page author made the effort, the screen reader should respect that
Matt_King: The reason having the difference between "must" and "should" applied consistently throughout the test--the reason that is important is that new or not-so-well supported ARIA features become available, it's really important that a web developer be able to look at the report and say, "it is basically safe for me to use this"
Matt_King: If there are too many "must have" behaviors that most screen readers don't support, then we end up in a situation where it inhibits the use of ARIA
Matt_King: I'm thinking about the future and how people will use these reports then
Hadi: I still push for "must"
Matt_King: In certain tests, it is failing. It would still be counted as a "fail" even if the assertion was a "should". In that case, the score is still not 100%. If we changed it to a "may", they might not fix it. If it's a "should", they're still gonna fix it
IsaDC: JAWS is failing this right now. I was quite surprised to see that
Matt_King: Me too
Matt_King: That's one of the reasons why I raised this issue. I wanted to make sure we had it classified correctly
Matt_King: And in a consistent way
Matt_King: It sounds like Hadi and James are both comfortable with a "should"
IsaDC: I also vote for a "should" to kind of soften the blow a little without losing the "hit"
Matt_King: Yeah. I want to be able to explain each of these assertions.
Matt_King: I don't know if we should be writing the rationales down, e.g. why is this a "should" or why is this a "must" or why is this a "may"
Matt_King: In this case, because we are not testing for aria-expanded, it becomes particularly important that aria-collapsed is conveyed because if you're interacting with this with touch, it's really important that you know at least one of the states. Otherwise you would be severely impeded
Joe_Humbert: Does the button always convey that it is a menu, though?
Joe_Humbert: The APG says that it has to have a role of "button"
Joe_Humbert: If aria-haspopup is true, does that convey that it's a menu?
Matt_King: Always
James: the element's implicit role is button, but as soon as you add aria-haspopup, it changes the role
Matt_King: Originally, aria-haspopup was a binary. We made "true" equal to "menu" in the interest of backwards compatibility
James: We should assert that the role of "menu button" MUST be conveyed
Matt_King: I don't know if haspopup is mentioned in the references, but it needs to be in the tests for "role"
IsaDC: I remember adding it to the references file...
James: We're saying that the user "should" convey them to the user, but if it doesn't, it wouldn't prevent them from using the button
James: In that way, WCAG is very binary, where this is three-state
Matt_King: Yup, aria-haspopup is in the test
Joe_Humbert: That answers my question
Matt_King: I think we'll update all three test plans
Matt_King: I'll do a pull request for this
Matt_King: It won't affect any of the test plan results. No one will have to re-test anything
Matt_King: Thanks everybody for this discussion!
Testing Disclosure Navigation Menu
Matt_King: We have four conflicts right now
Matt_King: There are still three open issues about conflicts that I think might be resolved. I've listed those in the agenda
Matt_King: I think they're all tied to Dean's previous testing that we discarded
IsaDC: I'll take a look at those
Matt_King: Right now, I think we want to talk about the four conflicts about link activation with JAWS
Matt_King: Test 14. Hadi has much more verbose output. When I run this test, JAWS is extremely verbose. The focus moves to the heading, and JAWS reads information about the page as if the page had just completely reloaded. It counts the landmark regions and hotspots and stuff
Matt_King: I get that same result as Hadi
Matt_King: Mine's even more verbose than what Hadi wrote done
IsaDC: I believe I cut that out of the output because I considered all of that as part of hints, not as part of the result. Because it was breaking it down into giving a description that explains what's on the page
Matt_King: Actually, I think we should include all the output for any command. I don't know how we should edit down.
Matt_King: Think about the automation, for instance. It's going to capture all of the output
Matt_King: I think what JAWS is doing here is navigating to the heading, and then starts reading at the heading, and reads the rest of the page
Matt_King: I'm not sure if that's what Vispero wants to happen
Matt_King: Personally, I think it should only say "heading level 3 overview" first and then say the name of the landmark region
Matt_King: I think Hadi did the right thing here
IsaDC: I'll change my results to match Hadi's
Matt_King: Let's mark this as excess verbosity with moderate impact
Hadi: I marked it as severe
Matt_King: I don't know if Vispero would agree with that
Matt_King: I waste more of my life due to broken same-page links...
James: What are "smart glance highlights"?
Matt_King: JAWS analyzes the page and calls out things that seem visually pronounced
Feedback: "Navigate backwards to an expanded disclosure button" (Disclosure Navigation Menu Example, Test 4, V24.10.10)
github: w3c/
Joe_Humbert: In the example, if the menu is open and the user is at the "navigate backwards from here" link--if I were to press shift+tab, should focus go to the last item in the menu or the last button?
Matt_King: the last link in the menu
Joe_Humbert: I observed that the last menu item is selected, but the menu is not open
Joe_Humbert: This doesn't represent realistic behavior
Dean: I agree with that
Joe_Humbert: When you run the test setup, the button is shown as "active" but the menu is not visible
Matt_King: That's a problem
James: The test has nothing to do with a disclosure menu, it's for a disclosure button
Matt_King: But you could easily do that just by opening the menu and placing focus on the first link in the menu, and...
James: I think there was a problem with that approach. I'm not certain, though; this was a long time ago!
Matt_King: I'm kind of thinking that should be changed...
Joe_Humbert: The purpose of this test is to see what it announces when the focus moves backwards to an expanded disclosure button
Joe_Humbert: If it's a hack to get it to move consistently, that's fine by me. It just seemed odd
Matt_King: Which test number?
James: Test number 12
James: test number 12 overall, but it's the fourth test in the progress list
IsaDC: This could easily break the test itself, particularly with VoiceOver. That might be why it's not moving
Joe_Humbert: I understand. I can close this issue
James: I'm unclear why we didn't use a different script that exists for a situation like this
Joe_Humbert: So, do you want to keep this issue so you remember to revisit this?
James: I think it's okay because it's only hiding an element. If it were doing something advanced like intercepting shift+tab, that would be one thing
James: But I think what we have is fine
James: If others think it's a major problem, then we can absolutely change it
IsaDC: Thank you for explaining this, Joe_Humbert, because I didn't understand before
Matt_King: I hesitate to make more work. I'm a little concerned by this, though. I suppose we can try to push it through the way it is. If we get feedback from screen reader developers there, then we can address it at that time
Testing navigation menu button
github: w3c/
Matt_King: Is it okay if the role is conveyed in the hint?
Matt_King: I think it would be very difficult to tell Apple that the role isn't conveyed at all
James: To be honest, I would find a menu infuriating if it said "menu" all the time
IsaDC: It's just a "may" assertion
James: I'm glad for that!
James: If we were talking about a button, and VoiceOver just said the name and only said the word "button" in the hint, I think we'd all feel a bit different
IsaDC: But in this case, you know that you're in a menu and that you're moving through items
IsaDC: Do we pass it now? Because we failed it elsewhere when it was in a hint
IsaDC: Recognizing that it's a "may" assertion
Matt_King: If it was a "must" and it was in a hint, then there's a problem when the hint goes away
Matt_King: But they're on by default
James: If you have to wait for the hint to tell you something, then you'd never get anything done
IsaDC: For consistency, if we pass it here, we have to revert that other test and pass it there, as well
James: So in the activedescendent plan, the text is present in the hint, and we failed it. You're saying that if we passed it in this one, it wouldn't feel good because we'd have to go back to the other and change it (even though it's already been removed from the test queue)
Dean: We shouldn't do it the wrong way now just because we did it the wrong way before
Matt_King: Let's decide what's correct right now and then do whatever work that entails
James: I'm going on three things. The first is Apple's assertion of how many people turn off hints. Secondly, the fact that it's a MAY assertion, so it does not reduce the support level (in order to notice that this is even a fail, someone would have to look very closely at the report--it's not a headline reduction in the support level).
James: Third, if we were saying that this is an important assertion and then it failed, we would feel much more strongly that this was an issue
Matt_King: The fact that it appears in the hint, though. The words are there. There's some cognitive dissonance there. We just need consistency
Matt_King: When we record the output, we definitely need to record all the output no matter what. If we are going to say that hints cannot convey role or state or name or value (which we could), then there's questions about properties
Matt_King: I'm not sure whether hints should be allowed to convey certain properties
James: I think hints are hints and should not be required to use anything
Matt_King: The thing for us to write down someplace, maybe the testing manual or in the glossary, is that hints are part of the output, but we won't process information in the hints to determine whether or not role, name, state, value, or properties are conveyed
James: I think that's fair. I disagree with Apple on many things. I fully agree on their stance, here, though.
James: They recognize that many people opt out of hints, and that among the 10-15% of users who have them enabled, that many of those people are developers or accessibility testers
James: And so that's why Apple views them as slightly redundant
Dean: I think we need to make a solid decision here. Either hint text or no hint text. If it's "no hint text", then turn off the feature and don't include it in the output
Dean: If Apple is saying that a low percentage of people use it, then we're creating problems by observing it
James: If we say that hints don't contribute to results but we include them in the AT responses, then we open ourselves to the kind of feedback that isn't helpful
James: With JAWS, hints are spoken immediately. With VoiceOver, there is a significant pause
Matt_King: Going back on our principle of testing out of the box...
Dean: In that case, we should do the opposite of what I described earlier
Matt_King: I'm trying to have my cake and eat it, too!
Matt_King: VoiceOver and JAWS gets hints wrong sometimes, and I would like to be able to call that out
Dean: It doesn't make sense to say that "We use out-of-the-box settings" when 80% of users don't use out-of-the-box settings
Dean: If 15% of people hear additional information, that's fine, but we need to decide what's mandatory in the output that everyone is hearing
Matt_King: The problem is that we don't have those numbers. Vispero tells us that beyond changing the speech rate, most users use out-of-the-box settings. Apple is telling us that most people turn off hints. I believe that's true on iOS
Matt_King: I would say that 100% of developers have the hints on.
Dean: If they're getting their 10-15% based on developers, then that's not relevant
Dean: My vote is that everyone turns off hints when testing with VoiceOver
Matt_King: We don't want different rules for different screen readers
James: I felt that James Craig was quite firm in his assertion that hints should be discarded
James: I know many advanced users that make many customizations yet they still have the hints turned on
Matt_King: The first thing I do with VoiceOver is turn down the verbosity
Matt_King: I'm feeling stuck between what Dean is saying and things that we've agreed to earlier. This concept of testing out-of-the-box is pretty important
James: If we write a test plan for aria-description, and a VoiceOver user navigates to the control that has aria-description, and VoiceOver says "more content available" without reading more. Would you, at that point, say the failure has occurred, or would you say the user should open the item and find the description, and assign a verdict based on what the screen reader says then
Matt_King: We haven't discussed how to interpret responses in that case
IsaDC: What should we do?
Matt_King: Dean believes it's unacceptable to include hints in the output but then ignore that information when assigning verdicts
Dean: That will complicate things far too much in my opinion
Matt_King: I guess we need to review the pros and cons. Maybe this is a slightly more complicated decision than we would like it to be
James: The question is, are we going to be able to find more information that makes this decision easier?
James: The only things I can think is: speaking to more community group members and speaking to Apple
Matt_King: I would like to create a table of pros and cons and presenting that to others. I think getting alignment on that information is important
Matt_King: Even though this is an optional assertion right here, it's going to come up in a "must" or "should" at some point. That's why I don't want to decide based on the fact that the issue right now is for a "may" assertion
Matt_King: I don't think we need to make a decision based on the suggestion of one screen reader developer. I want some consensus
Matt_King: I'll try to help us map out a decision path
IsaDC: So we won't move anything right now