– DRAFT –
Open UI Telecon

github-bot, take up openui/open-ui#1133

[Interest invokers] Keyboard inputs

<github-bot> OK, I'll post this discussion to https://github.com/openui/open-ui/issues/1133.

masonf: there are discussions on keyboard to "show/lose" interest

masonf: the explainer is up to date

masonf: it says to show interest with a hot key and lose it with another hot-key (most obvious is escape) and we denote that the focus rect should change should occur

masonf: we outline the pros and cons of this, is that it isn't easily discoverable that there is a special key to press when you get to these elements

masonf: the key other contender is to show the interest on focus after a delay

masonf: the tricky thing there is that it can be annoying when traversing through the document and items in the card and interest causes a change in the tree it can cause other problems

nmn: all of the things you said still hold with my proposal

nmn: when you're focused on the trigger, you mentioned a different indicator, the idea is to have a "rich tooltip" to provide information and is inert and won't provide any additional tree changes

nmn: you will still have to use a hot-key to invoke it but you'll be able to see it without having to use the hot-key

nmn: the benefits are the quick short-cuts in the hover card

nmn: the current solution doesn't give us the first value in order to get the short information of what you may/may not be able to achieve with the hover card

nmn: we've shipped this to about 50% of Facebook and it works this way at the moment

<nmn> +1

brecht_dr: a question, we were talking about the trigger being a key and someone mentioned keys and noted that arrow keys and that makes sense since commonly in a "select" when you're within in it the arrow keys traverse up and down

brecht_dr: I like the premise of another focus ring to inform the users

keithamus: seems like this feature is dedicated to rich tooltips and we're steering towards interactions based on rich tooltips and I wonder if simpler scenarios can just be focus and if it's a keyboard user they should get that information quickly in non-rich scenarios

keithamus: this begs the question, is there a way to automatically determine if it is a "rich tooltip" and allow two different modalities

keithamus: if it's plain text we have one behavior and elect to opt-in to different behaviors

<gregwhitworth> +1 to Keith on other usecases

masonf: I am definately aiming to solving all 3 usecases, hovermenus, plain text and hover cards

masonf: your point is a good one for plain tooltips this seems a bit heavy

masonf: my main concern is the interactive ones

masonf: there already is a switch for plain verse interactive content within it for accessiblitiy reasons defined by ARIA

masonf: I kind of like the idea, but honestly my concern is the complexity of it and landing this in standards

masonf: that is a fairly complex set of interactions that will be hard to standardize

masonf: up and down arrow may be a problem so you'll need a modifier key since that will impact scrolling

masonf: in the text of your proposal it sounds like the hover card is a modal dialog

masonf: that may be ok but I would point out where we shouldn't support dialog on interest target but we may want to re-open the issue since it sounds like the hovercard matches that usecase

masonf: there is also a comment on the proposal on moving focus based on the mouse and that's normally a "no-no"

nmn: moving the focus based on mouse-issue...

nmn: actually the dialog one first

nmn: I'm trying to make it work today and with the tech today it made sense for it to be a dialog for us

nmn: a way to think about it is there are two seperate views on the trigger

nmn: a rich tooltip that shows when you hover and when you interact with it the content changes and it becomes more rich

nmn: if you hover it, it is aria-hidden and inert and a aria-hint and the visual user with a mouse when you hover it shows you the view

nmn: we wanted to make it so that if you have keyboard and mouse you get the same experience

masonf: quick clarification, it doesn't need to be a dialog? You'd only lose the focus trapping?

nmn: yes, although focus trapping allows us to keep the traversal simple

nmn: this is a focus group so once it's invoked you can use the up and down arrow and the content will work. That may not translate to the web standards

nmn: so it may not invoke in the same location to where physical direction keys may not make sense

nmn: we hotly debated scrolling and arrow keys

nmn: where we landed was to use the up and down arrow keys but if you press the escape always gets you back and you can use the arrow keys

nmn: it was considered an OK trade-off

nmn: if it was more wide-spread we would need to think of a different solution. In the past we chose a keyboard use-case and used a shortcut and unless you were using an AT it wasn't discoverable

<Zakim> scott, you wanted to discuss aaron

<keithamus> https://images.expertreviews.co.uk/wp-content/uploads/2023/06/keychron_v4_1_1.jpg

keithamus: This image is going to unveil arrow keys, this is a 50% keyboard that may not have arrow keys

keithamus: what do we do in those circumstances?

masonf: how does a user with that keyboard scroll the page?

keithamus: the w,a,s,d keys, etc

flackr: we do have presidence with select and radio groups of using arrow keys

nmn: The rich tooltip usecase and I was listening to the original proposal and one thought that occurred to me was if I wasn't trying to define for ALL input modalities, one would be for the rich tooltip and one for the menu

<aaronlev> Do we need to think about remote control scenarios where the user is using arrows to navigate between items?

<aaronlev> maybe long press in that case

nmn: because then if you don't want the over-bearing rich experience you can just show a ring and then you can show the full card across that spectrum. I don't see how that works on touch screen

masonf: key thing I want to do is try and limit the amount of options because there will be foot guns

masonf: the goal is that "you've thought of hover and it works and then it works for everyone else"

keithamus: you could do something like "eager" allows mouse or keyboard events will display the tooltips immediately

keithamus: if you knew this you could have interactive content within the tooltip would block focus and then you can put a warning to devs in the console that it's not accessible

keithamus: then you can have one that is like "intentional" but the delays are longer and the abstraction still exists but there are popover hint vs popover auto

nmn: in your example for the eager situation if you want interactions after explicitly intentions somehow how would that be handled?

keithamus: eager is the menu example, you want submenus to open on hover and on focus and probably on touchstart, holding down, etc

keithamus: maybe it's the wrong thing to think that it's exclusive to think that it's only for non-rich since it covers menus

keithamus: I don't think that answers your question

masonf: for the mouse specifically, hovering over an element and waiting for a time that's not a problem and there is no distinction between interactive and not interactive wheras you need that for keyboard

masonf: so maybe you have something like "immediate" and if you have a submenu when I focus it I want to show it and I want it to be active

keithamus: ironically the arrow keys would be the way

keithamus: I have to use the arrow keys to traverse into the submenu

masonf: the arrow keys matter depending on the direction

flackr: I've heard disntinguishing between rich and non-rich. I think if it's not interactive it doesn't matter so we may not need to distinguish this

flackr: it may be valuable to not having the tooltip precluding other potential targets

flackr: forcing content to be inert may be valuable to explore for some of these automatic usecases

flackr: regarding arrows impacting scroll I don't think it's a huge deal as we already do this in select and radio groups, the page stops scrolling

masonf: there are a few exceptions that those are annoying if you have numerous of these

flackr: how common is that though where you want to scroll it when you tab to it

masonf: any time you want to scroll

<nmn> link to our implementation: https://www.icloud.com/iclouddrive/0cbA2oHnz7eTJXZtcoYUS-cVw#Screen_Recording_2024-11-21_at_6.35.59_PM

scott: I'm going to respond to scrolling first, I completely understand where you're coming from and those are 20+ years of known things but if things like links and buttons no longer works

scott: that's not good and we shouldn't do that

flackr: let's say if you have already shown interest and you've reached the limits in that container once you're at the end of the options it can then scroll

scott: yes that puts it back on the user though

scott: the popup is also scrollable and they're scrolling that to then scroll the page then they can press escape, etc

scott: so we used to be able to scroll the page and now they can't but I want to make sure that we don't do something that when we compount on top of each other it becomes a problem

scott: now back to what I wanted to talk about

scott: I don't understand the menu usecase

scott: for submenus and such I wouldn't expect a command to open one

scott: if you're navigating through them and there is an indicator I would invoke it or press the arrow key to open that opens submenu

scott: in most of those scenarios they are solely to open the sub-menu

masonf: I'm glad that you brought it up is because it's something that is commonly hover trigger; maybe we should get rid of it but it delays

masonf: activation almost always opens a submenu

masonf: only question would be if it's harmful or not

scott: my worry is that if this becomes a standardize thing then there is this hotkey to open it and things that should just be to "hit enter" we move away from it and it complicates it

scott: should popover hint on hover be how this is exposed for sub menus. We don't need interest targets

masonf: for popover hint you need a way to connect them I think

sarah6: what I wanted to discuss around to automatically detect if something is rich or not

sarah6: I actually think it's more common than rich verses not

sarah6: if you have a static popup that's really big and have magnification they'll want to kill their computer

sarah6: sometime it will be a short popup and it will have links in there but we don't intend for people to "really" go in there too often but that may be something that authors will need to determine

sarah6: I think the browser figuring this out will be very hard

masonf: quick follow up is that if we give authors control they may end up doing the wrong thing

masonf: if we just had a way to have the authors to say "this is complex" and it allows us to force it to be inert

masonf: the usecases you shared are good as they show the problems that can occur

keithamus: I agree with sarah

keithamus: it has to do with size and impacting hiding content

keithamus: but we need to solve issues with UI that are ephemeral and sub-menus sort of fall into this pattern

keithamus: the menus that I typically interact with the hover will open the sub-menu and focusing it will open the sub-menu

keithamus: the interestarget and commandfor both pointing to the user into the menu as there are subtle differences as that is what I'd like to see a menu coded

nmn: quick response

nmn: on the scolling and arrow keys and you get onto a script that is trigger you can press escape key or arrow key down twice then it will go back to scrolling the page

nmn: if you make the popover scrollable then that is a problem

nmn: I think it's fine to try something different than arrow keys if it's a web standard and people get used to it

nmn: another thought I keep having about the inert usecase

nmn: maybe we give the useragent control over what shows up

<scott> when a screen reader is running (on windows) space works the same as enter. so if space were to be used, then screen readers would need to change to account for this

nmn: our initial scenario was showing the hovercar at half opacity upon hover to inform the user that they can invoke it

nmn: the useragent can do some type of graphical change to inform the user and that may impact the size of what you show, etc

masonf: I keep trying to map this back to how we would "standardize this" and there is a discoverability problem with hot keys. But my concern is that the standard may not allow this so it doesn't get in the way

<sarah6> scott I think screen readers would need to have a fully separate way to indicate interest, since most keyboard options would already do something

masonf: maybe there is a pseudo element that when it's inert there can be a hot-key that allows the user to use it and you can change the visibilty

nmn: *thumbs up*

nmn: sounds better to me and I posted a link to a screen recording

<brecht_dr> because of what mason said.. +1

sarah6: on regular down arrow keys we have a number of keys that can be used

sarah6: navigation style stuff or tree or some other role, lists, interatctive lists, etc

sarah6: scott reminded me that enter and space trigger clicks in certain modes and space would be difficult there

sarah6: there is a lot of AT that would need their own way to show interest

sarah6: zooming software could override their own way to denote interest

sarah6: will there be an event that they could listen to in order to possibly do that?

<nmn> +1

masonf: please open an issue on that as that sound interesting

scott: I want to understand the talk about inert a bit more

scott: making the popup make it inert, how would they know it's existed but it's inert

scott: assume someone that is low vision with zoom and the popup is inert they won't be able to access it

scott: maybe I've missed it

masonf: we're coming close to the end of the meeting but I would like to summarize what I've heard

masonf: the behavior is after a delay and you've focused an element that has an interesttarget that target will be shown but force it to be inert

masonf: that makes it so that it doesn't add additional tab stops

<nmn> The actual relevant link: https://www.icloud.com/iclouddrive/093R9zAV5ic-A_hlfjXOBZxBw#Screen_Recording_2024-11-20_at_4.55.29_PM

masonf: however when it shows it, there will be a pseudo element that will allow you to have a hot-key that will remove the inertness of it and you can engage with the content

masonf: along the way there was a suggestion on the first state that aria-hint that informs the user on how to invoke it and remove its inertness

scott: an ARIA details would not be there if it's inert

scott: ok, that's making a bit more sense

masonf: no that's good

masonf: for plain tooltips that don't have interactive content they stay inert and their accname and description are a part of the triggering element

nmn: I added a new link that is a bit more dramatic than what we shipped

masonf: I'll take everything and try to summarize it in the issue