Spatial Navigation in the Web breakout session

<scribe> scribe: cpn

<jcraig> /www.w3.org/2017/11/08-spatnav-irc///www.w3.org/2017/11/08-spatnav-irc

hyojin: this is a TV specific issue, but could be broader, for other industries
... smart TV has a great screen, but i'd like to focus on input devices
... desktop and mobile have touch, keyboard, mouse interface
... i'll demo how the LG remote control works

[shows video]

<hyojin> LG remote controll video: https://www.youtube.com/watch?v=uoxYO9DslkA

hyojin: you can see the moving pointer, and other option keys on the remote control
... with this remote control, you can use commands such as pointing as well as voice and gesture
... as well as remote control by mobile
... android TV is pushing this method
... we need well defined specifications to enable the 5-way buttons (arrows and OK button)
... we want to propose an extension of the web for spatial navigation
... there are many web based TV platforms, eg opera TV, tizen TV
... they have the same issues

<igarashi> +Tatsuya_Igarashi

hyojin: we made a web framework for spatial navigation features, that we'd like to standardise for web engine support
... native platforms have considered spatial navigation (android TV and apple TV), but it's not supported on the web as of now
... i'm talking about the TV industry but other devices could be considered
... eg, desktop with keyboard for accessibility
... TV/STB and IVI
... also game console using gamepads
... on the desktop, we use the tab key for navigation, but it's hard to use the 4-way navigation as of now
... i hope to see the web embrace spatial navigation
... considering arrow key support on the web, they move the cursor between selected input items
... normally, the arrow keys are used for scrolling
... spatial navigation means using the arrow keys for moving focus.
... how can spatial navigation be used for this on the web?
... it's already supported in chrome, behind a flag

<dbaron> https://lgeweb.github.io/spatial-navigation/demo/heuristic/heuristic_testcases.html

jihye: this doesn't use any javascript apis for navigation
... the movement is quite nice, but doesn't cover all cases

<hyojin> Spatial Navigation Overview: https://github.com/lgeweb/spatial-navigation

[demo of some controls that are unreachable]

scribe: eg, a scroll element that can't be reached
... web pages on TV look like this: content with side bar menus, sometimes a long scrolling list
... spatial navigation can cover these cases
... we have two ways: combining with arrow key behaviour, and the touch screen algorithm
... with the heuristic spatial navigation, the arrow keys move between text elements
... sometimes the user wants to go to the next element without moving the cursor
... the heuristic spatial navigation moves the scroll bar of the scrollable area first
... the focus is moved when the element comes into view
... some users want to customise the navigation algorithm
... we want something easy to define for developers and users in CSS and JS
... it will make it easier to develop apps using spatial navigation
... we provide ways to override the heuristic spatial navigation

<jihye> https://github.com/lgeweb/spatial-navigation/blob/master/explainer.md

scribe: our github repo has an explainer page
... we provide 3 spatial navigation algorithms
... projection, direction, and nearest

[explanation of the three approaches]

scribe: we also provide focus looping, which can be applied to long lists in the scrollable area
... the heuristic spatial navigation doesn't support this right now
... it's useful for long scrolling pages, avoids the need to press the arrow keys a lot

[demo using the proposed API]

<hyojin> Timetable demo: https://lgeweb.github.io/spatial-navigation/demo/

scribe: direction option uses shorted distance
... when the focus is at the end of the page, it won't move any more
... if you enable focus looping, it will return to the top of the page
... pressing the up arrow key at the top of the page goes to the bottom of the page

[discussion of what the default behaviour is]

<jcraig> From the explainer doc: https://github.com/lgeweb/spatial-navigation/blob/master/explainer.md

<jcraig> arrow-key-behavior: auto | navigation | scroll

florian: opera used to have this, but no longer since they moved to blink

dtapuska: would like to see this specified

florian: we used to have as-smart-as-possible heuristics, lots of edge cases, so sometimes the author needs to control it

<jcraig> default Safari behavior seens to match the intention (If I understand the spec correctly) of the value for ‘scroll’

florian: this works for a small page with few elements
... maybe we want to remove heuristics entirely
... dtapuska: i'd like to see an algorithm come out that we can agree on
... not sure it will be possible to agree it

sangwhan: the spec should define algorithm, but could be updated with smarter algorithms later

<Zakim> jcraig, you wanted to mention FireFox caret move default behavior, with matches the Safari behavior if Accessibility is enabled.

dtapuska: auto mode is up to the browser, but each vendor could do different things, so no interoperability

jcraig: as an example of algorithms that can be updated, the WCAG has a color contrast algorithm that works in 99% of places, but sometimes perceived contrast differs
... looking at the explainer doc, something not covered in arrow key behaviour, i think firefox by default is caret position
... there need to be a few more properties, and these need to allow for the user context, and allow the user to control

<dbaron> In Firefox, Shift+arrows certainly moves caret position/selection; I haven't seen unshifted arrows do that.

jcraig: another thing is containerization of grouping, eg, tab navigation within a group, then go to spatial navigation

<dbaron> (but I'm not particularly knowledgable there)

jihye: we are trying to solve spatial navigation applied to grouped elements, but at the moment it's for the whole page. we're figuring it out
... we'd like to discuss via github issues

<hyojin> github issues: https://github.com/lgeweb/spatial-navigation/issues

jihye: remaining issues there: how to enable spatial navigation features, how to trigger the spatial navigation rule, and one-side axis considered nav-loop

hyojin: we can write a design document to describe the heuristic, but how can we make progress on that?

florian: this seemed to be a CSS topic, so we brought it up there
... but it's broader than that, need input from components people
... defining the rules, follow the extensible web approach
... how to treat events? event ordering, preventDefault, etc
... we need a group with overlapping skills: components, CSS, JS, a11y
... we are in the CSS WG, we're getting the spec into WICG, build a community

<Zakim> jcraig, you wanted to suggest js polyfill (arrow key interception) to try out in any browser

<sangwhan> Ditto on jcraig's suggestion, would allow iterating over different algorithms cheaply

florian: then we can go to the various WGs
... related work in CSS is override, stale for a while. it's in Presto, the old opera engine
... spatial navigation is in blink, HbbTV require it, so its in TVs in Europe

jcraig: it would be useful to try a JS polyfill
... could be a quick way to try things out, specifically about the examples demoed earlier

<robdodson> are there links to the polyfills somewhere?

<Zakim> aboxhall, you wanted to discuss use cases differing between devices

aboxhall: depending on the device. for a TV you want everything to be spatial, eg using a d-pad

<Zakim> IanPouncey, you wanted to discuss user expectation

aboxhall: on desktop, the idea of containers come into play, using tab key with spatial navigation

IanPouncey: with the different navigation methods, how do we express to users what will happen. the defaults are easy to get used to
... but when this may change based on author settings, behaviour may be unexpected
... bkardell: in CSS WG we talked about moving this to WICG, and other related concerns
... what should the scope of this include at WICG?
... eg, is focus an appropriate part of this?

aboxhall: focus ring and intert are distinct from this but adjacent
... should we have a group looking at all of this in one place

bkardell: that may slow us down

florian: it may actually speed us up

[ florian calls for support ]

florian: we're discussing this in WICG at 1pm

<Zakim> jcraig, you wanted to ask if linear navigation modes have been considered… e.g. menu (vertical), tab list (horizontal), grid (vert+horiz), and 9-key (5-key plus diagonals) and to

jcraig: tab panels are usually horizontal navigation, single-level menus are usually vertical only… Please consider subsets of “spatial nav” such as vertical-only, horizontal-only… 5-key, 9-key, etc.
... this would be useful from a a11y point of view to implement in a small containerised package

<aboxhall> The ARIA design patterns would be good to mine for use cases for this ^^

<aboxhall> https://www.w3.org/TR/wai-aria-practices-1.1/#aria_ex

jcraig: eg tabbing into a menu or grid, maybe able to easily specify elements of the spatial navigation to recognise

<robdodson> "roving tabindex", which jcraig mentioned, explained: https://developer.mozilla.org/en-US/docs/Web/Accessibility/Keyboard-navigable_JavaScript_widgets#Technique_1_Roving_tabindex

fremy: screen readers override the arrow keys
... a couple of questions: why do we need the arrow keys by default, and why not solve at the a11y and screen reader level?

johnfoliot: what about scrollable pages without many focusable elements?

tantek: webtv implemented that. if there are focusable elements close by, it goes to those. if not, it scrolls

johnfoliot: screen reader users may want to focus on a region that's not tab focusable

<tantek> +1 to johnfoliot's points

florian: it probably couldn't be enabled by default for arrow keys
... for desktop browsers, most don't support but not for arrow keys, only shift-arrow combos
... another approach: declare at the page level that it uses spatial navigation
... may be useful to have a switch

hyojin: thank you for your comments, we can continue the discussion

jihye: we'll be at the WICG meeting, starting at 1:30pm. please join us
... thank you for coming

[adjourned]

<jcraig> rrsagent make minutes

<jcraig> s/topic https://www.w3.org/2017/11/08-spatnav-irc\//

- DRAFT -

Spatial Navigation in the Web breakout session - TPAC 2017

08 Nov 2017

Attendees

Contents

Summary of Action Items

Summary of Resolutions

Scribe.perl diagnostic output