Meeting minutes
Aligning proposed APIs
Mike: Plan on today's call is to continue Second Screen CG discussion of related proposals: Multi-Screen Window Placement, Window Segments Enumeration API, Screen Fold API, and potentially Visual Viewport API
… I can give a quick overview on top of my head :)
… Multi-Screen Window Placement caters for a multi-monitor use case
… the web is catching up here
… with permission we hope to provide the user and extended display environment
… Window Segments Enumeration API targets foldables
… use case here is to provide a content area, DOMRect describing spread across multiple internal displays or a portion of a physical display
… window manager will align the window or an entirety of the display, the developer wants to align its content to a specific region
… align content to one fold or the another, that's an important distinguishing factor, how the content is laid on an existing window area vs. placing windows
… third proposal is Screen Fold API, the premise is to offer developers postures, so foldable device could be in a laptop or book posture, or in tent mode etc.
… another component of that proposal is to provide the angle of the hinge at different times
… any questions anyone about these proposals?
Mike: we tried to align these proposals in our previous CG discussions last week
… Zouhir had a good synopsis, some are targeting different device categories, some different usages(?)
Window Segments Enumeration API
Mike: I took an action to describe more specifically how we could polyfill Window Segments Enum API using the Multi-Screen Windows Placement API
… I discovered that w/o innerLeft/Top the content within the frame, we cannot calculate
… browsers do not expose the area(?)
Mike: another reason why we decided these proposals to proceed in parallel, Segments, and Multi-Screen, is a notion that it is much more straight-forward scenario to adapt content to windows shape
… if laid across multiple screen, or move its window, it is simply trying to determine the device characteristics to help with that
… punchhole cameras, screen safe areas via CSS MQ, and potentially related to Visual Viewport API
… providing zoom pinch area rendered
… while we could extend Multi-Screen and polyfill Segments, not sure we should take that direction at this moment
… Screen Fold API provides info on postures and angle, but not positional info on the hinge that could be combined with Window Segments
… also a topic, whether multiple Screen objects or a single Screen should be exposed, is a big open question for Screen Fold API especially
… would like to see an effort to look into aligning Screen Fold API and Window Segments
… Zouhir, thoughts on Screen Fold API and its alignment with Window Segments?
Zouhir: both the APIs target the same class of web sites, that want to utilize region information, leverage postures such as tent or book mode
… controls on the site such as YY might want to put the controls on the bottom region, and video on the top region
… from this perspective, these APIs enhance the UX for the same class of web apps and sites
… I don't yet have a position on aligning with Multi-Screen Windows Placement
… e.g. Outlook email app might want to lay out the email list on one side, compose / read view on the other
Alexis: on the other side of spectrum, devices for which the Screen Fold API is designed for, we don't have a big screen with foldable screen, but bigger devices are coming in the future, in this class of devices there's research in virtual hinge that the user can drop in the middle and the OS reacts and provides means to snap content on left and right side of the virtual hinge
Zouhir: question, why Screen Fold API is not like a generic sensor API, expose just an angle w/o posture?
Alexis: posture is a convenience
Jemma: ARIA co-chair, these proposals are really cool, have you thought of accessibility in design of these APIs?
Alexis: for dual-screen devices, the application itself spans across the screens, if you run two apps side-by-side it is similar to a desktop running two apps side-by-side, it is a single viewport so accessibility features should work
Zouhir: great questions, having two apps side-by-side is a key use case advertised for these devices
… we follow window manager and OS behavior, need to lay out the Web APIs on top of that foundation, rely on OS and window manager-provided accessibility hooks
… both Android and Windows 10X have great accessibility support for dragging and dropping etc. in a browser we do not have any control over that
Zouhir: if the browser is not spanned, on one side only, it means you're in a mobile responsive scenario and then span across two screens
… it would be interesting to see what patterns emerge, what knobs and primitives developers need
… think of a Google Doc use case, when you span you might have comments on the other screen
smfr: important accessibility feature is to be able to make things bigger, how in dual-screen devices the content is behaving in such a scenario when content is zoomed?
Zouhir: this needs its own breakout session :)
… when you zoom e.g. notch does not change, on dual-screen devices, when zooming, in the current prototype, we do not change the information re where the fold is, Daniel Libby is probably looking into this issue
[ accessibility issue should be filed at https://github.com/webscreens/window-segments/issues/ ]
Mike: how segment info should change when the posture or orientation of the screen changes?
… API does not have any special events?
… to give updated information, need to listen to orientation(?)/resize events
fantasai: my expectation is the pinch zoom does not change the layout?
<fantasai> fantasai: but layout-affecting zoom should change the values
Zouhir: great point, Daniel discussed this in CSS WG I believe, and Daniel will take an action
Mike: I wanted to clarify whether watching the window.onresize event would be enough?
<msw> please go on, i will try to fix it
Zouhir: while Mike fixes his mic issue, any questions?
Alexis: orientationchange event, is it enough or should we have its own event for segment changes?
Zouhir: if there's a use case, happy to discuss, I haven't yet seem a scenario for such an event
anssik: Good practice: "When in doubt, leave it out"
… One way to make progress is to put all the use cases on the table and see how they overlap
… Not easy for someone not familiar to see the full picture.
Zouhir: I agree with that
… some of the use cases we trying to solve here, in general there are some UX patterns that work better in wider viewing areas, in pocketable devices people think it is great to see e.g. list and reader view in email reader
… another example, Instragram, posting a photo, you want to edit the photo with edit controls on one side
… we do have these experiences documented, one thing I want to say, most of our current customers want to see these experiences on these portable form factors
… no requests yet for multi-monitor use cases
… I'd be interested in hearing whether these use cases should scale to desktop class of devices
… or Multi-Screen Window Placement API happen on a portable form factors
Mike: use cases for Multi-Screen Window Placement, request full screen on split screen
… start presentation on another screen, display additional info on the another screen
… also dashboard use cases, window across multiple displays
… creative scenarios with palette, edit controls on secondary screen
… dashboards showing e.g. stock tickers across multiple windows
… digital signage as well, and many more
Zouhir: want to let develops split the viewport across hardware screen, so one window and one viewport, in contrast to multi-screen scenario different child windows
Mike: right, Multi-Screen not targeted at spanning across displays
… w/o innerTop/Left, information about content relationship is unavailable, you don't have the positional information
… goes back to fundamental idea, your screen as you mentioned earlier, windows segment information is more akin to where is the punchhole camera, or safe rendering area
… in that theme, it might be reasonable to expect this info to be available without a permission
… whereas in Multi-Screen Window Placement API, we gate it with a permission
… it'd be challenging to merge these two proposals, not impossible
Alexis: mobile phone form factor now prevalent, but we'll see evolution where mobile phone might be connected to an external screen
… as a developer I might want to present to an externally connected screen but my primary window is on a foldable devices
… query available screens to know how many segments are available, something to keep in mind how to cater for this use case
Mike: agree, would be good to think more of these paradigms
Align terminology across related specs
Mike: couple of trivial examples, screens vs. displays, tricky to get right, not a 1:1 relationship!
… important to think in terms of legacy of the web
… you can access window.screen now
… if foldable is composed of multiple such screen objects, having access to both of those?
Anssi: happy to host a semi frequent virtual meetings to continue discuss these in the Second Screen group
… coordinate with CSS WG
Zouhir: we want to continue align with existing stuff
<fantasai> https://www.w3.org/TR/css-grid-1/#intro
fantasai: Flex and Grid differences and key use cases explainer
<fantasai> So extending https://www.w3.org/TR/cssom-view/#the-screen-interface ?
Yes, see https://webscreens.github.io/window-placement/#dictdef-screeninfo
Or, "Shape roughly matches", not extending per se
<fantasai> CSSWG discussion list - https://lists.w3.org/Archives/Public/www-style/