04:58:10 RRSAgent has joined #webscreens 04:58:10 logging to https://www.w3.org/2020/10/21-webscreens-irc 04:58:13 Zakim has joined #webscreens 04:58:23 Meeting: Second Screen F2F - Day 2/2 04:58:31 Chair: Anssi 04:58:34 RRSagent, make logs public 05:00:01 oyiptong has joined #webscreens 05:00:10 Agenda: https://github.com/w3c/secondscreen-wg/issues/1 05:00:16 present+ 05:01:57 mfoltzgoogle has joined #webscreens 05:02:08 zouhir has joined #webscreens 05:02:18 pwnall has joined #webscreens 05:02:28 present+ Victor Costan (Google) 05:03:04 present+ Mark Foltz (Google) 05:03:18 darktears has joined #webscreens 05:03:22 present+ Olivier Yiptong (Google) 05:03:23 present+ Mike Wasserman (Google) 05:03:57 present+ Alexis Menard (Intel) 05:04:01 masaya_ikeo_nhk has joined #webscreens 05:04:28 zouhir_ has joined #webscreens 05:04:57 amandy_ has joined #webscreens 05:05:02 present+ Takumi Fujimoto (Google) 05:05:29 RRSAgent, draft minutes v2 05:05:29 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html anssik 05:06:01 eric_carlson__ has joined #webscreens 05:08:10 eric_carlson___ has joined #webscreens 05:08:10 Present+ Alexis_Menard, Anssi_Kostiainen, Zouhir, Takumi_Fujimoto, Mike_Wasserman, Srikanth, Kenneth_Christiansen, Mark_Foltz, Eric_Carson, Arno_Mandy, Laura_Morinigo, Olivier_Yiptong, Victor_Costan 05:08:22 Regrets+ Chris_Needham, Francois_Daoust 05:08:27 RRSAgent, draft minutes v2 05:08:28 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html anssik 05:08:42 Scribe: Mark 05:08:46 Laura has joined #webscreens 05:08:49 scribeNick: mfoltzgoogle 05:09:39 Topic: Foldable use cases and requirements 05:09:47 kenchris has joined #webscreens 05:09:56 +present Kenneth_Christiansen 05:10:35 +present Laura_Morinigo 05:10:57 anssik: Today's meeting will be focused on exploratory work in the SSCG and in related groups. 05:11:19 ... I have asked Srikant to talk to us about new form factors in upcoming devices, 05:11:29 ... how they should be considered in new Web APIs. 05:12:32 Srikanth: Presentating on Web Support for New Device Features 05:12:59 ... Long history in VESA, HDMI, Wi-Fi display, other bodies. 05:14:07 ... There are multiple new form factors. Foldable displays with multiple independent display panels. 05:14:22 ... Ranging from laptop size to smaller sizes, like a mobile phone. 05:14:35 [ Srikanth presenting slides ] 05:14:44 [ MID = Multiple Independent Displays ] 05:15:15 ... Intel Horsehoe Bend, at CES earlier this year. Supports multiple configurations: laptop, table-top, flat, canvas or book 05:15:56 ... Depending on the amount of incline, you can lay it all the way flat, for input or drawing. 05:16:15 ... Your user experience will differ, and the camera and speaker are reconfigured accordingly. 05:16:26 RRSAgent, draft minutes v2 05:16:26 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html mfoltzgoogle 05:18:05 Present+ Wooglae_Kim 05:18:21 scribenick: mfoltzgoogle 05:18:56 Srikanth: shows a video demonstrating capabilities of the foldable laptop. 05:19:39 s/Independent/Internal 05:19:48 ... This is not a new concept. [Shows a slide of the Lenovo X1 Fold on the MID platform] 05:21:07 ... Even if you have a MID, the emphasis is on fold. Both are posture-based. User experience and resources will differ. 05:21:37 ... Many of these are supported in W3C today, but how to provide a seamless, consistent experience. 05:22:21 ... Consistency in how Web applications determine what the posture is. Posture enforcement as necessary. 05:22:33 ... Want this regardless of foldable vs. MID. 05:23:31 ... [Shows a device with a physical hinge]. This device can with a sharper fold; can have a 360 degree bend. 05:24:18 ... Devices without hinge, there is a less acute fold because of the physical layers of the display. Can't completely fold (0 or 360) 05:24:45 ... Number of displays, we are seeing two, but there could be more. Could be panels of unequal size, or three different panels. 05:25:07 ... Do the interface once, have these webapps work and be foreward compatible. 05:26:06 ... Foldables, there is typically one display. It depends on the display configuration, could be one or more depending on the posture. 05:26:42 kim_wooglae has joined #webscreens 05:26:44 ... Seam is always present on a MID. In a foldable, there is a virtual hinge, it's a user experience decision. 05:27:12 kim_wooglae_ has joined #webscreens 05:27:13 ... The beauty of a virtual hinge is it's a software abstraction. 05:27:38 ... Touch controller is associated with each panel. Is configurable to generate various HID reports. 05:27:51 RRSAgent, draft minutes v2 05:27:51 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html mfoltzgoogle 05:29:14 Srikanth: [Table showing attributes of MID vs. Foldables] 05:30:16 ... If you have an enlightened app, aware of the seam, it can adjust layout according to the seams in the display. 05:31:03 ... This is an example of how a seam can help the web app to choose. 05:31:47 ... [Slide shows how apps could customize themselves for different foldable postures] 05:33:12 ... Key point is, apps can benefit from other information, posture, occlusion, etc. 05:34:02 Alexis: New UX patterns as you use different postures on the device. 05:35:32 ... Show you kinds of problems you might get on web applications. We designed a gallery to respond to multiple displays. 05:35:46 ... Not to just adapt their app, but based on posture, provide a better experience. 05:36:33 ... Don't want to position across the hinge. 05:37:32 ... The current thing that we see, is covering the new form factors.I need the screen fold API to determine 05:37:51 ... If I am in tent mode. They are complementary, but I can't use them in a consistent manner, across the OS. 05:38:27 ... Next steps, MID are different from foldables. Different from an externally connected displays. 05:38:54 ... Whatever specification we put up, should cover these types of devices. 05:39:32 anssik: The future looks quite exciting, when it comes to new form factors. We should be designing new APIs in a future-proof manner. 05:39:58 ... This is the beginning of the discussion, but want it to be a kick-off for this type of exercise. 05:40:30 q+ 05:40:36 ack mfoltzgoogle 05:40:41 q+ 05:41:32 Mark: certain type of single displays can do virtual multiple displays, how does this work out with that, has that been considered a use case? 05:42:00 Srikanth: That is a use case as well. For example, you can use docked devices. 05:42:38 ... You can have software abstractions that virtualize the hardware. I fully expect the platforms to have multiple docked scenarios. 05:42:51 q+ 05:42:57 ack zouhir_ 05:43:21 q? 05:44:00 zouhir_: We have an API to get the geometry of each physical display, an API with an enum of postures. 05:44:27 ... Developer has to use two APIs to get geometry and posture. 05:44:49 Alexis: Edge doesn't allow me to get the posture, that's the inconstency. 05:45:22 zouhir_: Incomplete implementations, by having them separate, some browsers may not implement one or the other. 05:45:36 s/zouhir_/.../ 05:46:31 Srikanth: Many low-level details are exposed to the application, applications have to put them together to get the big picture. 05:47:03 ... Different applications may interpret them differently. What sort of things are okay to allow some inconsistency. 05:47:06 ack msw 05:47:51 msw: Do you wish to generalize the foldables API to handle additional use cases? Like MID / 360 hinges? 05:48:16 Srikanth: Given all the possibilities in the underlying hardware. What is the right abstraction to expose to webapps? 05:49:10 ... Tomorrow, if there are 4 internal panels, foldable in different ways, do we want to re-do some APIs, or create full abstractions? 05:49:35 kenchris: It has to be simple as possible for the common use cases, or developers won't adopt it. 05:50:19 anssik: Try to understand what are the experiences they want to build, and optimize for the common scenarios. 05:50:31 ... Make the less common thing possible, but maybe not as easy. 05:50:35 ack 05:50:37 q? 05:54:00 Topic: Multi-Screen Window Placement 05:54:14 msw: [Shares slides] 05:54:28 RRSAgent, draft minutes v2 05:54:28 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html mfoltzgoogle 05:54:42 msw: This proposal is related to foldables. Addresses a generalized set of use cases. 05:55:20 ... Generalize Web Platform for multi-screen displays. 05:55:37 ... [Slide comparing existing Web APIs and goals of MSWP] 05:55:57 ... [Slides showing examples of multiple-display devices, projectors, signage, lots of use cases] 05:56:27 Goals: Increasing initial information about additional displays; adding a change event when they change; 05:56:53 ... Request additional per-screen info; sites can request fullscreen on another window 05:57:27 ... Slide shows current properties exposed through CSSOM 05:58:05 ... Proposal is to allow sites to request additional information about connected screens. 05:58:27 ... Currently you can use window.open to create new windows, with left/top coordinates 05:58:35 ... Can move to/by (x,y) coordinates 05:59:20 ... Crux of the proposal is to allow cross-screen coordinates for these operations. 05:59:34 msw: One set of updates is that we have an ongoing origin trial with prototype code. 05:59:44 ... Addressing some good feedback, including some API shape changes. 06:00:37 ... There is a demo: https://window-placement.glitch.me 06:00:54 ... More info: https://web.dev/multi-screen-window-placement 06:01:13 ... Github: https://michaelwasserman.github.io/window-placement-demo 06:01:28 q? 06:02:38 https://docs.google.com/presentation/d/1Liom19iRqyuTvMYjygzLSj6cYa0xUfvvZDTgabQsPus/edit#slide=id.ga0ed95d99b_1_43 06:03:20 q 06:03:23 eric_carlson has joined #webscreens 06:03:34 msw: We have interested partners in this, they want to explore a new space. 06:03:52 ... We have Chrome apps APIs and OS APIs, we want to offer the same affordances to the Web. 06:04:09 q+ 06:04:40 anssik: You are catering for the established usages. Your use cases are more clearly defined. 06:04:51 ... For foldables, it's not as clear. 06:04:54 q+ 06:05:20 ... How much we can unify MSWP, and some of these new proposals. It might not be 100% overlap. 06:05:37 ack eric_carlson 06:06:16 Eric: What you've done to mitigate the many bits of entropy about users with non-standard configurations? 06:07:20 msw: Similar information is exposed if windows on placed on multiple screens. We are also proposing a permission that user agents can use to gate access. 06:07:30 Eric: What is exposed without a prompt? 06:07:59 msw: A single bit whether there is an external screen, so sites know whether to prompt multiscreen users. 06:09:33 Mark: Presentation API is slightly different in terms of availability detection, the Presentation API does not expose any info on external display, Remote Playback is similar 06:09:51 q? 06:09:57 ack zouhir_ 06:11:07 Zouhir: Want to work to create an API surface that is minimal, can talk again about how make these APIs work together. 06:12:04 ... Regarding privacy, if permissions is the way to protect customers, and I want to make sure my YouTube controls are laid out right. 06:12:34 ... Don't want to show a bad experience, then require a permissions prompt to show a good experience. 06:13:22 msw: Good opportunity to collaborate on an API surface. 06:14:19 ... Permissions are tricky, Web platform was built so that users can see screen info. If you install an app, it can access screens and system info. 06:14:47 q? 06:15:05 ... Possibly we could expose screens that a user is spread across, without requesting permissions. 06:15:33 s/user/application/ 06:16:50 https://docs.google.com/presentation/d/1Liom19iRqyuTvMYjygzLSj6cYa0xUfvvZDTgabQsPus 06:17:00 q? 06:17:04 anssik: Let's create follow-up actions through GitHub 06:17:23 -> https://docs.google.com/presentation/d/1Liom19iRqyuTvMYjygzLSj6cYa0xUfvvZDTgabQsPus Multi-Screen Window Placement presentation 06:17:34 q? 06:18:16 RRSAgent, draft minutes v2 06:18:17 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html mfoltzgoogle 06:26:28 Topic: Window Segments 06:27:09 Zouhir: [Presents slides] 06:27:26 ... Window Segment Enumeration Script API 06:27:42 ... Targeting dual screen and foldable devices. 06:27:55 ... Surface Duo/Neo. 2 physical screens, 1 fold. 06:28:14 ... Samsung Galaxy, 1 screen, 1 fold, 2 logical screens. 06:29:12 ... Email app example, responsive on a dual-screen, the content is not laid out in a user-friendly way. Snap content to fold. 06:29:59 ... CSS API to help developer lay out. In case CSS is not available for games and canvas, the JS API is handy. 06:30:55 ... Simple, one method. getWindowSegments() returns a list of segments. 06:31:42 ... API is an immutable snapshot, need to listen to resize events to know if it's invalid. 06:32:25 ... Difference between dual monitors and foldable. App launches is not consuming the full screen on desktop. 06:32:58 ... Maximizes, does not span dual monitors, but does on foldable. 06:33:34 ... Users want to avoid alerts, dialogs on fold. Want to move it to one side or the other. 06:34:32 ... Customers want to focus on portable form factors. 06:34:48 Zouhir: Don't want to create new, redundant APIs 06:35:03 ... Don't want a lot of API surface for developers. 06:36:16 ... Outlook, e.g. not requesting multi-monitor spanning behavior. 06:36:33 q? 06:36:38 q+ 06:36:41 ... Want the Web to be prepared for new form factors, versus sniffing around to optimize for them. 06:37:19 q+ 06:38:14 anssik: Start with the core use cases, and incrementally ship new features. "When in doubt, leave it out" 06:40:00 Zouhir: If use cases are different, e.g. multi-window applications versus app layout. Think about naming it carefully if we decide to keep APIs separate. 06:40:13 q? 06:40:20 ack msw 06:40:31 q+ 06:41:11 msw: The premise of segments, is potentially polyfillable given the intersections of screens and windows. 06:41:31 ... Are foldables doing something different than the desktop display paradigm? 06:42:21 ... There is a more general set of primitives, that could provide more information, that people could build libraries for different use cases. 06:43:30 Zouhir: We can use one API with more complex information, developers can optimize it. If I have to do calculations, developers will build a support library. 06:44:22 ... What if I am on desktop, and I span my browser. I feel like users don't expect layouts to change. Would there be surprises for users? 06:44:53 ... Looking into how we can make information available, without requiring permissions. 06:44:58 ack kenchris 06:45:02 q? 06:45:33 kenchris: It is hard to polyfill CSS. You might want to use an iPad as a secondary screen, and not be cut by a seam. 06:45:37 q- 06:45:37 ack eric_carlson 06:46:18 Eric: What do you do to mitigate the entropy that this API exposes about a non-standard configuration. 06:47:31 Zouhir: Right now we don't have permissions, developers can know a lot about the device form factor already. We have a policy to prevent iframes from getting info. 06:47:49 q+ 06:48:16 ... Gating it by a permission is providing a bad user experience at the end. 06:49:17 Eric: Agree that a user prompt isn't useful, but for perspective, we have device enumeration for getUserMedia. 06:50:07 ... When we looked at how they were being used, we found that enumerateDevices was used more than 2X times more often than getUserMedia. 06:50:15 ... s/2X/2 orders of magnitude/ 06:51:15 ... It was being used solely as a means of fingerprinting. Now, enumerateDevices returns a lie, until the user has given consent to capture. 06:52:16 ... We got feedback that it was a change that had to be made. You'll have to have a compelling argument that the bits of entropy are worth the cost. 06:53:01 Zouhir: We don't want developers to be able to achieve something natural for this class of device. Exposing the device hinge is not a big risk. 06:53:43 ... We can't lie to developers about the hinge position. We have to tell them something useful, position and dimension of hinge. 06:53:47 q? 06:54:44 anssik: What extra info does surface duo share, versus e.g. iPhone max? 06:56:28 Zouhir: You can infer device from geometry, because there are few foldables on market. For iPhones, developers need to know size e.g. of notch, to avoid rendering text on it. 06:58:12 anssik: We need to differentiate between multi-monitor setups, and other use cases. 06:58:21 ... (regarding fingerprinting possibilities) 06:58:52 Zouhir: Foldables are another responsive Web design target, I think it's something developers need to be aware of. 06:59:06 q? 06:59:53 ... As a website developer, I should be able to make it look nice. But if I have an ad iframe, it does not need this information. 06:59:58 ack darktears 07:00:42 Alexis: In the case of a dual screen, you have only one screen, so the viewport size is the sum, so without this API you can get the viewport size and aspect ratio. 07:00:58 ... We are not exposing anything, because it's viewport / 2. 07:01:41 ... We are already exposing much of this information in other ways. 07:03:02 ACTION zouhir_ to document fingerprinting concerns and mitigations in the spec 07:03:36 Zouhir: Want to respond to the discussion about libraries. Whether to have something simple, or require a library/NPM. 07:03:47 ACTION: zouhir_ to document fingerprinting concerns and mitigations in the spec 07:03:56 q? 07:04:41 ACTION: zouhir_ to open a GitHub issue to discuss use of libraries to provide window segments information 07:04:54 q? 07:05:34 RRSAgent, draft minutes v2 07:05:34 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html mfoltzgoogle 07:06:15 kenchris: Some of these devices have a physical hinge, with dual screens. 07:06:34 ... Want to change behavior based on posture. Or animate based on fold angle. 07:06:47 Topic: Screen Fold API 07:07:25 ... Expose as a ScreenFold object with a change event. 07:07:32 ... Look at different postures that might exist. 07:07:49 ... Some devices might have a second fold, or different screen sizes. 07:08:23 ... The hinge is not in the middle. Or maybe there are two hinges. 07:08:42 ... Looking at what OEMs are thinking of, dual fold devices. 07:08:59 ... Want this to be integrated into other specs. Could be used as a secondary screen. 07:09:02 present+ Francois_Daoust 07:09:20 RRSAgent, draft minutes v2 07:09:20 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html tidoust 07:09:22 ... Want to use the screen on the table for video editing. 07:09:57 kenchris: Important to standardize when things are in certain postures. What angles are they using? Talk to Samsung, Intel. 07:10:24 kenchris: Can be used in combination with other APIs. 07:10:52 s/Talk/Talked/ 07:11:34 anssik: Are we able to handle multiple, external monitors, which might be foldable? 07:11:47 ... To test how prepared we are for the future. 07:12:01 q? 07:12:05 q+ 07:12:12 kenchris: We need to integrate with Window Placement, and handle multiple folds. 07:12:33 ... I could imagine having the Intel Horseshoe, and multiple external monitors. 07:13:12 anssik: We may be able to handle this API in the future, how does the future look like? Don't paint ourselves into a corner. 07:13:14 q? 07:13:22 ack msw 07:14:07 msw: When there are multiple integrated displays, if they are separate Screen objects, how would a single fold between those screens be supported? 07:14:58 lauram has joined #webscreens 07:14:58 kenchris: We have seen this as one virtual screen. Are there any devices where the hinge separates the screen? 07:15:32 ... See if there is a hinge, but not screens right next to each other. 07:15:33 q? 07:16:12 Alexis: On the surface Duo, it will be two physical screens, two logical screens. 07:17:11 q+ 07:17:39 q? 07:18:11 anssik: Does the screen fold API address traditional form factors, that might have a fold sensor? 07:18:29 ... Does this API report the angle? 07:18:40 kenchris: Not sure if there is a sensor for these devices. 07:18:55 cpn has joined #webscreens 07:19:03 present+ Chris_Needham 07:19:11 anssik: Are there use cases for supporting this? 07:19:17 q? 07:19:22 s/anssik/kenchris/ 07:19:58 msw: Anecdotally, laptops and Chromebooks transition into tablet mode, when they fold beyond 180? 07:20:18 ack zouhir_ 07:21:41 Zouhir: Window Segment / Screen Enumeration have a similar concept, but different use case. Foldables API, complements these use cases. 07:22:07 ... Which way do we want to think about merging Window Segment. 07:22:52 kenchris: If we have a laptop with a hinge, that knows when it is converted. It's a different screen size, is there a use case for traditional laptops. 07:23:09 ... If so, it doesn't make sense for foldables. 07:23:42 anssik: Do native APIs expose hinge angle? If no, then Web content doesn't require it either. 07:24:10 Alexis: There is a sensor for the Pixelbook. It it's used to resolve the media query for hover/touch. 07:24:26 ... Web application can adapt to be touch friendly. 07:25:54 kenchris: That seems like a way to optimize for primary input. Also, on some devices, the granularity of the sensor varies. 07:26:06 q? 07:26:55 Alexis: Some devices with a stylus always report that a stylus is available, so input signals are not a good way to detect tablet mode. 07:27:11 Topic: Aligning Window Segments, Screen Fold, Multi-Screen Window Placement APIs 07:27:24 present+ Masaya_Ikeo 07:27:24 RRSAgent, draft minutes v2 07:27:24 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html mfoltzgoogle 07:28:04 q? 07:28:43 Zouhir: We have three APIs, two similar in concepts, two similar in use cases. 07:29:04 ... Don't rush and commit, open to ideas. 07:29:12 +q 07:29:45 anssik: Discuss opportunities for aligning. Make sure that these people are watching each others' work. 07:30:35 ... Useful to cross-link specs, that Screenfold and Window Segments are addressing similar applications. 07:31:36 Zouhir: Agree. We should add the foldables APIs to keep them in those use cases. 07:32:43 q? 07:33:10 ACTION: zouhir_ to cross-link foldable-related specs together 07:34:02 Zouhir: When I am dragging my window across two windows, how can web developers decide not to opt-in the foldable layout? 07:34:16 RRSAgent, draft minutes v2 07:34:16 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html anssik 07:34:59 https://github.com/SamsungInternet/Explainers/blob/master/Foldables/FoldState.md 07:35:05 https://w3c.github.io/screen-fold/ 07:35:11 msw: There are currently no window events on move. It would probably be valuable to have a move events on window. Maybe expose segments after the move loop is complete. 07:35:17 q? 07:35:20 ack msw 07:35:47 RRSAgent, draft minutes v2 07:35:47 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html anssik 07:35:57 ... Maybe only surface segment information if the window is maximized and spanning both monitors. 07:36:58 Zoushir: Different devices have different granularity, with respect to resizing. Also only maximizing, may exclude Samsung use cases. 07:37:49 ... Desktop always be opting in, unless you opt out with media queries, viewport size. 07:38:24 Present+ Masaya_Ikeo 07:38:51 msw: Part of the screen information exposed, could be whether the screens are physically adjacent. 07:41:15 ACTION: msw to reference complementary proposals of Window Segments and Screen Fold, Visual Viewport, open appropriate GH issues 07:41:21 [As part of cross-referencing specs, alignment on terms could be useful (and easy) too. I heard Alexis talk in terms of logical/physical displays. Screen Fold uses "screen", Window Segment uses "display" and "screen", Window placement uses "connected screens"] 07:41:24 q? 07:41:49 q+ 07:42:24 ack msw 07:43:04 msw: A breakout session could be the best way that we have something concrete to look forward to. 07:44:57 ACTION: msw to (with help from kenneth, zouhir, dlibby) organize a TPAC breakout session on aligning Window Segments, Screen Fold, Multi-Screen Window Placement APIs 07:45:09 https://www.w3.org/wiki/TPAC/2020/SessionIdeas 07:45:27 q? 07:45:58 q? 07:46:13 RRSAgent, draft minutes v2 07:46:13 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html anssik 07:49:23 Topic: Adjorn 07:49:34 Topic: Adjorn 07:49:40 RRSAgent, draft minutes v2 07:49:40 I have made the request to generate https://www.w3.org/2020/10/21-webscreens-minutes.html anssik 08:26:50 masaya_ikeo_nhk_ has joined #webscreens 10:34:53 Zakim has left #webscreens