W3C

Second Screen WG F2F - Day 1/2

22 Sep 2016

Agenda

See also: IRC log

Attendees

Present
Anton_Vayvod(Google), Mark_Foltz(Google), Anssi_Kostiainen(Intel), Shih-Chiang_Chien(Mozilla), Mounir_Lamouri(Google), Eric_Carlson(Apple), Mark_Watson(Netflix), Louay_Bassbouss(Fraunhofer), Francois_Daoust(W3C), Hyojin_Song(LGE), Kenneth_Christiansen(Intel), Hiroki_Endo(NHK), Ingar_Arntzen(MotionCorporation), Tomoyuki_Shimizu(KDDI), Chris_Needham(BBC), Kasar_Masood(Viacom), Kumanan_Yogaratnam(Espial)
Chair
Anssi
Scribe
Francois, Mark

Contents

See also the minutes of day 2.


Anssi: Welcome to the Second Screen WG 2016 TPAC F2F meeting!
... [agenda review]
... v1 issues for the Presentation API are the ones we want to fix right away to move the Presentation API to REC. v2 issues would be for a possible second version of the specification.

Anssi: The Presentation API spec got published as a Candidate Recommendation in July 2016 (see Presentation API publication history).
... We'll shuffle the agenda a bit to discuss everything related to v1 first, then talk about v2.
... We may also compact the agenda a bit to spend more time on CG discussions around protocols.

[quick round of introductions]

Presentation API issues

-> Presentation API open issues

Francois: Note some recent issues are not labeled as v1 or v2.

Anssi: Let's go through them

Clarify presentation ID generation requirements (#346)

-> Clarify presentation ID generation requirements (#346)

Mark: We "recommend" to use UUID, We'd better use "SHOULD" to clarify that this is a normative statement.

Francois: Why is that not a MUST? Do we anticipate good reasons to create other types of UUID?

Mark: I think it can be a MUST. I think I can take an action to research what other specs are doing.

Shih-Chiang: Just checked MediaStreams spec, they use SHOULD.

<scribe> ACTION: For #346, @mfoltzgoogle to check how other specs reference UUID generation and adjust the spec accordingly with SHOULD or MUST. [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action01]

Introduction could use some copy editing (#345)

-> Introduction could use some copy editing (#345)

Mark: Purely editorial.

Spec for PresentationConnectionCloseEvent conflicts with note (#344)

-> Spec for PresentationConnectionCloseEvent conflicts with note (#344)

Mark: The examples in the note do not include the reason why the error occurred, whereas we recommend that.
... We should update the examples in the note.

<mfoltzgoogle> ACTION: For #344, Mark to update close message examples to conform to spec. [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action04]

Establishing a presentation connection steps do not handle failures (#343)

-> Establishing a presentation connection steps do not handle failures (#343)

Mark: I'm suggesting moving the error handling part from the start connection algorithm into the establish a connection algorithm so that the reconnect algorithm also benefits from it.
... Also, the note in 6.5.1 is not explicit about which steps can be automatically retried.

Anssi: Are there cases where implementations may not want to retry?

Mark: That's a good question
... It depends on whether we want the developer to be fully responsible for error handling.

Anssi: I think good general principle is that the developer should always expect things to fail. Do we have implementation feedback?

Mark: I don't think the connection will ever go from an error state to a connected state without the developer having to do something about it.

Anssi: OK, so that's per spec. The note is more an implementation guideline.
... My mental model says that it's more intuitive to expose the error right away to the developer. That would suggest dropping the note.

Mark: I think I agree. I need to check whether that matches implementation.

Anssi: Similar to a discussion we had in the Device and Sensors WG around transient states.
... Anyone has concerns with dropping the note?

RESOLUTION: For #343, move error handling steps from the start algorithm to the establish a connection algorithm, and remove the note in 6.5.1 about retrying connection establishment.

Presentation display availability is not clearly defined (#342)

-> Presentation display availability is not clearly defined (#342)

Mark: I wonder whether that's an idiom we need to define, or is it just a reference to the object that we need.

Anssi: I think that's just a way to have an English name. This seems like a style issue. I would leave it to the editor.

Mark: We may need some idiom to say exactly what this means.

Francois: Personally, I agree it would be better to have a proper definition. I don't think there's a clear guideline across Web specs on this.

Mark: I do want to check that there is some proper definition of what availability means.

<mfoltzgoogle> ACTION: For #342, Mark to check that there is a proper definition of "availability." [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action03]

navigator.presentation is marked optional in WebIDL (#341)

-> navigator.presentation is marked optional in WebIDL (#341)

Mark: I think that's an IDL bug, the presentation attribute cannot be null in practice.

RESOLUTION: For #341, make "navigator.presentation" attribute mandatory to align with the normative prose.

Examples 5.6 does not need onconnect handler (#340)

-> Examples 5.6 does not need onconnect handler (#340)

Anssi: Editorial, needs fixing.

setConnection in example 5.5 may put UI in an inconsistent state (#339)

-> setConnection in example 5.5 may put UI in an inconsistent state (#339)

<scribe> ACTION: Kenneth to review and fix the example in 5.5 [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action02]

PresentationConnectionList is not required for receiving user agents (#337)

-> PresentationConnectionList is not required for receiving user agents (#337)

Mark: The Conformance section lists mandatory interfaces, but PresentationConnectionList fell through the cracks.

RESOLUTION: For #337, add PresentationConnectionList to the list of interfaces to be supported by receiving user agents.

Anssi: Moving on to older v1 issues

PresentationRequest constructor uses entry settings object (#336)

-> PresentationRequest constructor uses entry settings object (#336)

Anssi: Basically, things changed in HTML, and we should align.

[looking at another spec which fixed this, seems to use "incumbent object"]

Anssi: We need to look at how other specs have been refactored and follow that practice.

Francois: We could add ourselves to the list of specs tracked to reduce the use of "entry settings object" and ping Domenic about that.

RESOLUTION: For #336, examine how other specs have been updated to use the settings object and update our spec to match.

Define the behavior for consecutive calls to getAvailability (#335)

-> Define the behavior for consecutive calls to getAvailability (#335)

Shih-Chiang: The availability object will be created during the first call. Different unclear behavior. When are promises resolved when consecutive calls to getAvailability are issued?

Anssi: In the battery context, multiple calls will yield the same Promise (see Battery promise).

Mounir: I think we should add a new step after step 6 to reuse an existing Promise if there's one.

Anssi: that would be a one-liner fix

Francois: About the first bullet, my understanding is that the problem is that different contexts cannot return the same JS object.

Mark: Yes, they would have to be different JS objects. I think we need to be clear about when we refer to script objects or internal objects.
... I'll check our implementation but indeed, different calls to getAvailability from different contexts will return different script objects that are in the same internal state.
... I think we should check when we say promise or availability object whether we refer to a script object or an internal state, and make that consistent throughout the spec.
... The IDL has [SameObject], so enforces the fact that the same script object gets returned in a given browsing context.
... Actually the [SameObject] attribute is not there, so we need to be explicit about that.

Francois: Also, note that for bullet 2, we're missing "in parallel" in step 9.

RESOLUTION: For #335, check algorithms around availability objects and make sure they all refer to "script object" or "internal object". For bullet 2, add "in parallel" in step 9. For bullet 3, make sure getAvailability returns the same Promise object.

Define that "TV" should appear as token for UA-string (#303)

-> Define that "TV" should appear as token for UA-string (#303)

Anssi: Jonas is not around for the time being.
... Do we have new information?

Mark: I'm fine either way, provided we understand what TV means. There seems to be a set of features, e.g. overscan, input, resolution, etc.

Kenneth: It's not recommended to do UA sniffing at well.
... Reality is that people have to do sniffing from time to time.

Anton: In the receiving context, the "receiver" attribute will be set, so the application knows it is a presentation.

Francois: Right, that's what I raised in the GitHub issue as well.

Anssi: I see, so that seems functionally equivalent.
... An app can do feature detection through that attribute and react based on this. Additional specific CSS media queries could be useful in the future.

RESOLUTION: For #303, add a note about feature detection through the "receiver" attribute, no guideline on the UA string.

Check references category and stability (#295)

-> Check references category and stability (#295)

Francois: Basic rule is that a PR/REC cannot reference anything else than a PR or a REC. In practice, there are exceptions to the rule. We'll switch to HTML5.1, have a slight problem with NotAllowedError in WebIDL, mixed content and permissions, as well as a couple of specs references in the "Creating a receiving browsing context" algorithm, but we're only referencing high-level concepts there, no deep integration.
... We just need to keep track of those, and ensure we highlight exceptions we need when we request transition to PR.

Presentation API testing

Anssi: We need to demonstrate interop to go to PR, and for that we need to create a test suite.
... We should look at the current state and see what needs to be done.
... Louay is test facilitator. Some contributions from students, WG participants, Francois prepared a human-readable test coverage document.

Anssi: Proposed action plan for how to proceed with testing. Three milestones to complete tests.

Anssi: End of October, work on existing tests and fix open issues. Four minor issues.
... For each missing test, create an issue in web-platform-tests/presentation-api. Assign issues to test writers.
... If there is a normative change, editor should ping Louay and the test author that tests need to be updated.
... Need to move Chromecast app ids to a config file to avoid duplication.
... Two web pages are registered with Fraunhofer accounts. Not allowed to publish with our account.
... Need account to maintain published apps.
... W3C can create account to publish.

<scribe> ACTION: Francois to investigate creation of Cast developer account to publish test app. [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action05]

Louay: How to test different implementations (1-UA and 2-UA) in the same user agent and present a consolidated report.

Anssi: What is the status of 1UA?

<anssik> https://tidoust.github.io/presentation-api-testcoverage/#table

Mark: The bread of the sandwich is done. The yummy parts in the middle are in progress.

Anssi: How to forward JSON test results from receiver?

Anssi/Mark: Cleaner to POST these to a separate service than to use the presentation connection.

Mark_Watson: Test harness spawns windows, may not work in a single-window environment.

Louay: Controller runs test on receiver, so don't need to spawn windows.

Francois/Louay review test coverage report. Goal is to prepare implementation report by the end of the December.

Anssi: Move coverage report to w3c repo?

Francois: Not part of test suite. Anssi: Leave as-is.

<tidoust> [FYI, iframe tests use a stash.py file to store test results on the server]

Louay: Can check scripts to stash results. Or use WebSockets to exchange test results.

Anton: Would a test receiver help that opens a window? Mark: Mocks out the receiver, could be useful for testing controller.

Louay: Call for test contributors. Tomoyuki: Offers to help.

Implementation status

Mark presents Google Cast for Education, intended for class rooms. Could be used for 1-UA mode as well as for 2-UA mode

Virtual screen access control goes through cloud. Mark gives access to Anton, who shares images. Implementation is a proof of concept not a full 2-UA implementation, it does not support multiple controllers for instance. The receiving API needs to be implemented.

The spec was not an obstacle to implement this. There's a little bit of work to do around security.

The device runs Chrome OS. The stack is different from the stack in Chromecast. The receiver part is a Chrome app. Being a Chrome app helps leveraging some useful features such as user login.

Mark mentioned the fact that Chrome shipped support for the Cast API which is built on top of the Presentation API under the hood.

The only plan for native implementation of receiver mode is 1-UA mode for Google.

Shih-Chiang shows a demo of Mozilla's implementation of the Presentation API using a Firefox OS TV simulator.

Device selection part is currently missing from the implementation. The TV simulator gets told to open the Web page. It exposes the receiving API to the Web page that gets loaded.

Louay wonders whether the TV simulator can be used for testing the receiving side. The TV simulator is not publicly available.

<Tomoyuki> [FYI: Simulating Firefox OS for TV on your desktop (MDN)]

Tomoyuki sees a public simulator, but it is an old version of the simulator.

Kumanan wonders about underlying protocols. Proprietary protocols are used. Topic will be discussed in the Second Screen CG tomorrow. Mark points out that Google Cast for Education does not really use the Cast protocol despite the name, but a cloud-based protocol.

Moving beyond CR

Anssi reviews the requirements to transition beyond CR.

A discussion around re-chartering milestones follows. Is Q1 2017 too optimistic? If the group needs to publish a new CR, it only has until mid-January 2017.

Shih-Chiang mentions Q1 2017 as implementation target for 1-UA mode. Mark says he targets end of the year for 1-UA mode as well. 2-UA mode is not going to be until Q1 2017.

2-UA mode is part of our CR exit criteria. It is work in progress in Firefox OS for TV, with no clear ETA for shipping at this stage.

Participants agree that Q2 2017 seems a better target for completion.

[lunch break]

Presentation API v2

Anssi: Goal is to take version 1, fork it and start baking in new features.
... Let's look at v2 issues and see whether that triggers "excitment".

Cloud paired screens as presentation targets (#61)

-> Cloud paired screens as presentation targets (#61)

Mark: I think flashing out the requirements to do that would be useful.

Shih-Chiang: All of our envisioned developments so far target the local network. It would be useful to let the user decide on whether it allows a cloud-based screen that the app can interact with instead of trying to agree on a mechanism to achieve that.
... WebRTC already has a mechanism for pairing two devices, maybe it would be easier for us to leverage this.
... i.e. reuse the signaling channel.

Mark: Also, it could be useful to repurpose an RTCDataChannel for use in the Presentation API not for developers to change it.
... Worth investigating

Forced 1-UA mode for documents or frames (#347)

-> Forced 1-UA mode for documents or frames (#347)

Mark: In some cases, the controller may be the powerful end, and power may be needed. Or low-latency is needed (e.g. in gaming)
... Two possible options: 1/ a possibility to require 1-UA mode, 2/ a new API to present an offscreen frame or canvas (being able to access and control the remote DOM tree)
... The second part could be similar to requestFullscreen. A kind of requestOffscreen. It's like a simpler use case for the Presentation API.
... There's already some work being done in Chrome at least to turn a canvas into a MediaStream, so the first part could build on it.

Anssi: That reminds me of the Offscreen canvas proposal discussed in WHAT WG.

Allowing a page to turn itself in a presentation session (#32)

-> Allowing a page to turn itself in a presentation session (#32)

Mark: I think this use case is interesting. A device could pass by and become a controlling user agent for a presentation initially started on its own.

Anssi: How would you bootstrap this? NFC tapping?

Mark: It depends on context. For public devices, restrictions are obviously needed.

Can the same browsing context act as a controller and a receiver? (#338)

-> Can the same browsing context act as a controller and a receiver? (#338)

Mark: Problem is around the restrictions the spec places on a receiving browsing context.
... The spec does not prevent that case, but we may need to check the implications carefully.
... Also, you may need a way to delegate permissions if the receiving user agent does not support user input as such.

Francois: The use case is somewhat different from a controlling page willing to spawn a presentation on "as many screens as possible".

Mark: Somewhat related though. The first presentation could spawn another presentation. There are pros and cons to both approaches.

Louay: The video wall use case comes to mind.

Ingar: Sync in this use case is typically within the scope of the Multi-Device Timing CG. That does not solve the bootstrapping problem.

Compatibility with HbbTV (#67)

-> Compatibility with HbbTV (#67)

Francois: Part of CG discussions now.

Presentations without communication channel (#202)

-> Presentations without communication channel (#202)

Francois: Some overlap with the "offscreen" idea.

Francois: This could allow for some bootstrapping scenarios, e.g. where the user uses NFC tapping to pass the URL. No communication channel can be established, but both ends can go through the backend to communicate.

Presentation display capability detection (#348)

-> Presentation display capability detection (#348)

<mfoltzgoogle> Media capture from canvas

Mark: There may be different categories such as physical attributes, media capabilities, support for specific APIs (MSE, EME) or codecs, etc.

Anssi: HTMLMediaElement, getUserMedia come to mind for capabilities detection.

Mark: Higher priority should be around features that are independent of the content, so I/O characteristics typically.
... It might be expressed preferences. Controller could say "I would work better if I had this or that capability". Usability enhancement.

Mounir: Having that as preferences has issues. You may want to ensure that you don't get some of the remote devices show up in the list.

Mounir and Mark discuss preferences vs. constraints.

One issue is that this would leak more information to the app through "getAvailability".

Anssi: A good approach would be to look at concrete use cases. Keeping it simple is much better. "Input", etc.

Chris: When I discussed this internally, the discussion quickly turned into codecs.

Mounir: It's more an issue for the Remote Playback API.

Anssi: Looking at the streams API now.

Mark: Domenic and I look at it as streams evolve. No concrete action so far.

Anssi: Concluding this discussion, there seems to be valid use cases for v2. None of this should make its way to v1.
... Our new draft charter would allow us to work on this.

<mfoltzgoogle> FYI: <canvas>.captureStream() has shipped in Chrome.

Remote Playback API

Anssi: Looking at post-FPWD issues.

Implementation guidance for browsers that a media element with controls is remoted (#48)

-> Implementation guidance for browsers that a media element with controls is remoted (#48)

Anton: Some suggestion to convey that remoting is happening and that some poster image should be displayed.

Anssi: I think different implementations convey things differently. We can add a note.

Mark: From a privacy point of view, it seems important to ensure the user is aware that something is happening remotely.

Chris: Or would you have an event about remoting changes of state?

Anton: You will get events.

Anssi: The user should always be notified about what is happening.
... Generally, it is a bad idea to specify anything specific to UI in a spec.

Anton: The spec does not say whether the local playback should be paused.

Anssi: I would not force the local playback to pause. That can be useful in some cases.

Francois: It seems important to ensure that the app developer knows whether the local playback is still running or not.
... Also, you probably do not want the local audio rendering to continue, because synchronization will be very hard to achieve.

Anton: Except in the case where you want to watch a video in two different rooms, you probably want to pause local playback.

Anssi: So should we expose a way to know whether the local playback is still running?

Mounir: I think the spec should be explicit that local playback needs to be paused.

Francois: Yes, if the developer wants to continue playback, he can create another media element and handle the synchronization on his own.

Anssi: OK.

Anton: From the point of view of the page, the media will still be playing.

Define the relation with Media Session spec (#10)

-> Define the relation with Media Session spec (#10)

Anton: We're waiting on media session to move forward.

Mounir: We re-scoped this effort and transitioned to WICG.

Anton: So only action is to wait until the spec matures and revisits.

Guidance for HTMLMediaElement (#41)

-> Guidance for HTMLMediaElement (#41)

Anton: The issue is that media element is like a big interface. What should work and what should not work in the remote state?
... I've started to list what needs or does not need to be supported.

Francois: Interesting, because it looks like a "profile" of the HTMLMediaElement interface, which makes perfect sense, but may not please some people.

Kumanan: Feature detection is already used to detect whether some of these features are supported in the simple local case.
... You may not be able to seek for instance, and you cannot ask more than what the remote device can achieve.
... In the set-top box case, there may be legal restrictions for some of these features.

Anton: OK, so we should be careful for some of these MUST.
... we'll get to details tomorrow.

See the minutes of day 2.

Summary of Action Items

[NEW] ACTION: For #346, @mfoltzgoogle to check how other specs reference UUID generation and adjust the spec accordingly with SHOULD or MUST. [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action01]
[NEW] ACTION: For #342, Mark to check that there is a proper definition of "availability." [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action03]
[NEW] ACTION: For #344, Mark to update close message examples to conform to spec. [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action04]
[NEW] ACTION: Francois to investigate creation of Cast developer account to publish test app. [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action05]
[NEW] ACTION: Kenneth to review and fix the example in 5.5 [recorded in http://www.w3.org/2016/09/22-webscreens-minutes.html#action02]
 

Summary of Resolutions

  1. For #343, move error handling steps from the start algorithm to the establish a connection algorithm, and remove the note in 6.5.1 about retrying connection establishment.
  2. For #341, make "navigator.presentation" attribute mandatory to align with the normative prose.
  3. For #337, add PresentationConnectionList to the list of interfaces to be supported by receiving user agents.
  4. For #336, examine how other specs have been updated to use the settings object and update our spec to match.
  5. For #335, check algorithms around availability objects and make sure they all refer to "script object" or "internal object". For bullet 2, add "in parallel" in step 9. For bullet 3, make sure getAvailability returns the same Promise object.
  6. For #303, add a note about feature detection through the "receiver" attribute, no guideline on the UA string.
[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.144 (CVS log)
$Date: 2016/09/26 11:11:13 $