See also:
Anssi: Welcome, all! [going through the overall agenda: group discussions, joint meeting with Media & Entertainment IG in the afternoon, detailed agenda in Google Docs]
… Work on the APIs started several years ago, in a stable state. Work on Open Screen Protocol has been progressing since then.
… We try to meet twice a year, at TPAC, and usually one time in between. Last meeting in May in Berlin.
… We usually make decisions during meetings and then enact the decisions in between the meetings.
mfoltzgoogle: For the joint session with Media & Entertainment IG, I didn't prepare any specific material, planning to go through generic presentation.
anssik: You might reuse the material that you're going to present during the AC meeting
… Q&A style session
… Let's do a quick round of introductions.
… I'm Anssi, from Intel, chair of both Second Screen WG and CG. Small and productive groups with well-scoped problem.
ericc: Eric from Apple. Engineer working on Webkit at Apple.
Francois: from W3C, team contact.
takumif: Takumi, from Google. Working with Mark and Peter. I have some proposals for the Presentation API that I'll present tomorrow.
Peter: from Google. Looking at making the Cast ecosystem more open.
mfoltzgoogle: from Google. Spec editor for the Presentation API. Now focused on the Open Screen Protocol. I'm the de fact editor for that. Trying to make connected displays usable by developers in general.
Staphany: from Google. Working on Screen enumeration and window placement proposal.
Mike: Picking up work started by Staphany. Related work.
Victor: from Google. TLS 1.3, interested in authentication.
David: from Google. Chair of mDNS group at IETF. Interested in discovery.
Hyojin: from LG. Been looking at integrating displays.
Daniel: from Microsoft. Working on Edge.
anssik: Thanks. Going back to the agenda, any other topic you'd like to bring to the agenda?
… We recently added a slot for Josh to present accessibility use cases.
mfoltzgoogle: Day 1 is focused on protocol work. Day 2 is to discuss API work. I invited Jer and Mounir to join morning of second day. Afternoon around planning, as we need to think about rechartering by the end of the year.
anssik: Goal is to have rough consensus on a proposal for a new charter.
Mike: Happy to present screen integration, display positioning if you think that can be useful.
anssik: Good point, we may be able to discuss it today before we close, otherwise tomorrow before planning.
… Let's add "Screen enumeration, window placement & window segments" to the agenda then.
… Daniel, any specific questions you would like us to discuss?
Daniel: Nothing at the moment. More exploring for now.
See SSWG/CG Overview & Update (PDF, p3-19)
mfoltzgoogle: We'll put a public copy of the slides after the meeting.
mfoltzgoogle: I'm going to give a lightning talk on what we've done until now. We had a productive session in Berlin where we made a number of resolutions. I'm going to mention pull requests that have been merged since then, and look at new points not addressed during Berlin.
… We landed quite a few changes to support not only remote control but also streaming.
… Looking at history: it started with the Presentation API before I joined. Creating a new Web page and presenting it in another display.
… The Presentation API quickly transitioned to a WG, now a Candidate Recommendation.
… The Remote Playback API quickly came around as well.
… Issue with interoperability prompted work on protocol.
… In Berlin, we said we were at version 1.0 of the open screen protocol, but then we made quite a few changes since then.
… The Presentation API allows a controlling web page to present a URL on a receiving device.
… The receiving device could be a separate Web renderer. For Google, Chromecast is our main target for this API. It could also be a display connected through HDMI.
… The browser discovers the displays that can be used and lets the user pick up the display that it wants. Not an enumeration API, the controlling page does not see the list.
… The connection features a communication channel that pages can use to exchange commands, or whatever.
… Both sides can close or terminate the presentation at any time.
anssik: Whatever gets presented on the second screen is considered one window without chrome. No use case where we're considering multiple windows on the second screen.
mfoltzgoogle: [Demo of a slideshow using the Presentation API]
mfoltzgoogle: The group also worked on the Remote Playback API. Similar concept but focused on audio/video remoting.
… The page can request remote playback of a media element, user selects display in a list as for the Presentation API.
… Media state is synchronized between local and remote device.
… Most implementations don't have simultaneous playback, meaning local playback stops when remote playback starts.
… Two main ways this can be implemented: one is to take the URL and send it to the remote display (media flinging), the other is to stream the media to the remote display (media remoting).
… To address interoperability issues between displays, we decided not to force an existing proprietary solution, but rather to build up an open solution. That triggered the work on the Open Screen Protocol in the Second Screen CG.
… Initial scope is devices on the same LAN. For the Presentation API, we focus on the flinging use case, not the remoting use case. Same thing for the Remote Playback API, but we've also added support for media streaming.
… The goal of the protocol is to create a fully interoperable solution between browsers and displays.
… We started by looking at functional requirements (discovery, control needs, communication channel, authentication and confidentiality, media commands)
… There are a bunch of non functional aspects that we think are important, including usability, privacy and security, efficiency, making sure that implementation is doable on constrained devices.
… And also making sure that people can extend the protocol if needed, and that the protocol can be upgraded.
… Also want to preserve compatibility with existing devices if possible.
… Looking at the protocol stack, it includes discovery, authentication, transport and message format. We looked at many alternatives.
… We tried to reuse standards that already exist, or are being worked on in IETF.
… For discovery, we decided to support mDNS and DNS-SD.
… For authentication, we chose TLS 1.3, mostly because of link with QUIC. In the absence of broadly available mechanism, we chose SPAKE2 for exchanges on keys.
… For transport, QUIC.
… For message format, a language based on CBOR for efficiency.
… Agents will do various operations, [reviewing the life of an agent]
… We require authentication as soon as possible after discovering agent name, capabilities and status.
… Bringing us to where we are today, we looked at requirements, alternatives, came to consensus with the current stack.
… We rolled up everything to a document that we called 1.0, but then we decided to add streaming, so back to 0.9.
… We've been working on an Open Screen Protocol Library implementation, ~50% done.
… There's still about a dozen issues open on our GitHub that we should resolve.
… We're going to discuss some of them today, e.g. support for TLS 1.3, enums, refinements to remote playback protocol, when can agents can disconnect/reconnect using QUIC properly, etc.
… In parallel, we worked on a TAG Explainer to describe what we're doing and why we're doing it. We need to finish it.
… Also need to look at how we support other schemes like cast: or hbbtv:. We need to finish the document that explains the pros and cons of supporting custom schemes.
… In the Open Screen Protocol Library, we implemented mDNS, QUIC, code generation tool for CBOR, support for the Presentation API, demos in C++ and Go.
… Still some work to do such as SPAKE2 implementation, evolve the protocol in terms of agent capabilities, support the remote playback protocol properly and expose an API.
anssik: No coupling into any Google services?
mfoltzgoogle: No. We do import some code from Chromium. We developed the library so that clients can inject their own implementations of something when needed.
… We're looking at integrating the Open Screen Protocol Library into chromium down the lane.
anssik: Edge Chromium uses the same WebRTC stack?
Daniel: Yes.
mfoltzgoogle: Some interesting ideas for a "v2". We'd like to discuss them to some extent here. Making it more generic to inject metadata during remote playback (GenericCue/DataCue).
… Multi-device timing/media sync, could be brought up during the joint session with the Media & Entertainment IG.
… Attestation [scribe missed explanation]
… Game controller input could be a good use case. We left in a generic data frames. Are there more specific scenarios?
… There are cases where mDNS does not work. Do we want to have backup/alternate discovery schemes?
… And a few others that we probably won't discuss during this meeting.
Daniel: The implementations on non-browser agents, what does that look like?
mfoltzgoogle: The intention is to make it very easy to make products in this space.
… There will be multiple protocols for a while.
anssik: This is the only fully open specified protocol for this use case.
David: Question on authentication. How do SPAKE2 and QUIC interact? QUIC connection is non-authenticated.
Peter: You just validate that the other side has the PSK.
David: Crypto binding between the QUIC connection and SPAKE2?
Peter: No, we just remember the certificate for reconnection.
David: That's insecure. If someone man-in-the-middle the first QUIC connection, you may end up with a wrong local certificate. There needs to be a binding.
Peter: Yes, covered with a fingerprint.
[to be discussed later]
See Review of OSP changes since Berlin (PDF, p20-28)
mfoltzgoogle: Naming is hard!
… Current set of names, which are being used in the spec:
… 1. Open Screen Protocol Agent: general
… 2. Advertising Agent: mDNS responder that also plays a QUIC server role
… 3. Listening Agent: mDNS listener / QUIC client
… 4. PSK Presenter: during authentication
… 5. [missed]
… Also from an API perspective, we talk about the Controller, the Receiver for the user agents.
… For streaming, we talk about Media Sender and Media Receiver for agents that send/receive audio/video frames.
… We don't assume that these roles are tied together. You could imagine that the Controller is a Media Sender and vice versa.
… Agents can choose what role they want to play. Some use cases from Media & Entertainment IG for Smart TVs for instance.
… We made some changes to some of the protocols in the OSP.
… We decided that we don't need QUIC connection IDs, and length prefixing for CBOR messages. In the spec, we have a message type ID as CDDL comments.
… We use QUIC varints to encode message type keys, shorter ones for most common ones.
… For TLS, we require EC certs and ignore most extensions.
… We wrote in more detail how agents should collaborate. They can advertise "capabilities" for which we reserved a range of numbers. Capabilities describe not only message types but also rendering capabilities (supports audio, supports video).
… Possibility for an agent to change its metadata and notify other side of the changes.
… Some changes to counter for protocol messages.
… Registration for extended capabilities and messages should be done through a pull request on GitHub.
… Remote playback is complicated because you not only send audio/video but also text tracks, event tracks, and that's highly dynamic.
… Changes to text track are now supported by the OSP.
… We made some simplification on the messages, merging request and response into an event message.
… Now an event that tells the controller how many independant connections there are to a presentation.
… We made changes to authentication as well. Devices vary in their input capabilities (keyboard, TV remote, etc.). Agents can negotiate who shows the pairing code, to optimize and choose the length of the PSK.
… We added a token to mDNS to prove that you're part of the local network.
… [going through other changes]
… Added some guidance when mDNS only contains truncated display name.
… We just added streaming to the OSP.
… We kept data frames, added session negotiation, stats reporting (success of playing back the media, network stats), added support of attaching media session for remote playback (MSE/HLS video)
… And added support for screen rotation.
… All the terminology stuff has been percolated throughout the spec. We emphasized the link between conformance and protocol capabilities. From the OSP spec to the API spec, but not from the other way. We may also do a two-way thing through non-normative statements.
… That kind of covers the retrospective. Next is to deep dive into unresolved issues.
See OSP 1.0 Issues to Discuss (PDF, p29-45)
mfoltzgoogle: 4 high-level topics, some of which covered in Berlin to some extent, some of which need more discussion.
… We decided a while ago to use QUIC and TLS but we didn't define the profile that we wanted to use TLS with. TLS provides many options that endpoints can negotiate.
… We're mostly following the usual pattern. Curved cryptography was better than RSA.
Victor: If you use TLS 1.3, you basically get all of this out of the box. TLS does the handshake and QUIC does the encryption.
… The short version is that, if you use 1.3, you can get away of specifying profile, because 1.3 gets rid of all the bad choices.
… Constant time is a property of implementations, not of ciphers. Basic implementations of AES are not constant time.
… Just 1.3 gives you sufficient security.
Action: mfoltzgoogle to remove notes about the TLS profile; if you use 1.3, you can get away of specifying profile, because 1.3 gets rid of all the bad choices.
anssik: Could you document that in an issue?
Victor: Sure!
mfoltzgoogle: Banning the bad ciphers, the other is telling agents what ciphers they must support, and 1.3 has some mandatory and some recommended.
… Currently the spec does not have a distinction between what type of agents needs to support what. We wanted things to be symmetric. Support for constrained devices may require support for non hardware implementations.
Victor: For signature algorithms, it depends entirely on what you put in your certificate.
mfoltzgoogle: I have a separate presentation on certificates.
… The last TLS implementation choice is what extensions are mandatory (the ones that are mandatory in the spec), and basically all optional are ignored.
David: One quick point about extensions. You may phrase it iin such a way that some implementations can still use them (opaque and transparent for applications)
mfoltzgoogle: We wanted to make sure that there are efficient ciphers available. I couldn't find benchmarks.
… Proposal is to fill in the table. AES-128-GCM is likely going to be the fastest.
Victor: AES is faster if you have acceleration.
… On Intel processors, accelerated AES is about two times faster. Roughly.
mfoltzgoogle: I can spend an hour and fill that table.
… We'll likely recommend AES-128-GCM and CHACHA20 in the end.
… The next is deciding which signature algorithms to support.
… secp256r1 is mandatory. Again, no benchmark available. We don't have a clear data decision.
Victor: QUIC note about performance. You should also check for signature and verification. ECDSA is usually faster for signature, but slower to verify. About the same thing in the end.
mfoltzgoogle: I did find some benchmark about EdDSA that gives better benchmark.
David: Yes, it could be a good fit here.
Victor: ECDSA has a potential issue that you can leak your private key if you use the same random numbers. EdDSA has some internal protection. It's more fullproof.
mfoltzgoogle: The only action item I propose is to fill the benchmark. EdDSA 25519 seems the best candidate here.
Victor: TLS uses "mandatory" very liberally. I wouldn't worry too much about mandatory things in TLS 1.3.
Action: mfoltzgoogle to fill in benchmark for ciphers.
Action: mfoltzgoogle to fill in benchmark for signature algorithms. Also check for signature and verification. ECDSA is usually faster for signature, but slower to verify.
Action: mfoltzgoogle to update TLS section to "comply with the spec" for TLS cookies and session resumption, and that's it.
mfoltzgoogle: Last one is session resumption, which Victor suggests we don't do if we can get rid of it.
… In most cases, disconnection/reconnection is less than one every 10 seconds.
… We need to provide notes about agents close connections. I assume for QUIC, it's a memory saving to close the connection.
… We're leaning towards requiring session resumption. If we do get data that it's not viablfe, we may have to revisit.
David: Not planning on using 0-RTT data?
mfoltzgoogle: No.
Resolved: Specifically ban 0-RTT data.
David: That's reasonable. I wouldn't ban it though. What is the purpose of banning TLS features?
Victor: Most of the time, session resumption puts the session cache in memory.
David: There is no real downside to having session resumption. For 0-RTT data, you have to do spec work to identify when it can be used, e.g. to start with to request capabilities.
Victor: Both parts have to opt-in for session resumption.
… Almost all extensions you can ignore them on both sides, apart from a few restricted ones.
… It's really important for parties to be able to ignore extensions on the Web.
… 50 extensions registered or so, but only ~20 real.
… Some of them are mandatory. QUIC defines extensions.
David: Note the "extension" mechanism in TLS dates back from previous versions, hence the notion of "mandatory extensions".
mfoltzgoogle: How do I say that agents must support mandatory extensions for QUIC?
Victor: You just say "comply with the spec", and that's it.
[some more discussion about extensions]
mfoltzgoogle: Same thing for the cookie extension. Not something that we're going to use, I believe, and brings complexity. It requires the client to track additional state. So something we may disallow or simply not mention.
David: QUIC has its own mechanism for an endpoint to prove that it's not attacking the other endpoint. From your perspective, you take a QUIC/TLS library and basically you're done, so I wouldn't mention it at all.
Victor: Yes, something that QUIC does automatically.
mfoltzgoogle: OK, so no mention o the Cookie extension.
… The companion to this spec decisions is the format of the certificate described in the spec. I'm open to feedback on that.
Victor: Yes, will do that.
Action: Victor to provide feedback on TLS certificate requirements
mfoltzgoogle: The rest of the story around authentication is that we cannot mandate UI but we wanted to make sure that there are sensible guidelines. Based on prior art in this area, 5 guidelines:
… 1. render information that hasn't been verified differently
… 2. We have a set of flags such as need to re-authenticate an agent.
… display it differently in such cases.
… 3. UI hard to sppok
… 4. Make the user take action to input the PSK.
… 5. Meet accessibility guidelines when showing & inputting the PSK.
… Also some language around rotation.
… Question for the group: do we agree that this is a good list (with the added note on PSK rotation)?
anssik: I think this is good stuff, goes beyond what many specs do.
Action: mfoltzgoogle to make sure to note that there is a fresh PSK for each authentication.
mfoltzgoogle: Yes, I want to make sure that people see it, hence not a separate spec.
… Who do we allow authentication with? We devised a mechanism where through mDNS you advertise a random token, to prevent attack scenarios where devices force your TV to show a PSK.
… [goes into detail of auth-initiation-token]
… Is this good enough as a first mechanism to prevent misuse of authentication? We may need to add other mechanisms later on based on experience.
anssik: No concern.
mfoltzgoogle: Finally, a topic that has been through the group for quite some time is what PAKE to use. We started to look at J-PAKE, but it requires multiple round-trips and is not implemented in most libraries.
Victor: It's very hard to implement in practice.
mfoltzgoogle: Second Challenge/Response proposal led to concerns about memory usage.
… That's why we switched to SPAKE2, better fit for the use case. Any other alternative?
Victor: IRTF group looking at PAKE has plenty of PAKES on their agenda. Simple and argumented PAKEs. For pin codes, simple PAKE seems preferrable. They don't have guidelines to pick up simple/argumented. Working on it, but probably not done by the time you take a decision.
… Name of the group is CRFG(?)
… In short, I would say: "Go with SPAKE2 but leave some window for possibility to upgrade".
mfoltzgoogle: Contact info for SPAKE2?
David: That's on the draft.
Victor: There is also a GitHub repo.
mfoltzgoogle: I'm particularly wondering about the standardization timeline.
… OK, that covers the ground on authentication.
Action: mfoltzgoogle to contact the maintainer of the SPAKE2 RFC as the current note has expired.
See Remote Playback Control (PDF, p46-49)
Peter: Since Berlin, we added!refined the playback update algorithm. We filled up the table for defaults/required added. Which fields the receiver is required to honor from the controller.
… We landed the remoting PR (remote playback via streaming).
… The way we modeled it was that there is basically a new field in the start-request and start-response messages
… Basically you have an optional streaming session attached
mfoltzgoogle: This is in addition to the URL that is passed?
Peter: Yes.
mfoltzgoogle: How does the receiver know whether to load the URL or to wait for the remote stream?
Peter: That's actually a good question.
mfoltzgoogle: An implementation might want to create a streaming session just in case that the video element switches to MSE later on. We might need a way for the receiver to know whether it should start with a URL or to wait for a stream.
Peter: What we're lacking here is a way for the receiver to know what the controller is planning to do.
ericc: In the local case, just changing the source is not enough, you have to call load as well.
Peter: As currently written, we would expect the controller to send a source changed event.
… What we don't have is what should the receiver do with a blob URL.
… We need a "There's no longer a source, time to switch to remoting" message.
ericc: Yes, that makes sense.
mfoltzgoogle: Pre-roll ad is a use case I'm thinking about.
ericc: That is very commonly used.
… Youtube does that a lot. Start with MSE, and then use a URL.
Resolved: pthatcher to add a "There's no longer a source, time to switch to remoting" message.
Peter: OK, seems we have a plan there.
… Some things not done since Berlin: add extended mime types to remote playback HTMLSourceElement.type and CSS media query to remote playback HTMLSourceElement.media.
… Should these go in the availability request as well? That's how I wrote the PR.
ericc: Seems reasonable, yes.
Peter: Second one is use of Media Capabilities / CSS colorspace values. CSS Colors Level 4 is the right reference?
ericc: I believe so, but there are some planned discussions in the CSS WG around that.
mfoltzgoogle: Field on which message?
Peter: receiver-video-capability
Peter: Another things we haven't done is recommended HTTP headers because I didn't know which ones to recommend.
Victor: Related question, are there credentials in any way?
mfoltzgoogle: The Remote Playback API doesn't really require/prevent agents to send credentials.
… At the protocol level, question is "do we want to have support for that?".
… Only thing needed is locale exchange.
David: And I don't want to give my credentials to my TV. In general, I don't trust a TV with my credentials.
mfoltzgoogle: If you want to play restricted media in any way, that's more a use case for the Presentation API for the time being.
… Question for the spec is what capabilities do we provide?
… We should focus on what the API requires, which is the locale, and let everything else up to implementers.
… The only thing I might add to that is CSP, because if all the remote side is doing is retrieving media then we can perhaps restrict that, but I'm not sure CSP is meaningful for anything other than HTML MIME type.
Peter: Right now, there's only Accept-Language.
mfoltzgoogle: Right.
Peter: So that solves #146
mfoltzgoogle: with exploration of CSP headers, perhaps.
Action: mfoltzgoogle to look at CSP-related HTTP headers to pass with remote playback
takumif: Media Capabilities allow you to query things like bitrate but there's no way to tell the remote capabilities.
… There is no way for the sender page to say that this source element has these bitrate/framerate attributes.
mfoltzgoogle: When we discussed this in Berlin, we brought up the idea of querying the capabilities of the remote device but that raised questions about fingerprinting. Eric, did you dig into it?
ericc: No.
Peter: We should probably create an issue to address that.
mfoltzgoogle: Right now, the controlling page can only try different source URLs until one works.
Peter: On the question of multiple controllers, the idea is to do the same thing as in the Presentation API, but in the Remote Playback API.
… When we looked at how we might do this, it seems pretty clear that we need some API change. We could overload the prompt, but that seems confusing.
… For remote playback, should it require the same URL like the Presentation API does?
takumif: Same URL would mean source attribute has the same URL as the one playing on the receiver?
Peter: Yes.
ericc: If you want to support this, it seems that you want a way to connect to an existing session and then get the state so that you can reflect the state locally.
Peter: What would be the permission model?
ericc: I don't know, that's a great question.
mfoltzgoogle: The use case is that I navigate or close the local page, but remote playback continues, and then I want to reconnect through a different media element.
Peter: Then you will know the URL
ericc: Yes, but even if you do, your local state is not going to be the same as on the remote side.
… Another use case is something like a group playlist. Originates from one machine but you allow other machines to hook up.
… It might make sense to think of a read-only instead of read-write.
… One real use case is sitting on the couch, watching something on my phone, switching to my TV, then I may want to use another remote controller, or I may not want to keep my phone awake, so need something magic to reconnect.
mfoltzgoogle: If we did allow prompt to allow to pass this unique session, then that would allow the controller to say that it would be ready to reconnect to an existing session.
[more discussion on transfer of control]
See Remote Playback Control (PDF, p50)
Peter: There was an action item about HDR>
Peter: The crux of it is an enum describing which HDR capability the device has.
… The second issue is about width + height, which we already cover.
ericc: The enum discussion will be resolved soon.
Peter: At that point, we just map those over to capabilities and reference that document.
See Streaming (PDF, p51-55)
Peter: streaming, since Berlin merged outstanding PR for start session and stats
… Per-codec max resolution not done, new issues since Berlin include codec switching when remoting (issue #223) and bi-dir streaming (issue #176)
… problem with changing codec: sender might get stuck if sender says "can send you A or B" and receiver says "would like B", if then source switches to A the sender has to transcode
… proposal to use capabilities instead of codec
ericc: enumerateDevices used much more than getUserMedia, suggesting it may be used for fingerprinting
… exposing more information before user consent should be avoided
Peter: possible explosion of things the receiver could support a concern
… it might be receivers just simplify and claim to support a single codec
ericc: if we do not expose this information to script, does this still work with MSE?
… transcoding is expensive and lossy, so web app may have other alternatives -- transcoding should not happen in general
… changes made to the media specs so that web app can say upfront these are the things I can offer
mfoltzgoogle: we don't want pages to enumerate devices, only availability bit
ericc: how much info you include in the description of a device?
Peter: train of thought: if the JS provided all the things it might do, e.g. codec, resolution, find me a device that can do that without transcoding
mfoltzgoogle: sites can offer multiple URLs, there's a device that can represent at least one of them, we could have a sender with a list of preferences, then pick the first compatible for that list
… to do that matching, need capabilities on the receiver, this would be an implementation detail not exposed to web apps
Peter: multiple problems to tackle in this issue, breaking into pieces:
… - avoid transcoding
… - avoid picking device that would fail w/o transcoding
… - switch codecs w/o transcoding
… - don't expose too many bits of entropy to JS
… API side, possible solutions:
… A. Media Capabilities-like querying -- doesn't know ahead of time; can't be used to select the device
… B. JS says everything it want to do up front
… Protocol side, possible solutions:
… A. Sender offers; receiver picks once
… B. Receiver expresses capabilities once; sender picks dynamically
… looking at possible combined API+protocol solutions
… AA: no possible
… AB: possible, by itself does not solve #2
… BB: possible; solves 2
… BA: possible, solves 2
… additional problem, do not prompt if nothing is available
Riju: depending on whether H.264 is on sw or hw path, performance differs
peter: we cannot do that pre-prompt, possible post-prompt
… could we say you get to ask once per page load?
<Riju> Question is if vp9 hardware decode path is possible and av1 software path, whichbone to choose
mfoltzgoogle: we want to provide good APIs and allow implementers to make tradeoffs how much information to expose
ericc: UX should not differ too much based on privacy-decisions of implementations
PROPOSED RESOLUTION: We'll try 1. Receiver capabilities that allow the streaming sender to switch codecs, plus 2. An API on RemotePlayback that allow JS to query post-prompt for capabilities like the MediaCapabilities API does (issue #223)
Resolved: We'll try 1. Receiver capabilities that allow the streaming sender to switch codecs, plus 2. An API on RemotePlayback that allow JS to query post-prompt for capabilities like the MediaCapabilities API does (issue #223)
See Bidirectional streaming / stream request (PDF, p56)
Peter: if I want to receive media from you (TV pulling up a video doorbell feed), what do I do?
ericc: why not use WebRTC?
mfoltzgoogle: if WebRTC, needs its own auth step
… do we want a symmetric protocol, or asymmetrical as of today?
keeping asymmetrical allows as of now bi-dir with two independent sessions, just more work
mfoltzgoogle: is it easier to be symmetric or asymmetric, maybe bi-dir should be an extension
<peter> Proposal for bidirectional streaming and stream requests: do nothing for now.
<peter> You can always make 2 unidirectional sessions
PROPOSED RESOLUTION: Do nothing for bidirectional streaming and stream requests (issue #176)
Resolved: Do nothing for bidirectional streaming and stream requests (issue #176)
See Extensions and Capabilities (PDF, p57-62)
mfoltzgoogle: extending the protocol with new messages not understood by all agent a new feature fleshed out since Berlin meeting
… capabilities in messages not exposed to JS, implementation detail
… extended capabilities use IDs >= 1000, can add new type of messages or add fields to existing messages
… added a public registry for capability IDs to avoid conflict, in GH repo now
… maybe in the future migrate the registry to IANA
… any concerns using this model for documenting extensions?
[no concerns]
mfoltzgoogle: proposal to merge open PRs except HDR that has an external dep to MediaCapabilities spec
See Extensions and Capabilities (PDF, p65-67)
mfoltzgoogle: areas to explore in the future for the protocol, want to finish v1 before advancing here, also recharter the group
… v2 issues: Support for DataCue, Attestation, Data Frames use cases, Aternative Discovery, Multi-device timing (joint session topic)
peter: GenericDue / DataCue, proposing to add first-class support for TextFrame
… similar to DataFrame, but with a payload that matches DataCue/TextTrackCue
ericc: only reason WebKit has support for both .data and .value is to provide reasonable error if .data is touched
ericc: container can contain metadata, media files can have metadata that is not displayed, IDv3 tags can contain e.g. images
… initial spec just has .data and the script had to know everything about it in order to understand
… from script point of view, the cues show up as part of tracks
… from script can make cues as tracks and add them
… subtitles can come with the container or from the script, and it's clear we need support in streaming for text tracks and cues
peter: take datacue and put it in existing texttrackcue for remoting, and, add explicit text frames for streaming and remoting
ericc: if they can be created from script, we must be able to reconstruct them on the other end, because it can be arbitrary JS object
peter: datacue should be in both the places? right?
ericc: correct
PROPOSED RESOLUTION: add support for DataCue to both streaming/remoting (a new text-frame message) and existing text-track-cue
PROPOSED RESOLUTION: add support for DataCue to both streaming/remoting (a new text-frame message) and existing text-track-cue w/o object
Resolved: add support for DataCue to both streaming/remoting (a new text-frame message) and existing text-track-cue w/o object
See Extensions and Capabilities (PDF, p68-72)
mfoltzgoogle: more exploratory, looking for feedback on utility of this, maybe M&E IG is interested in this
… attestation is to describe how UA finds info about another UA, manufacturer, model, s/n, OS, other capabilities, standards compliance e.g. HDCP support
… typical means to achieve this via certificates, NOT related to transport auth certs
… there's a lot of things to investigate further, precedents with EME, WebAuthN
… do we expose this to apps? Fingerprinting and privacy implementations
PROPOSED ACTION: Start a companion note separately with use cases, requirements, and draft framework
Action: mfoltzgoogle to start a companion note separately with use cases, requirements, and draft framework
See Extensions and Capabilities (PDF, p73-77)
mfoltzgoogle: problem statement: what if mDNS does not work?
… proposal to define an Open Screen Beacon format to be obtained through BTLE, NFC, or a QR code
peter: signaling over QR code should be doable
mfoltzgoogle: beacon should have the info needed to prime ICE state machine
PROPOSED ACTION to mfoltzgoogle to write an Open Screen Beacon proposal outside of this CG repo and with an explainer
Action: mfoltzgoogle to write an Open Screen Beacon proposal outside of this CG repo along with an explainer