07:01:42 RRSAgent has joined #webscreens 07:01:42 logging to https://www.w3.org/2018/10/26-webscreens-irc 07:01:45 tidoust has joined #webscreens 07:02:08 Meeting: Second Screen WG - TPAC F2F - Day 2/2 07:02:19 Chair: Anssi 07:02:30 Agenda: https://www.w3.org/wiki/DAS/Meetings/October_2018_F2F#Agenda 07:02:34 RRSAgent, make logs public 07:02:45 RRSAgent, draft minutes v2 07:02:45 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html anssik 07:02:50 tomoyuki has joined #webscreens 07:06:44 GeunHyung has joined #webscreens 07:06:51 Present+ Mark_Watson, Francois_Daoust, Chris_Needham, Anssi_Kostiainen, Peter_Thatcher, Mark_Foltz, Geun_Hyung_Kim, Mark_Arita, Louay_Bassbouss, Tomoyuki_Shimizu, Takio_Yamaoka 07:06:55 Present+ Mark_Watson, Francois_Daoust, Chris_Needham, Anssi_Kostiainen, Peter_Thatcher, Mark_Foltz, Geun_Hyung_Kim, Mark_Arita, Louay_Bassbouss, Tomoyuki_Shimizu 07:07:29 scribenick: tidoust 07:08:49 jinkyu has joined #webscreens 07:10:52 Going through the agenda, we'll talk about HbbTV, library implementation (quick run-through), security implications, ICE support for cross-LAN scenarios, J-PAKE discussions, possibility to put streaming in scope, the ability to use a polyfill implementation 07:12:23 Topic: HbbTV and Second Screen investigation 07:12:54 Louay: More a discussion about using other schemes than HTTP/HTTPS. HbbTV is one candidate for this. I don't think we should consider HbbTV as a specific case, more as an example. 07:13:06 mfoltzgoogle has joined #webscreens 07:13:15 ... It's similar to mobile applications where you can start other applications using a specific scheme. 07:13:55 ... In the example I'm showing, I pass an "hbbtv:" URL to the PresentationRequest constructor, with a fallback on HTTPS and cast. 07:14:04 -> https://github.com/webscreens/openscreenprotocol/blob/gh-pages/schemes.md Schemes and Open Screen Protocol 07:14:22 [showing Schemes and Open Screen Protocol document on the screen] 07:14:27 ... The user-agent will pick up the right URL, although we need to discuss the order and priority. 07:15:50 ... Scrolling down to the diagram, here's an example with a controller and two receivers. The controller supports hbbtv, cast. 07:16:09 ... The first receiver has two UAs, one supporting cast, the other hbbtv. 07:16:22 ... What should we display to the user? 07:16:51 ... I don't think the user cares about the technology, she just wants to select a device, not the device/scheme pair. 07:17:30 ... The controller device will detect that the URLs are supported and by whom and display the right device accordingly. 07:19:49 ... The filtering is important. Supposing we want to pass a URL with an "hbbtv:", the controller could send the targeted scheme in the discovery request. Or the receiver could advertise supported schemes in the response. Third option, just do the discovery without info, and exchange the data afterwards during subsequent phases. 07:19:58 Present+ Eric_Carlson 07:20:28 jernoble has joined #webscreens 07:20:36 ... For the hbbtv scheme, it's important that we pass all the parameters required to launch the application. 07:20:59 Present+ Masaya_Ikeo 07:21:08 ... For now, in HBBTV devices, DIAL is used, with an XML request message that contains a few parameters. 07:21:43 ... Important ones are the "appId", "orgId", "appName", "appUrl". I don't think that's up to the Open Screen Protocol to specify that. 07:22:07 MarkFoltz: There's a number of things to unpack here. 07:22:45 ... Going back to the diagram, what is the user experience today with Android TV? 07:23:15 ... Today, the way it's implemented, the user would see two different displays that correspond to the same physical device. 07:23:48 Louay: I discussed that with Chris. We could add a Cast logo or an HbbTV logo, but that would be confusing for users as they don't know anything about these technologies. 07:25:22 Francois: I note the spec imposes which URL gets selected, using the array order. 07:25:53 MarkFoltz: It really comes down to whether the controller sees the Cast receiver or the HbbTV receiver as same or separate screens. 07:26:29 ... If the device platform is advertising OSP and accepts a single connection, there shouldn't be an issue. 07:26:46 MasayaIkeo has joined #webscreens 07:27:18 Francois: That supposes that there is a box, missing here, that connects to the two internal user-agents in the first receiver. 07:27:21 Louay: Yes. 07:28:03 ... On one devices, you have two or three different receiving agents. 07:28:31 ... We should consider ways to merge them into one physical device, perhaps using the IP address. 07:29:06 MarkFoltz: There may be solutions available, but it's important to think about impersonation issues, so there would need to be some keys shared by all the receivers at the platform level. 07:29:46 ... Different ways to have multiple listening endpoints on the same device, one is using mDNS subtypes. 07:30:06 ... One mDNS request should be able to trigger multiple mDNS subtypes responses. 07:31:46 Louay: The solution designed here won't work for existing HbbTV devices because DIAL is used. But that's ok, that's future-looking. We just want to make it attractive for HbbTV to be able to adopt this. 07:32:12 MarkWatson: It's been a long road to get DIAL into TVs, not only for HbbTV. Something new will take a long time as well. 07:33:06 MarkFoltz: Allowing a device that supports multiple schemes seems a good candidate for mDNS subtypes. 07:33:28 Louay: Then you get the info right away and don't need to connect to these devices with a scheme they do not support. 07:34:24 ... This is more specification work, design decisions for the Open Screen Protocol. 07:35:06 MarkFoltz: Proposed resolution here, I think, would be to discuss alternative ways to advertise schemes and come up with specs that allow for pull requests. 07:39:18 -> https://github.com/webscreens/openscreenprotocol/issues/93 [Meta] Add note to schemes.md about custom schemes and interop #93 07:40:26 The group discusses arbitrary URL schemes. Browser vendors will unlikely support arbitrary ones. There are trade-offs to consider, and different perspectives. If arbitrary ones are supported, interoperability is unlikely to be achieved in the sense that receivers won't converge to common schemes. 07:41:01 Some tradeoffs need to be considered, that is the purpose of #93. 07:41:52 Some question about how it affects user experience. Clearly, the user will think about it in terms of how many times it works and how many times it does not. 07:42:31 If you have only one way to do things, then interoperability is great. If not, then there is a combinatory issue. 07:44:15 MarkWatson: If you let vendors decide which schemes they can support, then obviously they will go with their own, and interoperability is an issue. But then, if you stick to HTTPS, you're pumping the problem into the domain name, where receivers may detect e.g. "netflix.com" and launch a specific applications for such URLs. 07:45:09 MarkFoltz: Two levels, device interop, and content interop. We need to address both, and we're more focusing on the first one for now. 07:46:00 ... We definitely have transitions across schemes, e.g. Web Intents, native app triggering. The goal here is to document the pros and cons. 07:46:46 anssik: What are the pros and cons of the custom schemes? That's a good investigation to make. 07:46:59 MarkFoltz: That's something we may want to bring to the TAG for comments at some point. 07:48:26 PROPOSED RESOLUTION: Explore and document the pros and cons of specific schemes in a format that the TAG would fancy reviewing (e.g. including thumbs-up cats) 07:49:19 RESOLUTION: Explore and document the pros and cons of specific schemes in a format that the TAG would fancy reviewing (e.g. including thumbs-up cats) 07:50:24 MarkFoltz: The other resolution I wanted to capture is, given the reality of existing schemes, we need a way to advertise supported schemes during discovery, e.g. using mDNS subtypes. 07:51:13 PROPOSED RESOLUTION: Investigate and include some proposed solution to advertise custom scheme in mDNS 07:51:36 RESOLUTION: Investigate and include some solution to advertise custom scheme in mDNS 07:51:54 ericc has joined #webscreens 07:52:57 Topic: Progress on open screen library 07:53:57 MarkFoltz: Will go quickly over the status and progress in the library 07:53:59 -> https://chromium.googlesource.com/openscreen/ Open Screen Library 07:54:21 ... Objectives are to make it easy for people to add open screen protocol support. 07:55:09 ... It's self-contained, embeddables, and there is also an OS abstraction layer if you want to integrate it at that level. 07:55:45 ... We're trying to keep it with a small footprint, although we're integrating a bit of WebRTC, which increases the footprint surface. 07:56:24 ... We're implementing things one after the other. We obviously want to support controller/receiver parts of the protocol for the Presentation API. 07:56:36 ... And local/remote for the Remote Playback API. 07:56:55 ... The library does not do rendering of HTML5 and media obviously, that's up to the embedding user agent. 07:57:06 ... Extension points are being considered. 07:58:14 ... Overview of the architecture: top API that exposes two different interfaces. One is an API that looks like the spec. The other is an API to expose some controls over protocol, e.g. turning on/off QUIC connections, controlling authentication flows 07:58:44 ... At the protocol layer, the CBOR control protocol and QUIC connections are the main ones. 07:59:48 ... For service implementations, we include some parts of the Chromium QUIC and integrate that in the library. Chromium QUIC is moving towards a more embeddable codebase, but for now that takes some size. 08:00:04 ... At the platform API, this is where things like ways to control sockets go. 08:00:14 MarkWatson: That's just one C++ library? 08:00:17 MarkFoltz: Yes. 08:00:55 MarkWatson: These may live in different processes on the TV, e.g. mDNS discovery. Is the QUIC stack loaded everywhere? 08:01:47 MarkFoltz: We cannot do that mapping ourselves. Some boxes can be substituted to platform boxes. It will be more modular as we integrate with different platforms. 08:01:58 ... As a starting point, you'll get a library that can do everything. 08:02:25 MarkWatson: A different approach would have been to expose CBOR and implement the rest in JavaScript. 08:02:55 MarkFoltz: We expose the interfaces in such a way that embedders can supplement their own implementations in some cases. 08:03:51 ... You would like to be able to update the core protocol at a faster pace? 08:04:20 MarkWatson: I'm wondering how you see that. JS is easy to update on a daily basis. 08:05:14 MarkFoltz: The CBOR messages would always be in C++ because we don't want to have a JS interpreter dependency there 08:05:24 ... Extension points are one of the design goal. 08:05:55 MarkWatson: Experience with CE devices is that the C++ update lifecycle does not even exist, unless there are big security issues. 08:06:06 ... That will progress, but that's a huge overhead. 08:06:36 MarkFoltz: I think it's certainly doable to implement some of that in the JS runtime. I don't think we'll do that as part of that library. 08:08:09 ... If for your use case, you want to draw a horizontal line in the middle and expose a QUIC connection to the upper runtime, then you can 08:08:34 Francois: It could be worth exploring how to compile the upper layers to WebAssembly for instance. 08:08:59 MarkWatson: Modularization would be key here. That would help with integration and maintenance in CE devices. 08:09:50 Mark Foltz presents a video screencast of running the library running on the command-line. 08:11:04 MarkFoltz: Obviously, we want to hook this up to something that can render the results for a more interesting demo. 08:12:08 ... We recently landed a way to generate CBOR serialization and parsing through CDDL. Links to the discussion we had yesterday. 08:12:53 ... We started doing this by hand for one message, and decided to automate that for 40 messages. 08:13:10 ... This does not yet use the comment convention we discussed yesterday. 08:13:46 ... The Platform API, we have an implementation that works for Linux. Other platforms are possible. 08:14:37 ... Looking at the roadmap, we started in February. Today, we have most of the embedder APIs done, a part of the control protocol. The next task is to develop the authentication part. 08:14:48 ... Then benchmarking, end-to-end features. 08:15:28 ... We're getting closer to enabling external contributions. Louay left the room, but we'd be happy to use some of his resources. 08:15:46 ... Our code system is a bit "interesting" but it is documented. 08:24:18 ericc has joined #webscreens 08:32:17 takio has joined #webscreens 08:32:26 present+ Nigel_Earnshaw 08:32:54 topic: Privacy and Security 08:34:16 rrsagent, draft minutes 08:34:17 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html cpn 08:35:41 RRSAgent, draft minutes v2 08:35:41 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html anssik 08:42:02 Mark: This is a two part discussion. I prepared material to talk about the considerations for thinking about a security architecture for OSP 08:42:10 GeunHyung has joined #webscreens 08:42:19 ... What information shuold we protect, threat models, high level techniques to address them 08:42:26 ... Peter will present on some of the mechanisms 08:42:46 ... W3C has a security and privacy questionnaire that covers some of the high level material 08:42:53 ... Used that to frame some of this discussion 08:43:07 ... Separately, I think we should use the questionnaire in more detail 08:43:32 ... What's important to protect? Personally identifiable information, high value data 08:43:52 ... For our protocol and APIs there are few things: the content of URLs and IDs used by the Presentation API 08:44:12 ... Presentation messages generated by the application are considered private data. They could contain passwords or tokens 08:44:27 ... Remote playback contains URLs, also streaming would be in scope 08:44:54 ... We'll look at mechanisms to implemented security, such as private keys and J-PAKE passwords 08:45:18 ... There could be more things to add to the list, so the questionnaire will be important to work through more systematically 08:45:18 tidoust has joined #webscreens 08:45:35 ... In the case where you have a public presentation, e.g., digital signage 08:45:50 ... You may want to advertise the URL, so it's no longer completely protected data 08:45:59 ... Should consider who should be allowed access to it 08:46:21 ... The second category of data I considered is device-specific data 08:46:30 ... Not about content or users, more about the device itself 08:46:33 MasayaIkeo has joined #webscreens 08:47:01 ... Could be discovered through network advertisement, could include GUIDs like device IDs, device capabilities 08:47:20 ... Asking whether a device can play a URL divulges information about the device 08:47:34 ... We should consider how much to protect this information 08:47:43 ... There's other information available such as MAC addresses 08:47:58 ... How valuable is the data, aggregation for different purposes? 08:48:17 ... Moving the friendly name to the post-authentication part of the protocol may address some concern 08:48:35 ... There are different threats, unauthorized actors who want to access this info 08:49:05 ... Three threat models in the questionnaire: a passive attacker, on the local network, all they can do is read data being exchanged, can't modify it 08:49:15 ... Learn what's advertised through mDNS 08:49:31 ... They can see who's conntecting to whom, but can't see the content of messages 08:49:45 ... We want to minimise the data expose before establishing an encrypted transport 08:50:42 ... A few things to follow up on. Even with TLS there are passive active attacks possible, so we should look at the history of those kinds of vulnerabilities 08:51:02 ... One mitigation is to rotate keys, could help address these concerns 08:51:26 ... The second kind of attack is where an attacker can inject or modify traffic in the network 08:51:51 ... There are ways to do that, by gaining access to the LAN, comprimised device or router 08:52:06 ... Have a potential for a much wider range of exploitable vulnerabilities 08:52:15 ... Three scenarios we should address in the design 08:52:45 ... One is impersonation, man in the middle. A device that intercepts the data intended for a TV where the user is not aware of it 08:53:10 ... Another is impersonation of a controller. Want to understand the vulnerabilities this may introduce 08:53:19 ... The third is denial of service. 08:53:57 ... Impersonation of a receiver. Want a way to validate the QUIC connection to the intended device 08:54:23 ... One mitigation is the J-PAKE proposal, to create a shared secret that's passed out of band 08:54:39 ... A password or QR code displayed on the TV, creates a shared secret 08:55:24 ... Something used in combination with out of band secret is to have a third part that can sign a public key and the device can use the key as a trusted identity 08:56:01 ... Want a distinction in the UI for trusted vs untrused devices. Analagous to the TLS connection indication in Chrome 08:56:24 ... Also in the advertisement process, look for collisions between MAC addresses or friendly names, and flag these to the user 08:56:52 ... If the user changes the name, it may imply that trust should be broken with that device, and have to go through the authentication process again 08:57:14 ... The second analogous case is a device acting as a controller that impersonates another device 08:57:35 ... Important, as we don't want to allow devices to access presentations without there being a relationship between those two devices 08:58:30 ... When the user verifies the pairing process, create a client certificate 08:59:11 ... Another thing we could do is look at the presentation id itself, for reconnecting to presentations and harden that by connecting to the authentication mechanism 08:59:21 ... Want to prevent brute force attacks 08:59:26 .. Don't have a proposal for that yet 08:59:26 ericc has joined #webscreens 09:00:04 ... Finally, in our protocol design, we shouldn't expose PII data without a valid client certificate and valid presentation ID 09:00:26 Peter: If I initiate a remote playback, could someone else connect and control that remote playback? 09:00:44 Mark: Currently not, but if in the future you want a remote media session where a device could have remote media controls 09:01:02 ... Want to look at which controls could be PII, e.g., don't expose URLs to other controllers 09:01:19 ... The third high level issue is denial of service 09:01:45 ... A malicious device could poison mDNS caches, publish fake records 09:01:50 ... Exhausting resources on the receiver 09:02:00 ... Don't have great ideas for how to mitigate 09:02:26 ... Maybe DNSSEC for filtering of mDNS responses, throttling of connection attempts 09:02:35 ... These may just be implementation concerns 09:02:52 ... If you have access to the LAN you can do things to prevent any network protocol from functioning, e.g., blocking packets 09:03:03 ... May not be worth spending a lot of effort in this area 09:03:18 ... We've been discussing protocol designs for authentication 09:03:36 ... Looking at the security of the entire software stack, there are things outside the protocol design 09:03:53 ... Security of the underlying platform and O/S, hardware backed crypto 09:03:59 ... Verified boot 09:04:17 ... Another attack vector is untrusted content that tries to exploit vulnerabilities in the platform 09:04:26 ... Sandboxing and origin isolation 09:04:38 ... Software updates are key, particularly in the TV ecosystem 09:05:02 ... Educating developers on how to use the APIs, and UI guidelines for how to display specific data 09:05:15 ... Auditing and managing the lifetime of keys 09:05:26 ... These are broad areas of interest, not necessarily in scope 09:05:48 ... What I think we should do as a group is look at the W3C security and privacy questionnaire 09:05:59 ... We should research best practices for authentication 09:06:10 ... TLS 1.3 uses cyphers more secure than before 09:06:26 ... in J-PAKE, length of passcodes and length of validity 09:06:47 ... We'll talk to people internally and get review fro,m other W3C groups 09:07:09 ... And get feedback from application developers, as they're the ones who will generate the high value data 09:07:17 s/fro,m/from/ 09:07:52 Topic: J-PAKE 09:09:07 Peter: [recaps state of progress from yesterday] 09:09:45 ... Two parts to this: how to use this with TLS, and what the CBOR messages are 09:10:03 ... Three options: JPAKE after TLS, JPAKE replacing TLS, or do something else 09:10:19 ... With JPAKE after TLS, it would have a number of round trips 09:10:29 ... Certificates from both sides, each side knows the other's 09:10:44 ... Each side authenticates to the other using the shared secret 09:11:04 ... JPAKE requires 3 RTT, but can be dome with 2 RTT using the response to trigger the next step 09:11:30 ... With round 1, the initiator provides some information and zero-knowledge proofs 09:11:35 ... The JPAKE spec goes into the maths 09:11:51 ... The initiator gives two sets of these, and the responder responds with two sets 09:12:23 ... In the second round trip, completing round 2 and initiating round 3 09:12:42 ... The third round is not actually part of JPAKE specifically 09:12:58 ... The result of the first two rounds is a key known to both sides, can be used as a replacement in TLS 09:13:21 ... If you want to verify that both sides have the same key, you need the third key 09:13:44 s/third key/third round/ 09:13:48 ... Pass a double hash 09:14:21 Mark: [discussion of bigint, bytes, bignum as the CBOR type] 09:14:58 MarkW: What is the zero knowledge proof? 09:15:32 Peter: To start, you have g, p, q 09:15:50 ... You provide gv, and r - it's all in RFC 8235 section 2.2 09:16:13 ... The signer ID picked by each side, we could put the fingerprint from the TLS connection there 09:16:32 MarkW: Is it something passed from one side to the other, or is there challenge-response? 09:16:39 Peter: The other side verifies it's correct 09:17:10 ... Both sides provide a shared secret, then we do all the messages, the result being some keys 09:17:24 MarkW: It's passed out of band 09:17:32 ... Are we proposing to standardise that? 09:17:45 Mark: We have to provide constraints to ensure a given level of security 09:18:19 ... e.g., printing the value on the device, want to have constraints for passcodes of a certain length, entropy, rotation 09:18:36 MarkW: Which way do these go? 09:18:47 Mark: Easier to input on the browser than on the TV 09:19:00 Peter: It's a one time code, then thrown away 09:19:32 ... Option B is that instead of doing TLS then JPAKE over the encrypted connection, we could use JPAKE to replace the handshake 09:19:41 ... There's an IETF draft 09:19:55 ... Not clear when this is going to happen, the draft is 2 years old, has some issues 09:20:17 ... If we did this, it would work by using JPAKE as the handshake on the first connection, then swap certifcates 09:20:24 ... and use TLS certificates on subsequent connections 09:21:14 MarkW: TLS has cypher suite negotiation, JPAKE is elliptic curve, so different message formats? 09:21:34 Peter: If there were another form of JPAKE, would have to be different CBOR messages 09:21:48 MarkW: How do we review the security of this? 09:21:54 Peter: Let's come back to it, good question 09:22:24 ... TLS with a challenge response, you can go from shared secret with mutual authentication 09:22:58 ... A message that includes a challenge, response includes a hash with the challenge or password 09:23:10 ... Two fingerprints avoids MITM attacks 09:23:23 ... With those two certs and the challenge given 09:23:30 ... I believe this would be secure, and simple 09:23:37 ... Comparing options 09:24:06 ... Having JPAKE replacing TLS is hardest to spec, easy to get wrong, but uses 2 RTT on the first connection 09:24:33 ... JPAKE after TLS is easier to spec and implement, so harder to get wrong, but use 3 RTT on the first connection 09:25:15 ... Third option, challenge response after TLS is easy to implement, and hard to get wrong - but we'd need security review. It has 2 RTT on the first connection 09:25:50 Mark: It seems there must be some precedent with the third option with WebRTC. Do you do the challenge response in a separate layer? 09:26:16 Peter: No, they're signalled out of band. You'd somehow convey the entire secret between the devices, show on the TV 09:26:33 ... The out of band communication channel is trusted 09:26:50 MarkN: We do something similar at Intel. Do you have caching? 09:27:10 Peter: Yes, you wouldn't need to go through the process on subsequent connections 09:27:18 ... You'd cache their certificate 09:27:26 MarkN: Does JPAKE account for key rotation? 09:27:38 Peter: New keys set on each QUIC connection 09:29:34 Nigel: Where you do the JPAKE and accept and store a cert, this implies you have trust from the JPAKE, so you'd have to independently verify the certs anyway 09:30:05 ... It only works if you have independent scrutiny of the certs, so it seems something's missing 09:30:28 Peter: With JPAKE after TLS, the cert is self signed, and JPAKE is used to trust it 09:31:27 MarkW: You trust it because the user told you 09:31:55 ... Would require collusion of the user 09:32:22 Peter: I recommend we don't do the JPAKE replacement of TLS, and consider dropping JPAKE for challenge/response 09:32:36 .. If we don't want to drop JPAKE, we should do JPAKE after TLS 09:32:57 Mark: Tradeoffs with usability, length of code to input 09:33:16 .. The challenge response seems simpler, reason to favour it for implementers 09:33:33 ... We could research some of the existing solutions from Google and Intel 09:33:41 ... A security review will be mandatory 09:34:11 MarkW: So we should put these in order of preference to focus the review 09:34:29 Peter: Anyone have a problem with dropping JPAKE? 09:35:24 PROPOSED RESOLUTION: Choose challenge/response model, ask for security review, and adjust based on feedback 09:35:40 RESOLUTION: Choose challenge/response model, ask for security review, and adjust based on feedback 09:36:22 Topic: Support for off-LAN devices using ICE 09:36:49 Peter: ICE converts a messaging or signalling channel to a data channel, not necessarily fast or high throughput 09:37:00 ... Use that to bootstrap to a direct connection 09:37:14 ... Needs and Id and password for the session 09:37:28 ... ICE candidates are IP+port combinations 09:37:38 ... Need to pass these to attempt the direct connection 09:38:07 ... STUN is a mechanism where if you're behind a NAT it conveys what it sees as the IP and port 09:38:29 ... TURN is a protocol for going through a relay server 09:38:49 ... ICE doesn't always result successfully in a direct connection, could be through a relay server 09:39:09 ... Two separate STUN servers don't need to know about each other, also for TURN server 09:39:31 ... Each site can initiate ICE. Complication if both sides do it at the same time, but this can be worked out 09:40:03 ... Two ways that ICE can be used. One is independently of OSP, then it's done over the discovered connection 09:40:53 ... The second way is to put something about ICE into OSP for bootstrapping a connection 09:41:37 ... For example, connecting from a browser to a TV on a LAN. Could use that to send the ICE session 09:42:35 ... If we did that, when ICE is initiated we'd have a ICE start request 09:42:50 ... [presentation of ICE messages] 09:43:25 ... These would only be useful for bootstrapping from an existing connection to an ICE based connection 09:43:47 ... For ICE to work well, you need a signaling channel the whole time 09:44:13 ... It might have some value depending on the use cases we want to cover 09:44:40 ... In the web context, we need some way to do the initial bootstrapping, exchange the candidates 09:45:21 ... One way we could do that is use RTCIceTransport, and pass to the presentation API 09:45:40 .. Or the Presentation API could create the ICE transport 09:46:07 ... Alternatively, use RTCQuicTransport 09:46:32 ... Might be useful where someone wants to do a presentation or remote playback to a server, which is then sent out to other systems 09:46:49 ... Need some object that acts as a transport to pass to the APIs, the browser takes it from there 09:47:20 Mark: We need to explore further whether we drive the ICE functionality more from the browser side 09:47:35 ... In which case we need to expose a messaging channel for doing the signalling back to the application 09:47:49 ... Or we allow the application to drive ICE and present that to the API 09:47:59 ... We'd need to figure out where the authentication handshake fits in 09:48:37 RRSAgent, draft minutes v2 09:48:37 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html anssik 09:48:55 Mark: [discussion of API shape] 09:49:21 Peter: Key thing here is we need another discover mechanism than mDNS, e.g., a service that the client knows to go to 09:49:51 Mark: Register the device as a presentation screen, or other mechanisms such as bluetooth 09:50:14 ... Other APIs in development could help, Web Bluetooth, Shape detection API 09:51:00 ... Could use bluetooth or high frequency audio to bootstrap a URL 09:53:23 Mark: Use case is a projector on another network than your device, need a way to bootstrap 09:53:34 Anssi: Next step for this topic? 09:55:01 Peter: Three things we could do: one is to add support for ICE to the OSP library, second is to make an extension spec for the Presentation and Remote Playback APIs that allow injection of the transport, third is to add OSP messages 09:55:33 Mark: We'll explore in our own implementation initially, will help inform future spec work in this area 09:56:30 angel has joined #webscreens 09:56:33 Anssi: This is currently out of scope of the CG, let us know if we want to bring it into scope 09:57:01 Peter: There's an RFC for SIP, but we don't want to do that 09:57:12 Mark: It's not OSP specific 09:57:35 Peter: There's a SIP way, and XMPP way, and if we do it, also an OSP way 09:59:34 RRSAgent, draft minutes v2 09:59:34 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html anssik 10:00:42 [breaking for lunch, back at 13:00] 10:00:46 tidoust has joined #webscreens 10:38:44 Zakim has left #webscreens 11:04:01 tomoyuki has joined #webscreens 11:07:29 Zakim has joined #webscreens 11:08:40 Topic: Streaming 11:09:05 Anssi: The purpose of this session is to decide whether we want to add streaming to the CG's scope 11:09:36 Peter: There are 5 sets of messages that would work for streaming. The most important is sending of audio and video data 11:10:06 ... It's possible to send audio in separate QUIC streams to get the frames out of order, and build on top of the CBOR we already have 11:10:46 ... Some messaging is needed to determine capabilities 11:11:03 ... The most important parts are sending the media, and key frame requests to the controller 11:11:14 ... [Audio frame example] 11:11:44 ... A video frame can be quite large, so some minor overhead due to the keys, but with audio the overhead would be larger 11:12:31 tidoust has joined #webscreens 11:12:34 ... Codec-specific payload 11:12:44 RRSAgent, draft minutes v2 11:12:44 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html tidoust 11:13:16 ... The start-time could have an numerator and denominator, can infer these from the encoding-id and start-time 11:13:20 i/Mark: This is a two part discussion/scribenick: cpn 11:13:24 RRSAgent, draft minutes v2 11:13:24 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html tidoust 11:13:46 ... Can infer the end time from the start time + sample count 11:14:16 takio has joined #webscreens 11:14:23 MarkW: Other specs would call this the sample duration 11:14:54 ... AAC uses a 24kHz clock but can get either 24,000 or 48,000 samples out depending on the encoding 11:15:41 ... The term "sample" is ambigious - meaning is different between MP4 and PCM samples 11:17:26 MasayaIkeo has joined #webscreens 11:18:01 Mark: If you change the encoding id, how do you specify that this is a new version of a previous id? Is there a 'track' concept? 11:18:09 Peter: I'll get to that 11:18:58 ... Regarding encoding, you can put anything in the payload and CBOR can tag it 11:19:30 ... You could put it as bytes, and put the codec info in the encoding-id, or you could have an opus specific payload 11:19:41 MarkW: But that indicates the codec in every frame 11:20:34 Mark: One option is to have a generic payload of bytes, or we have some tagged format with Opus bytes or AAC bytes, and then the receiver can send those bytes to the codec 11:21:03 Peter: If we're trying to squeeze bits, then we should have the payload be bytes and make the codec implicit 11:21:58 ... Things that change infrequently can go into the optional fields 11:22:45 Mark: So all new values would be optional 11:23:53 ... The thing to take away is making sure this is extensible without making the parser too complex 11:25:05 Peter: In the optional area, the duration may change. Opus is typically sent at 20ms. If you detect you need to send more bits, can set to 120ms 11:25:21 mfoltzgoogle has joined #webscreens 11:25:56 ... In WebRTC, the sequence number is used by the jitter buffer. I added it as an optional field here, if we think it's worth having 11:26:43 Mark: Since it's optional until we need it, we can mark it as potential for the future. We'd have to think about how receivers would behave if it were missing 11:27:29 Peter: It's nice to have a mechanism for synchronizing clocks. Not every frame, every few seconds, have a value that can be compared between frames of different encodings, e.g., audio and video 11:27:44 ... Not sure whether to use media-time field here 11:28:38 MarkW: Do you require the media time to advance at the same rate as the start time of the frame? 11:29:21 Peter: No, there may be clock drift. This is about synchronizing audio and video together 11:29:41 Mark: Does it need to be the common denominator between the audio and video clocks? 11:29:45 Peter: No 11:30:01 ... Video frames are much bigger, so relative overhead of metadata is less 11:30:20 ... The encoding id gives an implicit clock duration, rate and codec 11:30:32 jernoble has joined #webscreens 11:30:49 ... The video frame has a frame id. You don't always know the duration of the video frame. You need to be able to refer to previous frames 11:31:27 ... It's more common to have codec specific headers or metadata with video than audio 11:32:19 Mark: What would be inside the payload here? May be interesting to come up with some examples 11:32:51 Peter: The other thing common with video frames is dependencies between frames, unlike for audio 11:33:04 ... So we have an array of frame ids that the current frame depends on 11:33:25 ... For a keyframe, this would be empty 11:33:42 ... It gets more advanced with SVC 11:33:53 Mark: This has time dependencies and resolution dependencies 11:34:06 ... Can we capture those dependencies with this? 11:34:23 Peter: We'll find out when we implement it 11:35:48 MarkW: If i'm receiving a fragmented MP4 video stream, and copy this to an outgoing stream in this format. I wouldn't know which frame this frame depends on, MP4 doesn't tell me. I'd have to dig into the codec 11:36:04 ... Parsing the container format won't be enough 11:37:30 Peter: Having a field here that something in future will depend on it, and are we duplicating information 11:38:00 MarkW: Random access is a use case - how to know when to start without decoding the bitstream 11:38:34 MarkW: Would you send frames in presentation order or decode order? 11:38:43 Peter: They can be reordered over the network 11:39:14 MarkW: So the start times are not necessarily monotonic by frame id 11:40:04 Peter: Next is duration, optional and defaults to the encoding's value 11:40:32 ... There's a video rotation, defaulting to the encoding's value. It's common to have a 90, 180, 270 degree 11:40:40 Francois: Also horizontal or vertical flip? 11:40:50 Peter: Only rotation is used in WebRTC 11:41:24 ... I don't see a use case for that 11:42:13 Peter: The media time is for synchronisation, either between audio and video, could also be between two videos or two audios 11:42:25 ... Lastly, there are data frames 11:42:43 ... The most obvious case is for text tracks, but could be any data to be synchronised with the audio and video 11:42:51 ... All the real info will be in the payload 11:43:14 ... Again, we can discuss if this should be bytes or an arbitrary CBOR object 11:43:36 Mark: We're already going to have to define schemas for text tracks in remote playback 11:44:09 Peter: We can define things that we'd expect to go in there 11:44:31 Louay has joined #webscreens 11:45:03 ... The encoding-id describes the sequence of frames, e.g., a particular encoding of text cues 11:45:38 ... We could think about typical data sent along with video or from a capture device 11:45:51 ... e.g., input devices such as a mouse pointer 11:46:28 @: Are you thinking about AR and VR use cases? 11:46:37 ... There's the problem of sync there 11:47:17 Mark_Narita: The Immersive Web WG was discussing having multiple layers of video 11:47:26 Peter: We can support that 11:47:32 ericc has joined #webscreens 11:48:03 .. I didn't add anything regarding instructions to the receiver for what to do with them 11:48:12 ... You could send more than one video stream 11:48:23 ... We have nothing to talk about post-processing 11:48:32 @: Do you assume the same transport? 11:48:55 Peter: This shares the same transport, and you should be able to synchronize 11:49:22 Mark: It's a good starting point, could bikeshed the structure details 11:49:49 ... The video frame looks much like an audio frame with a few things added, could set up an 'extends' relationship 11:50:46 does it need a key-frame flags for video frames? 11:51:29 Peter: can have a keyframe flag by having an empty dependencies array 11:51:55 @: The payload may have info about whether it's a keyframe 11:52:53 Mark: But the dependencies is optional 11:53:26 s/@:/Alexandre_Gouaillard:/ 11:53:31 Present+ Alexandre_Gouaillard 11:53:36 @: Is there a use case to justify leaving it unknown? Do we have a problem making it optional? 11:53:42 s/@:/Alexandre_Gouaillard:/g 11:54:10 MarkW: [discussion of how to handle keyframes] 11:55:05 Peter: This information is really for the jitter buffer so it can decide when to feed it to the decoder, as frames can arrive out of order 11:55:22 Mark: You may need to wait for re-transmits to complete a frame 11:57:46 MarkW: This structure looks good to me 11:58:38 Anssi: The key thing is to decide whether this should be in scope 11:58:45 scribenick: tidoust 11:59:02 ... Of course, we can take a resolution to explore further before we commit to extend the charter. 11:59:19 ... The amendment process is written in the current charter 11:59:45 MarkFoltz: I don't recall if anyone objected specifically to streaming when we rechartered the CG. I think it was mostly a question of setting the scope. 12:00:25 ... Streaming would be important to add to the charter and to open screen at some point, because the 1-UA mode has streaming needs, which will need to be addressed at some point. 12:00:35 ... But we have other priorities to deal with. 12:00:45 ... So, mostly a timeline question. 12:02:44 Francois: No specific requirement from a W3C perspective. The CG process is whatever the current charter says it is. It sets rules for amendments. So that's it. It's up to you. 12:03:26 anssik: Maybe we can add it with a note that the priority is the current scope. 12:03:33 ... Streaming is a phase 2 deliverable. 12:04:01 PROPOSED RESOLUTION: Start the process to add streaming in scope of the Second Screen Community Group as a phase 2 deliverable 12:04:47 RESOLUTION: Start the process to add streaming in scope of the Second Screen Community Group as a phase 2 deliverable 12:05:09 ACTION: Anssi to start the process for charter amendment to add stream in scope 12:06:07 Peter: If we start working on it, we'll need to think about media streaming capabilities, key frame requests and a few other things. 12:06:32 MarkWatson: You might want to add some HDR color identification in the capabilities because that's not in the codec. 12:07:01 Topic: RTC polyfill 12:07:22 Peter: The idea here is "what can we do if the browsers do not implement the Open Screen Protocol?" 12:07:45 ... Assuming they do not but are implementing some RTC technologies, there are things they may do. 12:08:12 ... One example is Edge implementing ORTC specs and lookin into RTCIceTransport and RTCQuicTransport. 12:08:43 ... There are things you can do, but you cannot do without mDNS discovery and some mechanism to cause the browser's receiver selection dialog to pop up. 12:09:00 Present+ David_Baron 12:09:30 ... We could use ICE for discovery instead of mDNS and implement the Open Screen Protocol in JS, but the key here is that you wouldn't be able to use mDNS for discovery. 12:10:19 ... If you did have this mechanism, the selection of the device would be an in-page mechanism. 12:11:56 ... Actually, that's not really ICE for discovery. It's ICE to establish the connection following an out-of-band discovery mechanism. 12:12:15 MarkFoltz: Think about a list of friends on a social app, and you click on one. 12:13:01 ... There are overlaps between this use case and our ICE discussion earlier. 12:13:28 Peter: The mechanism for that would be entirely driven by the Web app. 12:13:47 Louay: Another mechanism would be using audio fingerprinting, à la Shazam. 12:14:17 MarkFoltz: Anything that's going to require a sensor is probably going to require permission. 12:15:15 Peter: User experience would be that they would select the receiver in the page. Web-app specific thing. There will be no way for a device to find devices connected on the LAN unless there's another mechanism to do that. 12:15:48 ... To stream out, you'd need some way to stream the content: getUserMedia, getDisplayMedia to stream a tab. 12:16:34 ... Last piece is that you'd need an encoder. In some cases, that's not strictly needed. Right now, encoder is embedded in RTCRtpSender, decoder in the receiver. 12:16:46 ... The WebRTC WG is looking at ways to making that more accessible. 12:16:50 RRSAgent, draft minutes v2 12:16:50 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html anssik 12:17:20 ... By far, the biggest missing gap is the lack of mDNS discovery. 12:18:21 MarkFoltz: [mentions the Network Discovery API and security issues triggered at the time]. It's a matter a finding a workaround this. 12:18:40 ... Or to use a cloud server. 12:18:59 ... For app-to-app communication, the second option is probably doable. 12:24:59 The group discusses layering of technologies, ways to split betwen natively supported features and features that could be provided at the JS level by some third-party library. 12:25:44 MarkFoltz: What I was trying to say earlier is that we'd be pushing the work from browser vendors to app implementers, which may or may not be an acceptable trade-off. 12:26:09 ... Generally, as there are security implications, that's something that we'd prefer to do natively. 12:27:27 ... The other rationale for doing more things in the browser is not share too much detail with the app about what's available on the LAN, but we could perhaps think of an mDNS variant specific to the Open Screen Protocol that only exposes the information that's absolutely needed, possibly on a per-origin basis. 12:28:00 cpn: Right, all the mDNS that had been raised previously still exist today. 12:28:05 MarkFoltz: Yes. 12:30:31 Francois: But an mDNS spec dedicated to open screen would still leak a lot of information on the LAN to the requesting Web app. 12:30:52 Peter: Right, but most of it is already known with WebRTC 12:31:45 Louay: We did some experimentation using an iframe from another domain as intermediate. 12:31:59 ... The iframe will use cookies to track things up. 12:32:07 ... Each device would have a connection to the server. 12:32:37 MarkFoltz: If we go this road, it would be interesting to understand whether browser vendors would converge to a registry of libraries that can initiate connections. 12:33:49 s/all the mDNS/all the concerns around mDNS/ 12:34:21 Topic: Goals for 2019 12:34:23 -> https://github.com/webscreens/openscreenprotocol/issues/98 [Meta] Open Screen Protocol 1.0 Specification 12:35:20 -> https://webscreens.github.io/openscreenprotocol/ Open Screen Protocol Editor’s Draft 12:36:49 MarkFoltz: The remaining work is to take the resolutions that we took in Berlin and during this meeting to GitHub and to the spec document with some appendix to describe all these CBOR messages 12:37:38 ... To complete the privacy and security questionnaire. If we resolve the issues currently opened, I believe we can consider that as the main deliverable of this community group. 12:38:26 anssik: Wondering when would be the right time to start a TAG review. 12:39:00 DavidBaron: The TAG has been reviewing specs at an earlier stage 12:40:09 ... Having specs to review without good ways to understand what the spec is about. Code examples are good. Pulling out things from several documents is doable, but if you have a thing that people reviewing the document can make sense of the spec, then that's a good moment to ask for a review. 12:40:29 MarkFoltz: We can back up an explainer from the API and WG/CG charters. 12:40:48 DavidBaron: Something that doesn't require following 15 links to figure out what's going on is good. 12:41:15 anssik: Part of this work has already been reviewed by TAG. We chose CBOR based on TAG's guidance. 12:41:59 ... OK, I'm hearing that we don't need a fully written document 12:42:38 MarkFoltz: Still not entirely clear whether we need an explainer document separate from the spec 12:42:59 dbaron has joined #webscreens 12:43:07 DavidBaron: as long as design questions are available somewhere, a full explainer may not be needed. We have an explainer's explainer 12:43:12 TAG's document on explainers ("explainer explainer"): https://github.com/w3ctag/w3ctag.github.io/blob/master/explainers.md 12:44:24 MarkFoltz: OK, that seems a priority. The spec itself will be fairly detailed at the end of the day. I'm not sure to what extent the TAG is willing to dive into implementation details. 12:44:40 anssik: We identified a few cases where we'd like TAG feedback 12:45:46 MarkFoltz: The explainer's explainer and the security and privacy questionnaire are the two things I'd like to focus on next. 12:46:15 anssik: That's probably even for this year. What are the goals for next year? 12:47:16 MarkFoltz: The goals of this group are completing TAG review(s), finishing a very detailed spec by the end of the year hopefully, and consider that as a 2019 outcome for the group. 12:47:37 anssik: Question about the formal standardization for this work afterwards. W3C, IETF. 12:48:21 MarkFoltz: Yes, assuming that we want to put it on the standards track, we need to figure this out. My initial thoughts are that most of the spec should go at IETF. 12:48:29 Peter: Certainly that would mirror the WebRTC model. 12:49:04 ... Are there other examples? What happened to WebSockets? 12:49:25 anssik: Same thing, protocol went to IETF, but not much engagement from the editor into IETF if I remember things correctly. 12:49:39 Peter: Any example of a protocol that was done in W3C but not in IETF? 12:50:31 sangwhan: Linked Data Platform 12:50:36 MarkFoltz: WebDriver 12:51:00 anssik: Any feedback from experience in WebRTC? 12:51:38 Peter: Definitely some overhead. Switching hats. The mismatch between TPAC and IETF meeting dates make things awkward from time to time. 12:51:56 ... The advantage would be if there would be significant overhead with other work at IETF. 12:52:19 ... If we had significant with CBOR group or mDNS group, then that would make sense. 12:52:32 ... I don't see any significant need for now. 12:53:15 MarkFoltz: QUIC and CBOR are the two immediate "new" needs that we have. We can simply wait until these things are done and use these. If we need to engage with them. 12:55:03 anssik: Can this work happen at W3C? 12:55:52 Francois: Provided there's enough support here, no objection from IETF, I don't see why not. In particular, that's not a pure protocol in the sense of a low-level protocol, it's an application-level protocol. 12:56:33 anssik: A bit too early to make decisions, apparently. 12:56:45 Peter: After we finish the first milestone and we look at streaming, balance may change. 12:56:49 -> https://www.w3.org/2014/secondscreen/charter-2018.html Second Screen Working Group Charter 12:58:10 MarkFoltz: Out of scope of this group, we will want to push our implementation of the open screen library. 12:58:44 anssik: We'll need to decide on transition(s) by the end of next year when the WG charter expires. 12:59:34 I think I misremembered the charter graph at https://w3c.github.io/charters-dashboard/ 12:59:51 Topic: Conclusion 13:00:36 anssik: I'm happy for the discussions here. Thanks a lot to Mark and Peter for the thorough work. We managed to attract a few other participants here, including Mark Arita with a security background. 13:00:41 ... We're making progress. 13:01:00 ... Thanks to our scribes! 13:02:08 ... No decision to meet before next TPAC yet, but we can plan that later on if needed. We don't need much. Room, projector, coffee, chairs, nice weather. 13:03:00 MarkFoltz: Minimizing total travel aligning with other events if there are other events where people would be going to, that would be good. 13:03:11 RRSAgent, draft minutes v2 13:03:11 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html anssik 13:03:13 ... FOMS next year on the East Coast. Perhaps the gaming workshop. 13:03:24 https://wiki.csswg.org/planning/hosting is some notes on things the CSS WG has learned in the past about things that people have done wrong when hosting a meeting :-/ 13:03:28 ... We can look at possibilities. 13:03:40 anssik: Yes, some time around May. 13:04:00 MarkFoltz: I also don't mind teleconferences as long as we don't do that too often. 13:04:07 ... We can talk offline. 13:04:23 RRSAgent, draft minutes v2 13:04:23 I have made the request to generate https://www.w3.org/2018/10/26-webscreens-minutes.html anssik 13:06:06 mfoltzgoogle has joined #webscreens 13:12:35 tidoust has joined #webscreens 13:16:55 tomoyuki has joined #webscreens 13:17:00 tomoyuki has left #webscreens 14:03:17 jernoble has joined #webscreens 14:09:14 Zakim has left #webscreens 14:53:21 tidoust has joined #webscreens 18:55:40 tidoust has joined #webscreens 19:09:34 tidoust_ has joined #webscreens 19:14:03 tidoust__ has joined #webscreens 20:38:59 tidoust__ has joined #webscreens