W3C

– DRAFT –
MEIG / WebRTC WG Joint Meeting

05 April 2022

Attendees

Present
Ali_Begen, Barbara_Hochgesang, Bernard_Aboba, Bill_Mears, Carine_Bournez, Chris_Needham, Christian_Timmerer, Dominique_Hazael-Massieux, Elad_Alon, Florent_Castelli, Francois_Daoust, Harald_Alvestrand, Jan_Ivar_Bruoraey, Julia_Kenyon, Kaz_Ashimura, Kensaku_Komatsu, Klaus_Volk, Ofer_Shem_Tov, Piers_O'Hanlon, Sergio_Garcia, Sergio_Garcia_Murillo, Tatsuya_Igarashi, Thomas_Stockhammer, Tim_Panton, Tony_Herre, Yasser_Syed
Regrets
-
Chair
Chris_Needham, Tatsuya_Igarashi
Scribe
cpn, tidoust

Meeting minutes

cpn: Welcome. Joint call with Media & Entertainment IG and WebRTC WG on W3C side, and DASH-IF participants to discuss recent report on use of WebRTC for media streaming.
… Julia will start with presentation, then discussion.

W3C considerations for WebRTC and DASH for interactive streaming

Slideset: https://www.w3.org/2011/webtv/wiki/images/8/87/DASHandWebRTCforW3C_20220405.pdf

[ Slide 2 ]

[ Slide 3 ]

Julia's slides: https://www.w3.org/2011/webtv/wiki/images/8/87/DASHandWebRTCforW3C_20220405.pdf

Julia: My colleagues and I have been working on delivering content with WebRTC for some time. We noted some topics that intersected with DASH.
… 3 categories: fallback (WebRTC or DASH as fallback), interleaved (switching between the two mechanisms), and concurrent (WebRTC content playing at the same time as DASH content, e.g. pre-recorded match from a participant in a sports message)
… We came to DASH-IF to understand whether there could be a standard way to address these use cases.

WebRTC NV use cases: https://www.w3.org/TR/webrtc-nv-use-cases

Julia: There are related WebRTC NV use cases, e.g. low latency P2P broadcast of "funny hats".

[ Slide 4 ]

[ Slide 5 ]

Julia: When we're talking about WebRTC broadcast, note there is a media server involved that distributes to plenty of users. Broadcast needs to scale.

[ Slide 6 ]

Julia: The way we see the current workflow is proprietary steps followed by regular WebRTC. We'd like standards all the line

[ Slide 7 ]

Julia: Some work: session negotiation and management, and continue development of methods for additional security of streams. Not just for meetings but also for premium content.
… If we can get through these today, that would be great. Other topics include synchronization mechanisms for metadata and DASH periods.

[ Slide 8 ]

Julia: For session negotiation, we need a specific transport method. We'd like that to be interoperable. This is covered in depth in the report

[ Slide 9 ]

Julia: There is of course the WHIP protocol, which is specific to ingest.
… There's a companion access protocol, called WHAP.

[ Slide 10 ]

Julia: We're working on something called WHSNP. Our goal is to be similar using HTTP and JSON.

[ Slide 11 ]

Julia: As far as security goes, there's SRTP.

[ Slide 12 ]

Julia: For DRM, we'd like to see secure key exchange.

[ Slide 13 ]

Julia: We'd like to see WebCodecs + EME without CMAF.

[ Slide 14 ]

Julia: As far as the future of developing technical proposals, we're seeking inputs and feedback from experts.
… Any feedback on content negotiation and content protection?

Bernard: The latest slide mentions WebCodecs. We do have an open issue related to content protection of non-containerized media
… We haven't made a lot of progress.

WebCodecs content protection issue: https://github.com/w3c/webcodecs/issues/41

Bernard: You're more likely to see progress on WebCodecs than in RTP.
… The other thing that I wanted to mention, I noticed that a great deal of scenarios are implemented using WebRTC but some could also be done through WebTransport and the like.
… Nuance between ultra-low latency (WebRTC) and low-latency (which you can perhaps get through other means)

tpanton: I implemented a WHIP client.
… Are you thinking you want these features provided by the user agent? Or through JS libraries?
… We also need to have a clear definition of end-to-end encryption. Everyone seems to have their own definition.

Julia: If features were implemented in the browser, that would save a lot of troubles.
… Is the usual path to go through a library first?

cpn: That can be a useful path.

dom: DASH.js is being widely deployed and yet not implemented by browsers.
… We need to determine whether it is worth standardizing specs to be implemented by browsers.
… There are cases when you must, when you cannot access e.g. some bits of the network stack.

hta: Same point as Dom.
… It's probably wise to use "ms" instead of "ultra-low" and "low" because the terms mean different things to different people.
… Most features presented here make sense to me.
… A common idiom is the notion of clock. Something we don't have is timestamps throughout all data.
… RTP timestamps are in, with translation to NTP timestamps. But RTP timestamps are arbitrarily set from the beginning of the streaming, so you can't figure what they mean unless you're at the origin.
… Mapping to NTP is ok for real-time use cases, not so great for stored data.

Bernard: There are specs like sending TTML as streams but they're typically not integrated with WebRTC.

tpanton: The data you send in the data channel is also not in sync with the media streams.

jib: Always great to see WebRTC used in new use cases. Impendance mismatch between what the API offers and the needs. Do these use cases rely on data channels mostly? Or do they also use audio/video WebRTC streams?
… Our WG tries to figure out what needs to be standardized in browsers because they cannot be done in JS. Signaling was left out of scope precisely because it can be done in JS.
… What are the problems that standardizing this would solve? Related to CDNs, other?

Julia: WebTransport + WebCodecs could be used. Chosen to use WebRTC because it has low latency and the browser support. Could use those but we're all in on WebRTC right now. Others are also looking at webrtc
… We switched to WebRTC because it was natively supported in the browsers. Preferable to WebTransport + WebCodecs which is not yet supported.
… The problem with signaling is that everyone needs to have its own client. To get to the point when you can start streaming, you need to download a lot of code, as opposed to say something like: "load a manifest and you're done"
… A common solution would make things faster.

Ofer: In the video streaming industry, there are vendors doing players and vendors doing streaming. Players need to integrate with different streaming systems.
… avoiding vendor lock-in.

dom: Part of answering the question would be to clarify who needs to implement the standards.
… Jan-Ivar's point is that the bar is high for browser vendors.
… For manifests and signaling, there could well be a standard, e.g. done in IETF, that would not be implemented by browsers, but that would be useful.

Ofer: I agree. It does not have to be part of the browser. Like DASH.

jib: Standardizing the manifest seems like a good solution. I wasn't clear about the exact problem. I'm understanding that there are vendors that don't have full control over the entire flow.

tpanton: To kind of paint the picture of what that might look like, you could add a URL to a video tag and the browser would do whatever is needed to turn that into a media stream.
… You can do that in a library-free way and I can see the attraction in being able to do that.
… Especially having played with WHIP on the ingestion side of things. That is quite compelling.
… Problem is that it is barely related to the initial scope of WebRTC.

Ofer: When we worked on the report, it was clear that part of the features we need can be done through extensions, e.g. to DASH.
… Some, we cannot do, though. E.g. DRM.

kaz: I also work for the WoT WG. Interested in other use cases, e.g. related to smart building for video camera surveillance.
… Potential sensors and actuators could be integrated in that mechanism, and time synchronization would be fairly important there.

cpn: Potentially. I think we're discussing here coming from the media industry that currently uses DASH or HLS for streaming.

<tpanton> Kaz - I've done work on webRTC in IoT cameras - I'd be happy to discuss this offline.

cpn: The broad question I have is, to what extent do the requirements that we're hearing, align with the future direction that WebRTC wants to go?

Bernard: When I talk to developers, e.g. game developers, there is a lot of interest in WebTransport and WebCodecs, as well as for WebRTC Data Channels.
… People put metadata with the media to deal with synchronization issues.
… Obviously, there remain some open issues on WebCodecs, but you can achieve a number of things already.
… WebRTC is more directly focused on two-way communications. For one-way, you may be better served through another approach.

jib: One note is that WebTransport is fairly low-level, and leaves a number of features up to application developers. To integrate that in a compelling streaming offering is not an easy task.

<dom> [this may be something M&E IG could track]

Bernard: I think it would be nice for W3C as a whole to have a list of use cases. Some of them are in the WebRTC NV document. Others could be done in other groups. It's hard to understand which group is responsible for addressing the use cases.

cpn: The DASH-IF produced a nice set of use cases. I'd like to map them to WebRTC use cases. This is something that the M&E IG would be happy to collect and track.

Thomas: A key issue is full end-to-end system. DASH gives you adaptive streaming, etc. That is very well defined.
… If we look in the future, is this going to be RTP? CMAF?

Bernard: It's pretty much CMAF.

Thomas: WebRTC has plenty of features built-in. You have an existing player in the browser and the beauty of WebRTC is that you can give it whatever you want and it will work. From a client perspective, WebRTC gives you the same thing as MSE.
… The only thing missing is DRM.
… Not convinced that we need another layer.
… I'd like to leverage both existing mechanisms: one focused on CDN scaling, another focused on low-latency.

Piers: If you're talking about single frame, CMAF, the difference on latency may not be that far off between the approaches.
… The timing levels in the browsers are pretty fine-grained.

Sergio: I find it disappointing that we push things to WebCodecs when WebRTC could be solving these use cases.
… I'm the author of the WHIP protocol.
… I think that having a standard protocol for media streaming is a good idea.
… For TVs, you cannot easily use WebRTC in devices, it would be easier to direct the TV at a manifest.
… It would be better to specify a protocol for that. IETF would be a better place, probably.
… The most critical missing piece is, for me, the DRM, which is going to be difficult to solve.
… We can provide end-to-end encryption, but that is meant to protect the user, not the content against the user.
… We need to check if it is really an issue. If we're talking about live streams, how much of an issue is it to access the live stream?
… The two other things that we can track, is how to improve the media quality of WebRTC streams.
… E.g. for music, pitch changes, etc.
… Also color formats, 4:2:2 in 10 bits.
… Last one is synchronization between the data channel and the media.
… How to deliver metadata with frame accuracy is a problem to solve.

cpn: Lots of good points. Doing an evaluation of that list of use cases. Breaking them down to concrete needs. And understanding which ones are in scope for the working groups, and the WebRTC working group in particular.
… What are DASH-IF next plans?

Thomas: We have an interest survey going on. To what extent we believe that solutions should be done as interoperability guidelines. To basically get some reference library.
… Secondly, are there new requirements that need to be pushed onto IETF, W3C, etc.?
… But main is to check whether there is enough interest to work on these guidelines.

cpn: I will add a link to the survey in the notes.

<kaz> DASH & WebRTC Survey

cpn: From a W3C perspective, the Interest Group is certainly here to facilitate conversations that you want to have and continuing the exploration on requirements.
… Thank you everybody for the great discussion!

Julia: Thank you for your time and interest. Greatly appreciated.

Next meeting

cpn: May, 3rd. Agenda to be confirmed.

<kaz> [adjourned]

Minutes manually created (not a transcript), formatted by scribe.perl version 185 (Thu Dec 2 18:51:55 2021 UTC).