W3C/SMPTE Joint Workshop on Professional Media Production on the Web

Being not an end-to-end but a peer-to-peer technology and multi conferencing is the main use-case, the perception of the broadcasting industry around WebRTC has never been very good.

And it has not been seen as a viable solution for streaming media because it doesn't scale. It is weak on quality at best and it is hard to use.

The industry is really dominated by video-on-demand models like Netflix or Hulu, or even Disney with does not require real-time.

However, the pandemic has really changed all of this and it has accelerated the adoption of real-time media workflows that have finally bridged the gap between web and broadcast.

Real-time media has enabled a need for interactivity that requires near zero latency, with new use cases that big professional-grade workflows possible in consumer grade devices.

WebRTC is the perfect technology for delivering this real-time media.

It did what it was designed for in the beginning.

And despite some misconceptions, it is possible to deliver a high quality media with broadcast quality at web scale with additional properties like end-to-end encryption or simulcast and SVC support.

However, the lack of a standard signaling protocol has made it impossible for WebRTC to use a wide range of tools available and use on a daily basis by the streaming world. For example, OBS, FFmpeg, or vMix.

Not to mention that the lack of hardware encoders and physical input sources that are available here and integrated into many professional media workflows.

In the broadcasting industry, RTMP is still ubiquitous for live streaming and for media ingest, into hundreds or even thousands of media platforms.

WebRTC offers a lot of technical advantages when doing live media ingest like better network resiliency without increasing the end to end delay, by using SVC codecs or simulcast where they reuse the server-side processing required, with per-codec choices like VP9 or AV1, an end to end content as video for content, but over the delivery network.

When we tried to use WebRTC for media ingest, we realized that while WebRTC is the best media transport protocol for real time streaming, the lack of a standard protocol has made that each WebRTC streaming service requires implementing a custom ad-hoc protocol, making it a no-go for hardware encoders and broadcasting tools to adopt it.

Other media transports could be used for ingest, but using WebRTC for both ingest and delivery allow us to work natively in browsers, avoids protocol translation which adds delays and implementation complexity. Avoiding the transcoding by setting common codecs and use WebRTC features end to end.

So the solution is simple, we just need a reference signal protocol. The requirements for this new protocol would be that it might be simpler to implement. And easy to use at the current RTMP URI, support the specific ingest use-case which is a subset of the WebRTC possible use cases because we only need support for unidirectional flows. And we don't need support for re-negotiations.

We need to be fully compliant with WebRTC and RTCWeb specs but we have to lower the requirements and the complexity to reimplement it so it can be adopted by both hardware encoders and brodcasting tools.

One thing that was clear to us is there was that this new protocol should reuse the current Web technologies as much as possible. So we did use HTTP POST for exchanging the SDP offer and answer. And the connection state is controlled by the WebRTC ICE and DTLS states. The authentication and authorization is supported by most of the SDP rest nowadays does by the Authorization HTTP header. The standardization of this protocol is being done at the IETF.

And there are already some platforms that are implemented like Millicast or media server like Janus that support it, and we have some client implementations based GStreamer and of course in JavaScript.

The goal is to gain traction so we can get the hardware encoders to implement it and be able to use it in the same tools that are available for RTMP.

But this is all that is needed for using WebRTC in a professional media flow? Unfortunately not. And it is not mainly due to the lack of specifications, but mostly because the implementation of many of the features required is lagging behind in most of the browsers, and even with (?) API available to enable them, most of the time, they are in experimental status or hidden behind flags, undocumented or difficult to discover.

For example, some of the issues that we have found with audio is that WebRTC... we can support multichannel audio with Multiopus. Multiopus is not official standard and it's only supported by Chrome. And it is hidden, that requests SDP mangling in order to support it.

NetEQ, that is a jitter buffer implementation in all WebRTC browsers has issues with music. For example, Lorenzo has made a great presentation on the topic, describing all the issues that he has found when trying to use WebRTC for music. So I will encourage everyone to watch it.

Or for example, the integration between WebRTC and Web Audio has implementation issues on Chrome that either adds echo, when you play audio with audio captured by WebRTC and it has been acknowledged by Google itself.

Another example is that there is a lack of integration between WebRTC and WebVTT making live subtitles impossible.

Also on the video side. For example, SVC extension APIs only working in Chrome and it's an experimental feature, although this is likely going to change in the following weeks.

AV1 is only supported by Chrome and not enabled in Edge even if they share almost the same codebase.

VP9 profile which supports 10 bits is only supported by Chrome and only on the receiving side. And it is experimental in Safari.

The playoutdelayhint is an optional extension and it is only supported in Chrome.

Also, another sample is that abs-capture-time that is a header extension that could be used for synchronizing a video with standard metadata is only supporting in Chrome but it is again hidden and requires SDP mangling to enable it.

And at last, as an example of how professional-grade features are not always thought to be used with the WebRTC, video alpha is not supported, never meant to be supported in WebRTC but it is under consideration for being implemented in WebCodecs.

So as a conclusion, WebRTC today is available. And I will say, even, it is the best option for delivering professional broadcast quality media with minimal latency, but there is a still a lot of work to do in order to be able to WHIP WebRTC into shape.

But the good news is that we are on it.