W3C/SMPTE Joint Workshop on Professional Media Production on the Web

Hi everyone, I am Pierre and I co-chair the Joint W3C/SMPTE Workshop on Professional Media Production on the Web.

Today I want to share with you recent advances that make it possible to work with professional media in any web browser... and demonstrate it is actually possible to play lossless UHD video, directly from an HTTP server, without proxies and without plugins. But first some background.

Once upon a time there was a huge gap between professional content and the web browser. The two used different media formats and codecs. The two targeted different qualities of experience. And professional content was primarily on-prem whereas the browser was Internet-first … but times have changed.

First, the web browser has come a long way since 2000. Back then, the web was text with the occasional video playing through plugins.

Fast forward to today, the web now allows complex multimedia experiences through standard APIs.

Particularly important for professional content, WebAssembly now allows C/C++ libraries to be ported directly to the web.

This makes it possible to use professional media formats and codecs that are not natively supported by web browsers.

Second, something that seemed improbable just 10 years ago has happened: studio content is now processed in the public cloud.

This is a stunning reversal for an industry that was largely based on video tape and perceived the Internet as the wild west.

There is probably not a single event behind this transition. Perhaps it was the disruption in tape supplies caused by the Tōhoku tsunami in 2010, or Netflix’s own experience with on-prem database corruption in 2008.

In the end, the cloud is not only more efficient but also more secure since fewer humans handle the content and fewer copies are made. … and all major studios now have moved some of their workflows to the public cloud.

So I think the time is right for professional media to embrace the web platform. My personal epiphany came actually in the middle of pandemic when I ask myself: can UHD lossless video be played in a web browser without proxies?

Professional UHD lossless video is different than the typical video supported natively by web browsers.

The images are often sampled at 48-bit per pixel, instead of the typical 24-bit per pixel for consumer video.

The images are routinely losslessly compressed using codecs like JPEG 2000, instead of using AV1 or AVC lossy compression. This results in bitrates in Gbps, as opposed to Mbps.

The compressed image is ultimately wrapped in file formats like MXF, again different than the formats used for consumer delivery.

The traditional approach to playing professional video in web browsers has been to transcode the professional video into a format understood by the web browser.

This results in a proxy.

For example, starting from a 5 Gbps JPEG 2000 MXF file, you might end up with a 50 Mbps AVC MP4 file.

This approach has significant drawbacks: there is latency in that conversion, and metadata and image fidelity are reduced.

A no-proxy architecture takes a different approach.

In this approach, the professional video is compressed using JPEG 2000 and wrapped in an MXF file, which is stored on an HTTP server.

Using the MXF index table, the web app can access each individual image frame directly using standard HTTP byte-range requests.

Depending on the network conditions and the size of each frame, the web app will end up reading complete or partial frames.

Using an open source JPEG 2000 C++ library compiled to WebAssembly, the web app can then decode the partial or complete image frames.

Because frames are encoded using JPEG 2000, decoding a partial frame will not fail and instead will result in a decoded image with reduced resolution.

This ability for a partial read of a frame to result in a lower resolution image is a standard feature of JPEG 2000 resolution progressive codestreams.

In such a codestream low spatial frequency information is stored in the lower bytes of the codestream. Whereas the high spatial frequency information is stored in the higher bytes. As a result, reading a partial frame results into a lower-resolution decoded image.

Here’s a live demo and proof-of-concept of the no-proxy architecture, which I encourage you to visit directly.

The video is lossless UHD content at 48-bits per pixel, which amounts for about 5 Gbps.

The file is stored on S3 and distributed through CloudFront.

The quality of the video will depend on your available bandwidth, but is unlikely to be pristine given the bit rate involved.

Notice, however, that when the video is paused, the app can download the full frame, which results in pristine quality.

This is of course only a proof of concept, so much improvement is possible.

It however demonstrates that it is possible to play back UHD lossless video in the browser in a way that adapts to network conditions, using standard web APIs and open source libraries, without any special web server.

In summary I really believe the time is right for professional workflows to embrace the web platform.

The web can support professional codecs and formats and professional media is moving to the cloud.

Some gaps nevertheless remain. For example the web platform really lacks support for high-dynamic range and wide-color gamut images. And sample and accurate media synchronization can really be improved. Hopefully these gaps can be filled in time.

Thanks again for listening. Do not hesitate to contact me. Bye!