13:54:33 <RRSAgent> RRSAgent has joined #me
13:54:38 <RRSAgent> logging to https://www.w3.org/2023/04/25-me-irc
13:54:38 <Zakim> Zakim has joined #me
13:56:28 <nhk-ryo> nhk-ryo has joined #me
13:58:22 <tidoust> tidoust has joined #me
13:58:27 <cpn> Meeting: MEIG monthly meeting: HDR on the web
13:58:30 <cpn> scribe+ cpn
13:58:34 <cpn> Agenda: https://www.w3.org/events/meetings/1a833aba-d317-49ca-8e5f-f68194a21c24
14:00:31 <cpn> present+ Chris_Needham, Hisayuki_Ohmata, Pierre-Anthony_Lemieux, Kinji_Matsumura, Tatsuya_Igarashi
14:00:42 <nigel> nigel has joined #me
14:00:44 <cpn> Chair: ChrisN, Igarashi
14:01:04 <cpn> present+ Kazuyuki_Ashimura
14:01:15 <cpn> present+ Ryo_Yasuoka
14:01:19 <tidoust> present+ Francois_Daoust
14:01:29 <nigel> Present+ Nigel_Megitt
14:01:30 <cpn> present+ Simon_Thompson
14:01:31 <igarashi__> igarashi__ has joined #me
14:01:34 <kaz> zakim, who is on the call?
14:01:34 <Zakim> Present: Chris_Needham, Hisayuki_Ohmata, Pierre-Anthony_Lemieux, Kinji_Matsumura, Tatsuya_Igarashi, Kazuyuki_Ashimura, Ryo_Yasuoka, Francois_Daoust, Nigel_Megitt, Simon_Thompson
14:01:47 <cpn> present+ Nigel_Megitt
14:02:22 <kaz> agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2023Apr/0008.html
14:02:59 <cpn> present+ Xabier_Rodríguez_Calvar
14:04:26 <ohmata> ohmata has joined #me
14:04:40 <cpn> Topic: Introduction
14:04:50 <calvaris> calvaris has joined #me
14:04:57 <cpn> ChrisN: Welcome all, today's topic is HDR support on the Web
14:05:48 <cpn> ... Thank you Pierre for proposing the topic, and we can figure out next steps
14:06:00 <cpn> Topic: HDR support on the web
14:06:13 <kaz> q+
14:06:52 <kaz> ack k
14:07:08 <cpn> Pierre: https://mit.webex.com/mit-en/url.php?frompanel=false&gourl=https%3A%2F%2Fgithub.com%2Fw3c%2FColorWeb-CG%2Fblob%2F380d4a74eff227f31a4ed3c958398842bbf2c507%2Fhdr_html_canvas_element.md
14:07:24 <cpn> ... This has been a long adventure, the goal has been to bring HDR imagery to the web platform
14:07:39 <kaz> s|https://mit.webex.com/mit-en/url.php?frompanel=false&gourl=https%3A%2F%2Fgithub.com%2Fw3c%2FColorWeb-CG%2Fblob%2F380d4a74eff227f31a4ed3c958398842bbf2c507%2Fhdr_html_canvas_element.md|https://github.com/w3c/ColorWeb-CG/blob/380d4a74eff227f31a4ed3c958398842bbf2c507/hdr_html_canvas_element.md|
14:07:42 <cpn> ... I'm sure you're all familiar with HDR. It's in widespread use in consumer streaming, media, and TVs
14:07:44 <kaz> rrsagent, make log public
14:07:48 <kaz> rrsagent, draft minutes
14:07:49 <RRSAgent> I have made the request to generate https://www.w3.org/2023/04/25-me-minutes.html kaz
14:08:23 <cpn> ... It's available in the video element in narrow use cases. It works on Edge with specific monitor and video card combinations
14:08:39 <cpn> Nigel: When you say imagery, is that video or still photos too?
14:08:46 <cpn> Pierre: It's both
14:09:18 <cpn> Pierre: There are different ways of doing HDR, the way in photography is different than that used for motion picture and TV content
14:09:36 <cpn> ... And different for authoring of motion picture and TV content. That's part of the complexity of doing HDR
14:10:06 <cpn> ... The fundamental limitation on the web today, is the canvas element, that allows drawing of arbiratry images, lacks three main features for HDR images:
14:10:29 <cpn> ... Color spaces used for HDR images, today it only supports Display P3 (on some platforms) and SRGB (on all platforms)
14:10:48 <cpn> ... Limited to 8 bit per color channel, so you see banding. That becomes intolerable with HDR images
14:11:01 <cpn> ... It's also missing metadata that's used in some HDR applications
14:11:24 <cpn> ... The goal of the Color CG is to address that, how to modify the HTML canvas element to allow HDR images to be drawn
14:12:14 <cpn> ... Typical canvas use cases: drawing images not supported by <img> tag, collage of images of different sources, adding graphics to an HDR image, or draw an HDR image that matches a video in a <video> tag
14:12:47 <cpn> ... With video, there's a path to supporting HDR video. It's basically an opaque element, you just give it a video stream, and it plays
14:13:03 <cpn> ... Same thing with the <img> tag, but HDR isn't supported there at all, AFAICT
14:13:16 <cpn> ... Interaction with canvas element and modify it at will
14:13:43 <cpn> ... As a group, we've been through multiple iterations on a strawman proposal, over the years. It's gone from simple to being complicated
14:14:01 <cpn> ... Over last few weeks, I've been working to bring it back to a minimal proposal for HDR in HTML canvas.
14:14:31 <cpn> ... Not intended to be the ultimate answer for doing HDR on the web, but based on current practices in the M&E industry, where there are standards in widespread use
14:14:41 <cpn> ... Not intended to preclude other ways of doing HDR on the web
14:14:56 <cpn> ... Any questions so far?
14:15:24 <cpn> ChrisN: What's driven you towards simplifying the proposal?
14:15:47 <cpn> Pierre: We're three years into the process, but still didn't have a concrete proposal that people are implementing, it's been a moving target
14:16:24 <cpn> ... We wouldn't be discussing today if there were some level of support in HTML canvas. So it's an attempt to get to consensus on a minimal starting point, be less ambitious than we were at the beginning
14:16:51 <cpn> ... Initially, people looking at new ways of doing HDR images, using linear color spaces, things more close to what might be done in photography
14:17:17 <cpn> ... So the starting point was more complex, speculative, risky - hence people that wanted to implement concluded it's complicated so wanted to wait
14:17:38 <cpn> ... So proposal is to go back to a more well understood path
14:18:22 <cpn> ... The proposal is straightforward, add colorspaces for HDR images - I mean colorimetry systems. HDR canvas only supports srgb and display-p3. Add rec2100-hlg and rec2100-pq
14:18:59 <cpn> ... These are non-linear spaces, using r', g', b' as specified in ITU BT2100. These aren't supported in CSS Color 4, unfortunately
14:19:16 <cpn> ... But many things in CSS Color 4 aren't supported, may be prototype implementations?
14:19:48 <cpn> ... One thing more controversial is how to increase the bit depth of canvas beyond 8-bit. Proposal to start with float16 - that's not super controversial, it's widely used for GPU operations
14:20:07 <cpn> ... The part that is controversial, surprisingly, is the format used when exporting the pixel array outside the canvas element
14:20:27 <cpn> ... An interesting issue is float16 arrays aren't supported in JS today, only float32 arrays
14:21:36 <cpn> ... There seems to be two camps. It seems hard for the community to decide whether to export as a float32 array of numbers or float 16 array. float16 requires modifying the language, no objections to that, but requires going to ECMA TC39
14:22:14 <cpn> ... Some see that as a bridge too far. Some argue using float32 array, but it's a waste of memory bandwidth - and becomes an argument against processing float16array
14:22:36 <cpn> ... This proposal doesn't care, anything greater than 8 bit would work
14:23:38 <cpn> ... Last item, that requires some work, and not explored in detail by existing use cases: It's customary for some kind of HDR images, e.g., PQ, is to use metadata to indicate what mastering display was used in creating the images
14:24:23 <cpn> ... Color volume, primaries, white point of the display. No spec for how to use that data for how to use or tone map with the data. But without it it's not possible to match the content of a video element that would have similar metadata
14:25:00 <cpn> ... Another fundamental issue identified is what happens if you draw multiple images, each with different metadata, to the canvas. The canvas has only one set of metadata - so ignore, take the superset?
14:25:33 <cpn> ... But most use cases in M&E are single full screen content, so there's only one set of metadata. Needs investigation, figure out how the metadata is used, or decide to completely ignore it
14:26:34 <cpn> ... Something else that's a significant part of the complexity, is there's a good understanding of how to convert between SDR color spaces, and a lot of questions about convering from SDR to HDR color spaces and the reverse
14:27:00 <cpn> ... and between HDR spaces. Progress made in recent years. ITU and SMPTE have recommendations, so it felt important to include in the strawman proposal
14:27:18 <cpn> ... How to tone map is an inexact science, as it loses information. But there are guidelines and recommendations on how to do that
14:27:44 <cpn> ... I put together a demo of tone mapping HDR to SDR using various methods
14:27:47 <cpn> ... @@
14:28:08 <cpn> ... The first image the colors are wrong, it looks flat. But when tone mapped it has more contrast
14:28:27 <cpn> ... This uses two methods: SMPTE and ITU
14:28:59 <cpn> ... So it shows it's not impossible to do it, e.g., if you're doing a collage with multiple images
14:29:38 <cpn> ... That covers the proposal, I'm presenting it to get consensus among various groups. Progress on issues such as how to combine metadata from multiple images
14:29:50 <cpn> ... Can discuss now, raise issues, or contact me privately
14:29:52 <nigel> q+
14:30:00 <tidoust> q+ to wonder about alignment with 3D contexts (webgl and webgpu)
14:30:32 <nigel> scribe+ nigel
14:30:36 <cpn> ChrisN: Simon also has a demo
14:30:58 <nigel> ChrisN: [shares screen with BBC demo]
14:31:03 <ChrisLorenzo> ChrisLorenzo has joined #me
14:31:21 <nigel> SimonT: We took Pierre's demo and added some more methods.
14:31:27 <nigel> .. We wanted to look at HLG as well.
14:31:46 <nigel> .. Still from a camera test chart as the HLG original image.
14:32:04 <nigel> .. Simple tone mapping method. The colours look as you'd expect on an sRGB image.
14:32:17 <nigel> .. Question of an HLG canvas with PQ, or vice versa, can you do the transform?
14:32:34 <nigel> .. We took PQ into HLG, relatively straightforward transform, undo one and apply the other.
14:32:48 <nigel> .. If you've watched e.g. the NFL Superbowl recently you'd have seen this transform in action.
14:33:02 <nigel> .. The SMPTE and ITU transforms are both very nice, a couple of hundred lines of code,
14:33:22 <nigel> .. whereas the simple transform to HLG we put into HTMLCanvas, with a colour space transform only,
14:33:28 <nigel> .. takes only a couple of lines of code.
14:33:45 <nigel> .. Then we wondered if we could do PQ -> HLG -> sRGB, and we found we could.
14:33:51 <kaz> present+ Alicia_Boya_Garcia
14:33:58 <nigel> .. It could be improved, it doesn't have tone mapping and it does clip colors, but as
14:34:01 <kaz> present+ Chris_Lorenzo
14:34:10 <nigel> .. a basic demonstration, it works, and is computationally a lot simpler.
14:34:11 <kaz> chair+ CHrisL
14:34:18 <kaz> chair- CHrisL
14:34:24 <kaz> chair+ ChrisL
14:34:29 <kaz> rrsagent, draft minutes
14:34:30 <RRSAgent> I have made the request to generate https://www.w3.org/2023/04/25-me-minutes.html kaz
14:34:36 <nigel> .. As a minimum viable product, we'd probably end up with some different but slightly better appearance.
14:34:36 <cpn> q?
14:35:37 <cpn> Nigel: Pierre, in your presentation you showed something about a canvas mastering display. Does that mean that there isn't a canonical exchange format for the data? It seems strange to send data about the mastering display
14:35:54 <cpn> Pierre: It's often called SMPTE 2086 metadata
14:36:19 <cpn> ... The latest PNG spec draft that includes HDR, as well as every codec and file format that includes HDR will include that metadata
14:36:45 <cpn> ... It describes the mastering display, the implication is that none of the pixels in the image extend beyond that color volume. Intended to simplify tone mapping
14:37:02 <cpn> ... But most applications ignore the metadata today. And it's not specified how to use the metadata
14:37:32 <kaz> rrsagent, draft minutes
14:37:33 <RRSAgent> I have made the request to generate https://www.w3.org/2023/04/25-me-minutes.html kaz
14:37:57 <cpn> ... I'm not a big fan of the metadata, but the reason it's in the strawman proposal, is to match the content in a video tag. So if there's a Netflix video in the video tag, and the Netflix logo in an image - if you dont' use the same metadata, the two may not be rendered the same
14:38:17 <cpn> ... This is an opportunity to get to what the metadata is for, and potentially ignore it
14:39:08 <cpn> Nigel: On a technical argument that it's important to convey for presentation, I can see it working if there's a non linear mapping between the mastering display and rendering display - so it depends on capabilities of both dispays. Is that the case?
14:39:48 <cpn> Pierre: In terms of luminance, the tone mapping uses knowledge of min an min luminance of the image to map it to SDR, so you map between the max and min
14:40:05 <cpn> ... So that's an example where it helps do a better tone mapping.
14:40:23 <cpn> Nigel: But that's sourced from the mastering display and not the image itself
14:40:32 <kaz> q?
14:40:33 <cpn> q?
14:40:35 <cpn> ack n
14:41:24 <cpn> Pierre: [shares screen again] The ITU method assumes the HDR has a typical average and median luminance to do the mapping, so it uses no information about the image nor the mastering display
14:42:00 <cpn> ... The SMPTE method tries to map only the part of the HDR image that actually matter. So this method takes as input the max, min, avg luminance when mapping to SDR. So you don't have to compute it
14:42:24 <cpn> ... Can use the mastering display min/max luminance as a proxy for the image luminance
14:42:39 <kaz> -> https://www.sandflow.com/public/tone-mapping/index.html demo page
14:42:50 <cpn> ... So knowing some characteristics of the display can help with tone mapping
14:43:11 <cpn> Nigel: So having this standardised is beneficial, rather than proprietary?
14:43:27 <cpn> Pierre: Yes, metadata without a normative spec to use it isn't super useful
14:44:02 <cpn> ... I'm not a browser implementer, but the debate about float16 and float32 should be a lot more complex
14:44:30 <kaz> rrsagent, draft minutes
14:44:31 <RRSAgent> I have made the request to generate https://www.w3.org/2023/04/25-me-minutes.html kaz
14:45:08 <cpn> Francois: Thanks for the proposal and discussion. So this would extent canvas 2d contexts, it can also use WebGL or WebGPU. Each would have to define its own way of handling HDR. Have you aligned with people behind those specs?
14:45:32 <tidoust> -> https://html.spec.whatwg.org/multipage/canvas.html#dom-canvas-getcontext-dev canvas.getContext()
14:45:56 <cpn> Pierre: Earlier versions of the proposal link to discussions in the WebGL and WebGPU communities. Don't think anything is incompatible, float16 is uncontroversial with WebGPU. Goal is to have a unified approach, use HDR canvas as a test case
14:46:27 <cpn> ... Trying to resolve all those questions, including those about metadata, then propose extending it to WebGL and WebGPU, as an approach
14:46:38 <cpn> ... Have the same approach applied to those other contexts
14:47:12 <cpn> Francois: I value this progress, but asking just to check about thinking about those aspects
14:47:57 <cpn> Pierre: Because of the nature of W3C, those communities are somewhat independent, hard to wrangle, so start simple
14:48:04 <cpn> ChrisN: What can our group do to help?
14:48:44 <cpn> Pierre: Please review the proposal, and send feedback. Please let us know now. Talk to your web developers, would it help them? If you have a strong view of whether HDR metadata is important, that would help
14:48:54 <cpn> ... Next quarter is about gathering feedback from a range of groups
14:49:03 <cpn> ... At some point the TAG should look at this
14:50:12 <cpn> ChrisN: Is the complexity coming from general web pages combining images from different sources?
14:50:53 <kaz> q+
14:50:53 <cpn> Pierre: Don't think having multiple having multiple images or canvases is a big issue. You can do display-p3 and srgb, mix canvases. On Windows I have a monitor in HDR mode, SDR and HDR windows, it works
14:51:28 <cpn> ... I don't think the complexity comes from that. From a page author point of view, it's a different issue
14:51:29 <cpn> ack t
14:51:29 <Zakim> tidoust, you wanted to wonder about alignment with 3D contexts (webgl and webgpu)
14:51:58 <kaz> q+ Alicia
14:52:05 <kaz> q- later
14:52:06 <kaz> ack a
14:52:41 <cpn> Alicia: I don't have an HDR display myself, I have a display that's more that SRGB. I've dealt with colorspaces in photography. How could we handle, either now or with this proposal, colorspaces attached to image files that we draw in a canvas?
14:53:29 <cpn> ... For instance, I can have an image in display-p3 or a custom monitor profile, embed ICC profile in a JPEG or PNG image. Generally if an application does color profiles correctly, you'll see the image somewhat OK depending on capabilities of your display
14:53:54 <cpn> ... I read a post from WebKit, which I think was the only browser that did this in the img element. Not sure how much traction it got
14:54:38 <cpn> ... One thing is rendering in an img element, but what happens when you try to blit that element to a canvas, then try to fetch the pixels back from the canvas? Or what happens in between, what kind of pixels could browsers be expected to store in the canvas?
14:54:48 <kaz> s/SRGB/sRGB/
14:55:03 <cpn> Pierre: If you instantiate a rec2100pq canvas, and the img tag has a PQ image, I'd expect the same pixels to be blitted
14:55:44 <cpn> Alicia: But it's different to other pixels inside the canvas, the canvas needs to be aware: in a rectangle there are pixels outside sRGB?
14:56:16 <cpn> Pierre: So you pick the color space for the canvas. If you instantiate the canvas with rec2100 color space and write an sRGB image, you won't get sRGB pixels out, there'll be a mapping
14:56:30 <cpn> Alicia: I think of it as the color space for the canvas
14:57:11 <cpn> Pierre: You can pick between display-p3 and srgb for the canvas today. So we're proposing to add ability to create as a rec2100-pq and rec2100-hlg, and the implementation tone maps
14:57:47 <cpn> Alicia: Is this already implemented? There has to be some kind of API
14:58:06 <cpn> Pierre: Today, when you create the canvas there's a parameter that specifies the color space
14:58:17 <cpn> ... It's not supported universally
14:58:24 <kaz> q?
14:58:28 <kaz> q-
14:58:41 <cpn> Alicia: So we'd be giving it new values so that the color spaces can be used.
14:59:57 <cpn> Topic: Chartering MEIG
15:00:45 <cpn> Kaz: We discussed draft charter in previous call. We haven't added a Background section. So suggest moving ahead without it, to get reviews
15:01:04 <kaz> -> https://w3c.github.io/media-and-entertainment/charters/charter-2023.html Draft Charter
15:02:27 <cpn> ChrisN: I agree. Proposed resolution is to go ahead with the charter as currently drafted, but remove the template Background and Motivation section (which is just a placeholder currently)
15:02:33 <igarashi__> +1
15:02:43 <cpn> ... Any objections?
15:02:44 <cpn> (none)
15:03:01 <cpn> ... So that's our group's decision, resolved
15:03:19 <cpn> RESOLUTION: MEIG would like to move forward with the current proposed charter
15:03:34 <kaz> -> https://w3c.github.io/media-and-entertainment/charters/charter-2023.html proposed Charter
15:05:12 <kaz> [adjourned]
15:05:38 <kaz> rrsagent, draft minutes
15:05:39 <RRSAgent> I have made the request to generate https://www.w3.org/2023/04/25-me-minutes.html kaz
15:08:02 <cpn> s/@@/https://www.sandflow.com/public/tone-mapping/index.html/
15:09:22 <cpn> i/We wanted to look at HLG as well/-> https://waveform.prototyping.bbc.co.uk/hdr/ HLG demo images/
15:09:33 <cpn> rrsagent, draft minutes
15:09:35 <RRSAgent> I have made the request to generate https://www.w3.org/2023/04/25-me-minutes.html cpn
15:09:40 <kaz> rrsagent, draft minutes
15:09:41 <RRSAgent> I have made the request to generate https://www.w3.org/2023/04/25-me-minutes.html kaz
15:12:37 <kaz> rrsagent, stop