Color on the Web CG meeting

Introductions

<cyril> scribe: Cyril

Introductions

Chris, vice-chair of the ICC working group

ICC secretary

Francois Daoust, W3C staff

Chris Needham, BBC

Mark Watson, Netflix

Cyril Concolato, Netflix

Greg Freedman, Netflix

Chris Lilley, W3C

JEan-Yves Avenard, Mozilla

Jer Noble, Apple

Chris Cunningham, Google

Simon Fraser, Apple

chris: 3 topics for discussion
... HDR on the web
... Image format
... Working color space and CSS

simon: prefer to start with CSS

Working Color Space

chris: in CSS, sRGB was used from the beginning
... nobody cared about that for a while
... then we added transparency with compositing in gamma-encoded space, unfortunate but for compatibility for Photoshop
... then in CSS Color 4 we added more color spaces
... but when you have 2 color spaces, there is a need to describe what happens with compositing
... and for compatibility we still used sRGB for compositing, which has problems as it is not linear-light
... to my mind, we need at least XYZ and Lab

but Apple uses linearized sRGB

scribe: with negative values for wide gamut

markw: I think Microsoft also uses that

chris: I looked at scRGB
... but it seemed scene-refered
... linearized seems a reasonable option
... but currently browsers store a triplet of RGB in 8 bits
... there's going to be an increase in space
... storage space
... maybe some parts of the page could still use sRGB and other the linearized space

simon: it seems that linear sRGB is an option?

chris: yes

simon: having a compositing color space as a CSS property would be problematic
... with impact on the stacking context
... implementations often pop out elements into GPU textures
... on Apple platform, the architecture that we use does not allow the control of the color space
... we would only have control on a per page basis

chris: I can see advantages of that
... but we you switch it on, it changes the whole page

markw: the obvious question that bleeds into the next topic
... true sRGB are going to map to 0,1
... how do I specify luminance values that are outside the sRGB range

chris: sRGB says the luminance is 80nits, but it's not true

markw: [scribe lost]

<markw> how do we get HDR colors into this sRGB linear space ?

<chris> ITU-R BT.2408-2 states "Graphics White is defined within the scope of this Report as the equivalent in the graphics domain of a 100% reflectance white card: the signal level of a flat, white element without any specular highlights within a graphic element. It therefore has the same signal level as HDR Reference White, and graphics should be inserted based on this level."

markw: there are a number of specs here and there that indicate what the peak value is

<chris> For PQ this is defined as 58% and for HLG 75%. If you display those signals on a PQ monitor or a 1000cd/m^2 HLG monitor it comes out at 203 cd/m^2. On different brightness HLG monitors, the HLG adaptive system gamma implemented in the screen will maintain the perceptual contrast without any requirement of metadata.

markw: in windows you have 2 settings to control that
... the peak white of sRGB is whatever the user has selected what the paper white is
... the relation between SDR and HDR is not fixed and depends on the user settings

<markw> if we have e.g. a PQ coded pixel what do we map that to ? There is no canonical way to match together SDR and HDR

markw: is there some reason why some of these have to be exposed to the web user?

chris: some people may prefer things brighter

jeanyves: can you have a generic SDR to HDR conversion?

chris: no you can have 2 for HLG and PQ

jeanyves: is there a point in defining a default that will never work?

chris: I'm optimistic that it can work
... sometimes the UI of some service on some TVs is really blinding

markw: it depends on how the TV does the SDR/HDR compositing
... we don't have the control

chris: the best we can do is define a mapping, that is the default mapping
... and then say that the user may adjust it for preferences, and the system also may adjust

markw: how can we specify that a color is in an HDR color

chris: CSS Color 4 has support

markw: for the gamut not the luminance
... how do I tell CSS that I want an HDR color not an SDR color
... we could use a boolean to indicate if a color is an HDR one
... and leave the relationship unspecified

<chris> so it would be rec2020(r, g, b, bool)

markw: metadata is a different thing

chris: It could say SDR, PQ or HLG

markw: if it is a coded value yes

<chris> chrisbai: so you are looking at this from media pov, not display pov

markw: one of the ways we could define our objective is to say that I have an HDR video frame rendered in a video element and rendered side by side using CSS and they look the same
... not sure it's feasible
... you could also use capability detection to know that the browser does not support that
... but having the same rendering for the video pipeline and the graphics pipeline

cyril: in TTML, there is a way to specify a gain to apply to the subtitle compared to 80nits

chris: we moved from color space to HDR, and it's good, it should not work on SDR only
... but the question is control at element level or page level

simon: it's hard from an implementation perspective to have per-element control

chris: Google?

chris_c: I'll defer to the experts in the room

markw: for each video that is on the screen, we need the metadata to apply the correct tone mapping to each video
... so if we want the graphics to match the video, we need to apply tone mapping to each graphics area to the match the video
... I don't know if this needs multiple working color spaces

chris: in PiP on TVs are they in separate planes?

chrisbai: not every vendor can do that
... some do separately, different color gamuts, tone mapping

markw: the metadata drive tone mapping is an optimization

chris: if much of the web content to be composited is going to be SDR, we should signal

markw: there is no tone mapping issue for SDR, you don't care
... the peak is 80nits or 100nits which is less than the display capability

chris: Adobe RGB is 160nits
... and proPhoto is even more

markw: I scoped my comment to video

chris: but we have photographers who want to display their photos

cyril: where would the control at page or element level be specified?

<chcunningham> Re: "but the question is control at element level or page level" - I'll source an opinion from compositing / color folks in Chromium

chris: it would be in the CSS Color 4 spec
... the problem is that you would have to opt-in for a slower path for the whole page?

markw: how does the current renderer deal with luminance differences between sRGB spaces?

chris: it is ignored
... some browsers will convert image data to an intermediate space and then to the display, using ICC profiles
... at this point, we will capture that as an issue with drawbacks
... I don't get a sense that there is a solution
... but it seems that people care about designing for HDR from the beginning

[observer]: I created an experiment using OpenCV.js, and Canvas is 8 bit so I have to tone map

scribe: are we proposing to increase canvas bit depth?

chris: there is discussion going on

simon: I think it's generally accepted

chris: I think they used half floats

riju: what is the percentage of displays capable of doing HDR?

chris: not known, but we can assume it is going to increase
... people consume even ordinary content in bright sunlight
... displays are having that capability
... PQ assumes a dark room, HLG too to some extent

riju: we are shipping some ambient light system for that

markw: ambient light system is going to happen
... there is no need for APIs to do that

HDR Color in CSS

simon: there is a desire for people to specify color in HDR
... this is going to have an impact on the user experience
... like power usage
... we may want to limit the use of these colors

markw: the outer page could specify limits on the inner page

sushraja: feature policy may be a good way to restrict

chris: good point
... anything more on this topic?

phil: bgsRGB is another flavor of sRGB
... it's part of the IEC specification that includes sRGB

markw: are we in agreement in what is needed and what is the process?
... the question of tone mapping and metadata, is it part of the problem?
... what is the next steps to gather the requirements and develop them further

chris: in this case, we need a requirements document
... because we don't collectively understand the same issues

phil: there are different views in the ICC org
... ICC is not going to be the answer for dynamic content
... we need to be realistic in where ICC can contribute
... we have practical implementations in 2 areas:
... Adobe have main profile for HLG scene and HLG display and PQ display
... it is LUT-based
... those are used in Adobe products
... in ICC version 4, we have a single white point
... that's worked very well, until HDR
... diffuse white/graphics white and display

chris: you have 3 whites: diffuse white,
... fullscreen peak white, small highlight peak white

phil: you map peak white to peak white using a variety of transforms

phil: the Adobe implementation is based on mapping the diffuse whites
... and that's been quite successful
... we haven't decided which way we want to go and we'd like to get feedback from this group

markw: it'd be good to have a document to send to our experts

<markw> scribe: markw

<chris> scribenick: markw

phil: ICC v4 based on mapping white point to white point - media white to media white
... however, with HDR need to decide on white point - media peak white can't be mapped to sdr peak
... as that would compress. One solution is to map graphics white to sdr peak white.
... PCS white point is based on illuminant D50 - 100cd/m2

chris: ICC working on better write up of the problem statement - how to deal with HDR in ICC profiles

pal: not convinced that the solution based on ICC is necessary for HDR color on the web

chris: what info do we need ?

pal: enumerated color spaces - not that many
... for entertainment media ITU-T etc. have enumerated the color spaces that are used

phil: ICC is just a way of packaging those things

pal: but it is very general. We should start with the ones people use

phil: ICCmax has a reference encoding profile feature - instead of specifying a transform you point to it - e.g. with a URL

chris: ICCmax also has calculator and floating point

pal: what's the problem starting with something simple

<cyril> -> CSS Color Level 4: https://www.w3.org/TR/css-color-4/

chris: CSS Color 4 has enumerated set of color spaces. There is a proposal to have an Annex which maps those enumerated spaces to full ICC profile
... also have @profile way to fetch a full ICC profile. At least Google don't want to fetch arbitrary profiles
... publishing people want full profile support

<cyril> CSS color space enumeration: https://www.w3.org/TR/css-color-4/#predefined

chris: print formatting a rather different space

pal: so why not expand that enumerated list with HDR?

chris: which ones ?

pal: basically there are two: HLG and PQ
... typically used with 2020

markw: not just typically, universally

chris: so we could have those two - mark had suggested a boolean

markw: that was for color specified in linear space: need to say which linear space. Agree with enumeration for PQ and HLG when the data is coded with the TF

cyril: TTML has a way to specify the gain applied for PQ and there is a default way to do that for HLG
... so why not do the same here ?

pal: you need to specify whether the pixels are coded in HLG or PQ. Is there any downside to that path ?
... suppose I am creating a professional authoring application for images - just can't do that in canvas today

chris: why not

pal: no way to specify HDR text / colors in CSS

markw: new enumeration values for 2020pg and 2020hlg would be great, as long as it is understood that what you get after converting to linear is in a different scale than sRGB

chris: great, but will anyone implement it ?

jer: for Apple have to check with Dean Jackson and Simon Fraser

chriscunningham: problem has been lack on consensus to date

jean-yves: interest in HDR so far has been for video

HDR image format

chris: w3c would like there to be an image format that everyone can use
... don't want fragmentation, like we have today
... SVG requires PNG and JPEG
... W3C does not want to develop a new image format from scratch
... if there is something that exists that is open and RF we would prefer that

pal: not just format, but full image codec ? target

cyril: at least one candidate: AVIF

chris: HEIF ?

cyril: HEIF is the file structure, but you can put any codec in it: HEVC, AV-1 etc. (AVIF is the name for AV-1 in the same codec)

pal: HT J2K
... It's J2K with lower computational complexity

markw: many people unhappy about the licensing uncertainty associated with HEVC

pal: HT J2K format is called JPH

markw: HEIF is a container, when used with AV-1 it's called AVIF

cyril: HEIF is a container specification, MIAF is a restriction that provides context for different codecs
... AVIF further restricts MIAF for AV-1 codec

<cyril> -> ISO/IEC 23008-12: https://www.iso.org/standard/66067.html

jer: HEIF on Apple platforms is usually HEVC. Also supports live photos (short video)

<cyril> -> ISO/IEC 23000-22: https://www.iso.org/standard/74417.html

chris: are the libraries that can be picke dup and used, that are widely deployed ?

<cyril> -> AV1 Image Format: https://aomediacodec.github.io/av1-avif/

markw: AV-1 has open source RF library - not yet widely deployed because it is new

pal: same for HT J2K

cyril: is it lossless or lossy ? compression ratio
... ?

pal: either lossless or lossy, range of compression ratios as you would expect
... just a lot more computationally efficient as it changes the arithmetic coder

chris: how should we proceed ? got some candidates. Should we put together a doc with pros & cons etc. with an object of defining one winner ?
... could have more than one but one would be ideal
... I assume there are test images ?

cyril: colorist tool (made by a Netflix engineer) can generate synthetic images and do ICC conversions

<pal> pal: HTJ2K is defined in ISO/IEC 15444-15 | ITU T.814 ("Information technology - JPEG 2000 image coding system: High-throughput JPEG 2000") https://www.itu.int/rec/T-REC-T.814-201906-I

<cyril> -> Colorist: https://github.com/joedrago/colorist

markw: there are evaluations of AVIF on JPEG test set from their last CfC

chris: sounds like a document on this would a good idea. I'll start something.

chrisn: I had started a document already that was for internal use but could be a basis for a CG report

chris: can we put that in the W3C giuthub ?

chrisn: sure - I already converted it to Respec

<cyril> -> HEIF, freely available spec : https://standards.iso.org/ittf/PubliclyAvailableStandards/c066067_ISO_IEC_23008-12_2017.zip

pal: who was asking for image format ?

chris: came up last year
... content creators would probably like some guidance
... and if there is consensus that browsers more likely to support

<chcunningham> https://bugs.chromium.org/p/chromium/issues/detail?id=960620#c1

chris: do browsers already support these things ?

<chcunningham> ^^ chrome

jer: we disabled a few because we didn't want to expose them to the web due to security

chris(c): we have a bug for AVIF support in Chrome

jean-yves: pushing WebP

chris: anything else on image formats ? sounds like formats exist and we need to choose

markw: next steps ?

chris: I will take on chris n's document and get it or something based on it the GitHub and then we can work in the GitHub repo

cyril: ... breakout session tomorrow at 5.30 on images on the web, not specifically HDR. Question of MIME types, use of video vs image for sequences
... AVIF has an Annex defining two media types, one for still images and one for sequences, but Mozilla ask why have two if everyone supports both

<cyril> -> Proposed break out on images tomorrow: https://www.w3.org/wiki/TPAC/2019/SessionIdeas#Images_on_the_Web

chris: similar discussions on APNG, but it still doesn't have a media type ... ideological video vs image debate has derailed things in the past

markw: shall we have a call in a couple of months ?

chfris: suggest early december or january
... consensus for december ...

chris: I'll do a doodle

chrisn: where's the mailing list /

<chris> public-colorweb@w3.org

https://lists.w3.org/Archives/Public/public-colorweb/