[css-color-4] Disagreements over gamut mapping

[sesse]

There's been some discussions around gamut mapping: https://csswg.sesse.net/css-color-4/#css-gamut-mapping

The questions in point are:

• Is §13.2 normative? There have been discussions about the use of the phrasing “This CSS gamut mapping algorithm” versus “The CSS gamut mapping algorithm”, and the non-normative marker on §13.1 (does it extend to all of §13?). Must out-of-gamut colors be mapped, using that specific algorithm, on both conversion-to-HSL and for display, to be compliant with CSS Color Level 4?
• What, if anything, should be done with out-of-gamut colors in images and videos? Do we risk that using the same color in e.g. CSS and on an image mapped to different colors on the users' screens, and is that okay?
• What about HDR images and video; are those subject to the same rules, different rules, or undefined?

[svgeesus]

Is §13.2 normative? There have been discussions about the use of the phrasing “This CSS gamut mapping algorithm” versus “The CSS gamut mapping algorithm”, and the non-normative marker on §13.1 (does it extend to all of §13?).

13.1 is, as stated, a non-normative introduction to the topic. 13.2 is normative.

This CSS gamut mapping algorithm

Thanks for pointing out the possible ambiguity, I agree that The would be better.

Must out-of-gamut colors be mapped, using that specific algorithm, on both conversion-to-HSL and for display, to be compliant with CSS Color Level 4?

Yes. HSL (and HWB) are unable to represent out-or-sRGB-gamut colors.

What, if anything, should be done with out-of-gamut colors in images and videos? Do we risk that using the same color in e.g. CSS and on an image mapped to different colors on the users' screens, and is that okay?

"If anything"? Colors which are out of the display device gamut cannot be displayed (by definition).

Gamut mapping for images (and video) has different constraints to solid colors, such as preservation of image detail. For photographic images, a perceptual rendering intent should preserve most of the overall image appearance but may change in-gamut colors. For non-photographic images such as charts and diagrams, a relative colorimetric intent will give more similar results to the CSS GMA (but not identical, because the ICC code path will be using CIE Lab as the gamut mapping space). Yes, that means that out of screen gamut colors may map differently in images and in CSS, particularly wildly OOG colors such as those outside the spectral locus. These are much less likely in photographic images but can occur in synthetic images.
It isn't desirable but there isn't a better option because relative colorimetric handling of photographic imagery gives terrible results.

What about HDR images and video; are those subject to the same rules, different rules, or undefined?

The same "this is a different code path" rules i.e. not the same as CSS. In addition to the GMA there will also be tone mapping, unless the display has the same peak white as the mastering display. All of that is generally a black box, especially for HDR video.

[sesse]

Yes. HSL (and HWB) are unable to represent out-or-sRGB-gamut colors.

Sure, but for display, does the spec require this specific algorithm and not e.g. just clipping each color component to [0,1]?

[ccameron-chromium]

Suppose a page specifies the color "color(display-p3 1 0 0)", and then has an image element with an image in Display P3 color space, with pixel value (1, 0, 0). Should those two colors be identical when displayed on-screen?

[ccameron-chromium]

Similarly to the above, what if both CSS and a 2D canvas use the color "color(display-p3 1 0 0)"? What about a WebGL canvas that writes a pixel value (1, 0, 0) and has specified its color space to be "display-p3".

I believe that it is a goal to allow color matching between CSS, images, and canvases (ideally video, too, but there is longstanding historical divergence in interpretations of rec601 and rec709 -- something for another discussion -- I'd love to resolve that, maybe WebCodecs will let us).

In order to guarantee color matching between CSS and images, it is necessary that the same processing be applied to pixels coming from CSS and pixels coming from images. This precludes a scheme of "apply gamut mapping to just CSS colors and not (necessarily) images".

This can be fixed by gamut mapping images and canvases. That, however, comes at a significant performance cost. For CSS colors, there are lots of places in the pipeline to allow injection of various transforms at very little (or no) extra cost. For canvas elements, they are often handed directly to the display controller, which has only fixed-function hardware for color management, and is incapable of performing gamut mapping. The hardware is evolving, and there is a big push to support per-plane 3D LUTs (which would allow hardware tone mapping and hardware gamut mapping), but I'm not sure they exist in shipping devices (let alone are universal).

If we were to require gamut mapping for canvases, it would mean that no canvas that can express any color value outside of the gamut of the display can be represented as a hardware overlay. That would be catastrophic for battery life, especially on mobile devices. (There's almost no limit to the work that we will do to keep a buffer in a hardware overlay plane).

This line of reasoning leads me to the position that gamut mapping should be put as the responsibility of the underlying operating system, display controller, and even display device(!). For the next few years (until display controllers all have 3D LUTs, completing yet another turn of the wheel of reincarnation), this will mean that out-of-gamut colors may not look as good as they could on some devices. Content authors for whom this is a matter of grave concern may use the media queries (or the exact query of primaries, which is part of the HDR canvas proposal), to ensure they do not serve content outside of the gamut of the target device.

[sesse]

Cc-ing @weinig, since he seems to have done the work on gamut mapping in WebKit.

FWIW, I don't think “someone makes a DCI-P3 canvas or WebGL context on an sRGB device” is a use case that should be driving for our general color decisions. It seems very niche compared to the amount of pages that simply use CSS colors and regular images (or sRGB canvases). And AIUI, “colors won't look good on some devices” essentially means “on every non-Apple device in existence” for at least the next couple of years…

[foolip]

@weinig might you be able to summarize the "architecture" of color handling in Safari?

From testing in Safari Technology Preview 151 it's clear that something has changed from the previous behavior of clamping each component separately. One of the test cases that @sesse used was color(srgb 2 0 0) which in Safari stable clamps to color(srgb 1 0 0) in computed style, but in STP remains as color(srgb 2 0 0) and renders as something close to color(display-p3 0.9992 0.4399 0.3215) according to Web Inspector's color picker.

It would be great to understand a bit about where in the pipeline such mapping happens, and whether it only applies to out-of-gamut cases like color(srgb 2 0 0), or if something similar would also happen when color(display-p3 1 0 0) is to be displayed on an RGB monitor?

[ccameron-chromium]

Bear in mind that the color (srgb 2 0 0) is not just out-of-gamut, but also has a luminance that is out of the SDR luminance range. That's likely the reason why it is being bent towards white.

[foolip]

@ccameron-chromium good point, it's also worth testing somewhere where luminance is in SDR range. I think color(srgb 1.05 0 0) is such an example, which STP renders as color(display-p3 0.9639 0.2133 0.1489). I'm not sure what to conclude from this, except that it's not merely clamping to RGB red, as Safari stable did.

[facelessuser]

good point, it's also worth testing somewhere where luminance is in SDR range. I think color(srgb 1.05 0 0) is such an example, which STP renders as color(display-p3 0.9639 0.2133 0.1489). I'm not sure what to conclude from this, except that it's not merely clamping to RGB red, as Safari stable did.

It's not clamping to sRGB Red, it is clamping in the Display P3 color space. It isn't doing any fancy gamut mapping, just simple clipping in Display P3, as most new Macs have Display P3 monitors. Safari, due to its "color management" then works in Display P3. I guess this is new behavior between stable and STP. That's all that happening.

>>> Color('color(srgb 1.05 0 0)').convert('display-p3').clip()
color(display-p3 0.96358 0.21239 0.14773 / 1)

As far as I can tell, Safari doesn't do anything but simple clipping currently. On Display P3 systems, it will clip to the Display P3 color space, and on sRGB systems, it will clip to sRGB. We can see this by using color(srgb 2 0 0).

>>> Color('color(srgb 2 0 0)').convert('display-p3').clip()
color(display-p3 1 0.44226 0.32203 / 1)

If it was doing gamut mapping as described in the CSS spec, the results would probably be white as the color would be beyond the luminance of Display P3. The CSS spec currently recommends gamut mapping in Oklch, so if we look at the slice of Oklch in which gamut mapping would occur, limiting the gamut to Display P3, we can see why it would go to white.

>>> Color('color(srgb 2 0 0)').convert('display-p3').fit(method='oklch-chroma')
color(display-p3 1 1 1 / 1)

EDIT: I don't have anything to do with Safari, and this is all just based on simple observation. I don't know if they have more advanced gamut mapping planned or not.

[sesse]

When I tested something like color(srgb 10 0 0) in Safari TP, it would go towards a very pale yellow on the internal Display-P3 screen. This is incompatible with the notion of a simple clipping in DCI-P3 space.

[facelessuser]

That is not what I see at all in Safari TP, I get white on my Display P3 monitor. Unless there is some other special feature you have enabled that I don't.

This makes sense as color(srgb 10 0 0) is so far out in the land of imaginary colors (past the spectral locus) that all components exceed 1 and it just gets clipped to white. Even gamut mapping will leave you with just white. You can try it here.

[sesse]

But that is, indeed, incompatible with the theory that it's just clipping each component in display-p3.

Anyway, let's wait for the people who actually implemented this to chime in?

[svgeesus]

In order to guarantee color matching between CSS and images, it is necessary that the same processing be applied to pixels coming from CSS and pixels coming from images. This precludes a scheme of "apply gamut mapping to just CSS colors and not (necessarily) images".

Feel free to test that out with some photographic images (you will need to override the rendering intent in the ICC profile, which will likely be set to perceptual).

Wanting solid colors and images to gamut map the same is a great goal, and sounds reasonable until one looks into what is currently done for gamut mapping images.

For an in-depth overview, I recommend Color Gamut Mapping by Ján Morovič

[svgeesus]

“colors won't look good on some devices” essentially means “on every non-Apple device in existence” for at least the next couple of years…

Yes but to clarify: there are plenty of non-Apple devices (laptops, tablets, phones) with WCG (P3-ish or Adobe RGB-ish) screens. It is not the hardware that is lacking, but WebKit browsers being confined to Apple hardware, and non-WebKit browsers being confined to sRGB.