[css-color-hdr-1] Compositing SDR in HDR outputs

[simontWork]

Care must be taken to ensure any transform defined is robust, works for all inputs and does not greatly adjust the perceived image. For example, a transform cannot simply scale SDR images in linear colour space, as the increased brightness and, if composited with HDR video, the presence of specular highlights will affect the perception of shadows and midtones - a perceptual gamma adjustment is required.

Furthermore, there are 2 transform routes available:

1. "Display Referred" - mathematically apply the BT.1886 EOTF to give linear output screen light, scale brightness and map into the target gamut, perceptual correction, mathematically apply the Inverse HDR EOTF to get the HDR signal
2. "Scene Referred" - mathematically apply the Inverse BT.709 OETF to give linear input camera sensor light, scale brightness and map into the target gamut, perceptual correction, mathematically apply the HDR OETF to get the HDR signal

For this application, I think variant 1 (which visually matches the displayed signal - as closely as possible) is the correct one to use. By carefully choosing the parameters of the SDR and HDR EOTFs, we can simplify the transform and achieve a very close match.

Further information is given in ITU-R BT.2390-8: https://www.itu.int/dms_pub/itu-r/opb/rep/R-REP-BT.2390-8-2020-PDF-E.pdf

[nigelmegitt]

Cross reference #554.

[svgeesus]

For example, a transform cannot simply scale SDR images in linear colour space, as the increased brightness and, if composited with HDR video, the presence of specular highlights will affect the perception of shadows and midtones - a perceptual gamma adjustment is required.

This is clearly true, but I wonder which part of the specification text triggered the worry that SDR content would be linearly scaled to put diffuse media white at HDR peak white?

Currently the spec says, firstly, that compositing happens in CIE XYZ and is thus in a linear-light space. Then:

Relative HDR, using the HLG transfer function, must map SDR media white to the same luminance as is used to display the 75% HLG value.

and

Absolute HDR, using the PQ transfer function, should map SDR media white to 203 cd/m², the same luminance as is used to display the 58% PQ value. However, implementations may chose to incorporate a color re-rendering step (OOTF) to account for non-reference viewing conditions.

Both of these values are from BT.2408 and neither maps diffuse white to peak white.

Transfer function happens after that compositing, and depends on the destination colorspace.

I also agree that display-referred is a better approach, because scene-referred and artistic intent somewhat interact so there is no one true scene lighting.

[simontWork]

If you're mapping an SDR image to cover the whole screen, then yes I agree.

The issue appears when the SDR image is mapped to be part of an HDR image (e.g. as picture-in-picture on a video stream) or alongside an HDR image. The presence of specular highlights will affect the shadows and mid-tones perception of the scaled SDR image - the peak brightness will look correct.

[svgeesus]

The issue appears when the SDR image is mapped to be part of an HDR image (e.g. as picture-in-picture on a video stream) or alongside an HDR image.

I expect that to be the majority use case.

Does that mean the HDR highlights become the primary white adaptive stimulus, or that the viewing condition changes? It seems we are getting into color appearance modelling (simultaneously viewed colors).

The presence of specular highlights will affect the shadows and mid-tones perception of the scaled SDR image - the peak brightness will look correct.

So, that requires re-rendering of the SDR image? Also, we are talking about scaling a linear-light image not a gamma-encoded image, right?

[svgeesus]

From ITU Rec BT.2100

[Myndex]

I have not read the following book as yet, but the article abstract sounds interesting:

https://www.sciencedirect.com/science/article/pii/B9780128094778000029

In essence it "seems" the be taking the lower 5% and upper 5% of the SDR image (by luminance) processing it and expanding the low and the high. Considering how typical color correction soft clips the low and high, this makes some sense as an expansion — part of the processing involves some noise cancelling, that's all I could gleen from the abstract.

[simontWork]

So, that requires re-rendering of the SDR image? Also, we are talking about scaling a linear-light image not a gamma-encoded image, right?

The effect of a specular highlight on the midtones depends on its size and distance from the midtone of interest. So it's not something that can be easily accounted for in static transform. It's best seen by looking at an HDR and SDR copy of the same image. The overall eye adaptation is related more to the average light level and the surround than the specular highlights.

As for expanding the highlights, many SDR video cameras have a "knee" which compresses the highlights to give some details as opposed to clipping. Some transforms do some expansion - there's quite a few methods in literature, patents etc.

[Myndex]

This paper from Xu Zhau and Luo is interesting, they discuss a gamut extension algorithm to take sRGB into a WDG gamut. It occured to me while reading it that it applies to this problem:

https://www.osapublishing.org/DirectPDFAccess/EDFE583F-526E-4332-ABC3DA557FC14600_392700/oe-26-13-17335.pdf?da=1&id=392700&seq=0&mobile=no