[css-color] [filter-effects] Should no-op filters produce different output from no filter?

[smfr]

The filter property states that "Functions must operate in the sRGB color space". This implies that all the input colors to filters, like display-p3 colors and images with P3 color profiles get flattened to sRGB before filter processing.

This has the unfortunate side effect that the results of filter: none and filter: saturate(1) are different when the filtered content contains P3 colors, and that authors can't apply nice filter effects to P3 images while retaining out-of-sRGB colors.

Chrome appears to not follow this spec text precisely, and seems (on macOS at least) to apply filters in some extended sRGB colorspace (tested on iMac Pro with DisplayP3 display).

Testcase: https://codepen.io/smfr/pen/PoOVmPX

cc @svgeesus

[svgeesus]

Certainly in 1998-2000 the SVG spec was created as sRGB-only. The color-interpolation-filters property has the values auto | sRGB | linearRGB where linearRGB is the initial value, and corresponds to the CSS Color 4 srgb-linear colorspace which (like all the predefined RGB spaces there) is unbounded.

There is an open issue SVG color-interpolation-filters to use more color spaces which could be solved by adding additional values to color-interpolation-filters. As you say, another option is to allow extended sRGB as a way to access WCG colors, which would then not require new values on that property but would require careful review of wording about clipping out-of-range values. Such as this one:

The RGBA result from each filter primitive will be clamped into the allowable ranges for colors and opacity values. Thus, for example, the result from a given filter primitive will have any negative color values or opacity values adjusted up to color/opacity of zero.
https://drafts.fxtf.org/filter-effects/#FilterPrimitivesOverviewIntro

Note that processing in unbounded srgb-linear should give the same result as processing in CIE XYZ, both are linear-light.

Notice too that feColorMatrix has the sRGB-linear to XYZ conversion matrices baked in for saturate, hue rotate and luminance. So using unbounded srgb-linear as the filter working space does seem a promising approach.

The only worry is whether existing content is depending on the clamping behavior; if so, some form of opt-in might be needed. Perhaps a linearRGBextended value, or some such?

[svgeesus]

Related: Impact of colorspace on filters

[svgeesus]

So: given that

• SVG filters already operate in linear-light sRGB, and the sRGB to XYZ conversion matrix is baked into the spec with the saturation and luminance operators
  • but they assume 8-bit and hard clamp at 0 and 255 on each stage in the filter chain so can't do WCG
• extended, linear-light sRGB is already defined in CSS color 4 as srgb-linear]() and can represent wide color gamut colors, such as P3 colors
  • plus it is already proposed for use in Canvas for WCG
  • plus there is existing GPU hardware support for extended, linear-light sRGB
• color-interpolation filters exists in Filter Effects 1 with the values auto | sRGB | linearRGB and the initial value is linearRGB

Then I propose to extend color-interpolation filters to take an additional value with a definition link to CSS color 4, and that value is explicitly an extended, WCG space and has a 16-bit precision requirement.

Extended means that negative values, and values greater than 1.0, are allowed. For example color(display-p3 0 1 0), which is outside the sRGB gamut, is color(srgb-linear -0.225 1.0421 -0.079)

This will add WCG support to SVG filters, is opt-in so existing content continues to render the same, and can also be opted into to get the improved precision (8 bits linear light has visible banding) even for sRGB-only content.

The new value could be sRGB-linear for compatibility with Canvas and with CSS Color 4, although above I suggested linearRGBextended which I could also live with.

I'm guessing this would go in Filter Effects 2 ?

[smfr]

A new value for color-interpolation-filters seems OK, but because it would be opt-in for web developers, doesn't address my initial concerns. I'd prefer that no-op CSS filters (like blur(0), saturate(1) etc) act like no filter is applied, which means they can't use sRGB interpolation by default. I'd prefer this for two reasons:

1. Web developers working on sRGB displays don't unintentionally clamp filtered Display-P3 assets to sRGB inside no-op filters
2. User agents can optimize away no-op filters from their rendering pipeline

Doing this would require a spec change along the lines of "CSS filters operate in the current compositing colorspace" (with some handwaving because we specify that the working colorspace is sRGB but that's not what browsers do now).

[svgeesus]

OK so this would be additional wording on no-op filter in particular (and means that saturate(1) and saturate(0.99) work differently) but is certainly doable.

Doing this would require a spec change along the lines of "CSS filters operate in the current compositing colorspace" (with some handwaving because we specify that the working colorspace is sRGB but that's not what browsers do now).

What do they do now? Do they all do the same thing?

[smfr]

What do they do now? Do they all do the same thing?

WebKit does compositing in the colorspace of the display. I don't know what Firefox and Chrome do.

[css-meeting-bot]

The CSS Working Group just discussed [css-color] [filter-effects] Should no-op filters produce different output from no filter?.

The full IRC log of that discussion

<fantasai> topic: [css-color] [filter-effects] Should no-op filters produce different output from no filter?
<fantasai> github: [css-color] [filter-effects] Should no-op filters produce different output from no filter?
<fantasai> github: https://github.com//issues/7100
<fantasai> smfr: Issue here is that CSS filters are specified to use sRGB as their color interpolation model
<chris> they use linear-light sRGB
<fantasai> smfr: This is a problem when authors apply filters to colors outside of sRGB gamut
<fantasai> smfr: e.g. display-p3 or color specified with LCH color space
<chris> q+
<fantasai> smfr: This is important because often authors will apply no-op filters, with intent to animate later
<fantasai> smfr: This means such filters will flatten the colors
<fantasai> smfr: In testing, looked like Chrome isn't doing this: they allow out-of-sRGB colors through filters
<fantasai> smfr: WebKit does limit colors to sRGB
<fantasai> smfr: Don't think this is desriable
<fantasai> smfr: One, I think authors expect no-op filters to not have an effect
<fantasai> smfr: And second as an implementer, I'd like to optimize away such filters by not doing anything special
<fantasai> smfr: Since CSS specifies sRGB compositing and filters are specified to do this also
<fantasai> smfr: CSS uses non-linear sRGB
<fantasai> smfr: so I'm not sure how to resolve this, because I don't think we specify our color compositing model in a way that matches browsers
<Rossen_> q
<fantasai> smfr: maybe Chris has an opinion
<fantasai> chris: Correct that filters use sRGB and clamp at each stage in the pipeline
<Rossen_> ack chris
<fantasai> chris: They do use linear-light sRGB
<fantasai> chris: They don't use gamut-?? values
<fantasai> chris: Very obvious when you're doing it wrong
<smfr> s/gamut-??/gamut-encoded/
<fantasai> chris: I suggested a way forward for this, beefing up filters to have a wider color gamut
<fantasai> chris: Simon said very nice, but doesn't solve my issue which is no-op filter
<fantasai> chris: My concern about is that as soon as someone starts to animate, from 0 to 0.01, then there's a sudden change
<fantasai> chris: I can see that it would help with no-op filters, but not with an animation case
<fantasai> smfr: That's very true, and that's why I think we need to specify that CSS filters interpolate in the working color space, so that they don't truncate colors?
<fantasai> chris: The concept of a working color space isn't well-defined
<fantasai> chris: we have different ones for different operations
<fantasai> chris: Talkinga bout compositing, but not just compositing, it's also the convesion from luminance or conversion from saturation, all use constants from sRGB and not display-p3
<fantasai> chris: need to fix that, and the way to fix that is to continue using sRGB ones, and because in linear-light make them extended, and in that way can represent all other colors as well
<fantasai> chris: I think that's the best minimal change, otherwise you have [problems]
<fantasai> smfr: That would be fine...
<Rossen_> q?
<fantasai> chris: If we can bash this out, would be great
<fantasai> smfr: Would be interested in hearing from Chromium and Gecko, what their filter color space is
<fantasai> chrishtr: I'll have to check
<fantasai> emilio: same here
<fantasai> emilio: but pretty sure we have code to optimize away no-op filters
<fantasai> Rossen_: so, do more investigation on Chrome and Gecko side and then come back to this?
<fantasai> chris, smfr: sounds fine
<chrishtr> will investigate

[ccameron-chromium]

What do they do now? Do they all do the same thing?

WebKit does compositing in the colorspace of the display. I don't know what Firefox and Chrome do.

Chrome does compositing in the colorspace of the display. We also do rasterization in the colorspace of the display.

There are some exceptions. If the display has a color space that is too far from sRGB-like interoplationbehavior (e.g, has a linear or PQ transfer function), then we do composite and raster in extended-sRGB, and do a final blit converting to linear or PQ.

We haven't carefully quantified exactly what "too far from sRGB-like interpolation behavior" means (you can examine the current logic here).

[LeaVerou]

Then I propose to extend color-interpolation filters to take an additional value with a definition link to CSS color 4, and that value is explicitly an extended, WCG space and has a 16-bit precision requirement.

P3 can get away with 8 bit, so I'm worried that requiring 16 bit will discourage implementations.

[svgeesus]

P3 can get away with 8 bit, so I'm worried that requiring 16 bit will discourage implementations.

No. Gamma-encoded P3 can get away with 8 bits per component, agreed. Linear-light needs considerably more bits as the space is not perceptually even, so avoiding banding requires more bits.

To see this, iterate from 0 to 255 and at each stage

• convert to float 0..1
• convert to linear-light
• multiply by 2^n, round (to simulate using n bits)

Then count how many unique values there are. I wrote code to do this and 12 bits precision was the minimum to retain 256 unique values. (That code is for srgb-linear, but display-p3 and srgb use the same transfer function).

[svgeesus]

In addition, covering the full gamut of display-p3 in srgb-linear requires more than 8 bits of gamma-encoded values because the range is a bit greater than 0.0 - 1.0. For example, color(display-p3 1 0 0) is color(srgb-linear 1.22494 -0.0421 -0.0196)

[css-meeting-bot]

The CSS Working Group just discussed [css-color] [filter-effects] Should no-op filters produce different output from no filter?.

The full IRC log of that discussion

<dael> Topic: [css-color] [filter-effects] Should no-op filters produce different output from no filter?
<dael> github: https://github.com//issues/7100
<dael> smfr: This is the problem where filter like blur-0 flattens p3 colors to srgb per spec but impl, chrome always and some webkit will propegate p3 colors through filters
<Rossen_> q?
<dael> smfr: chris suggested new value for color interpolation, but doesn't solve problem i'd like. Spec doesn't match impl and I'd like them not to flatten P3 colors
<dael> chris: suggestion seemed to be copositing is in device color space. All filter operations or just compositing?
<dael> smfr: I believe impl are running filter operations in display color space which is P3 and close to srgb
<dael> chris: Looks okay but is wrong. What I suggested would fix that but doesn't fix existing issue
<dael> chris: Slightly concerned about making it do what you want b/c makes it hard to test if it's behaving correct. not completely opposed but want ot get it right
<dael> smfr: svg filters are spec in srgb color space and all constants work in srgb. One approach would be how to spec those in lab color space so impl can filter in that
<dael> chris: lab not good choice. svg are linear srgb not gamma encoded. I suggested an extended linear srgb. outside P3 clor would be fine. if want to upgrade whole thing to another color space then xyz would be a better choice then lab.
<dael> chris: doing in extended linear srgb would give you same result if not clipping
<dael> smfr: Might be possible
<dael> chris: You and I are attaching different bits of problema nd maybe can do both
<dael> smfr: You mean add new color interopolation and change default?
<dael> chris: yeah
<dael> Rossen_: Possible in current impl or compat issue?
<dael> smfr: Won't match what impl do. Would be signifinct work in one webkit codepath
<dael> Rossen_: Academically good, but would be avoided by impl
<dael> smfr: Our slow path, CPU based which is 8-bit unsigned. We'd have to change that to floating point or non-8bit
<dael> smfr: Our codepath with GPU is assuming srgb so applying that to values in P3 which happens to work most of the time. Probably what Chrome does too
<dael> Rossen_: chrishtr do you know if that's what youre doing?
<dael> smfr: For GPU accelerated you're feeding through P3 color values when using srgb matrix?
<dael> chrishtr: not sure, need to talk to Chris Cameron (?)
<dael> smfr: Would be good to see if willing to change to chris suggestion
<dael> chrishtr: Is that in the issue?
<dael> smfr: Yes, thing so
<dael> chrishtr: I'll re-read and ask Chris
<dael> chris: Please also get back to me in issue if something not clear. linear srgb extended should be GPU friendly
<dael> smfr: This should come up with canvas filters too so someone should think about that case too
<dael> Rossen_: Sounds to me like we need to have more eyes on the problem
<dael> chris: Agree
<dael> Rossen_: Want to capture proposed combined solution
<chrishtr> +1 to issue
<dael> chris: I'd rather write it on the issue instead of try and word on the fly
<dael> Rossen_: Alright, please add to issue
<dael> Rossen_: Anything else on this?