[css-animations] Resolving dependencies in keyframes

[andruud]

This came up (again) during @kevers-google's in-progress implementation of getKeyframes in Chrome. Since getKeyframes directly exposes computed keyframes, it forces us to deal with how/when dependencies in the keyframe are resolved in more detail.

We should in my opinion take a "broad" approach of thinking about all dependencies, and not specify detailed behavior per case. For example, 1em has a dependency on font-size, and var(--x) has a dependency on --x. In terms of dependencies, 1em and var(--x) are very similar, so for consistency, these cases should be treated the same way.

Browsers generally don't agree on how interpolation actually happens when complicated dependencies are involved. I won't enumerate all differences, but instead highlight this (well-known) example:

@keyframes test {
  from { font-size: 2px; width: 10em; }
  to { font-size: 4px; width: 20em; }
}

div {
  font-size: 10px;
  animation: test 10s -5s linear paused; /* t=0.5 */
}

The computed value of width on div for the different browsers:

• Firefox: 150px. (1)
• Safari: 50px. (2)
• Chrome: 45px. (3)

@kevers-google, @flackr and myself have had several discussion about this recently, and three different ways of dealing with it surfaced in those discussions:

1. Resolving against the pre-animated style

In this model, the em units would resolve against the would-be computed value of font-size without animation effects, i.e. 10px in the first example. The interpolation for width would take place in absolute values, between 100px and 200px. For var() references, they would behave similarly: the value would be substituted using the pre-animated computed value of the referenced custom property.

I think this solution is a nice one in terms of technical complexity, but I'm not sure if it matches author expectation.

This appears to be what Firefox is doing (for em units).

2. Resolving against local values in the keyframe

In this approach, each keyframe is basically a list of declarations that's added to the cascade, and we derive computed values from that. In approximated spec language:

• Let base cascade be the cascade as it would be without animation/transition affects added.
• For each specified keyframe:
  • Let keyframe cascade be a copy of the base cascade
  • Add the declarations of the keyframe to the keyframe cascade, at the animations level.
  • Produce a computed keyframe by substituting each value in the specified keyframe with the corresponding computed value from the keyframe cascade.
• Interpolation then happens between computed keyframes.

In the first example, em units would then resolve against the font-size specified locally in the keyframe (if present). The interpolation of width would be between 20px and 80px.

This approach would match my initial expectation as an author, I think. But not sure how practical it would be to implement this. Although Safari has the apparent behavior of this model already, at least for the example I gave.

Note that an interesting side-effect of this suggestion is that !important declarations from the base style can end up in the computed keyframes.

3. Resolving against the (animated) computed value

This is what Chrome is currently doing for em units (but not for var()). In the example, width interpolates in em-space from 10em to 20em, at the same time as font-size interpolates from 2px to 4px. The em unit is resolved computed-value time (as apposed to before the effect value is added to the cascade), against the animated font-size at that time. So Chrome produces 45px in the first example via 15em * 3px = 45px.

For var() it would mean that, for this example:

@keyframes test {
  from { --a: 10px; --b: 30px: width: var(--a); }
    to { --a: 20px; --b: 40px: width: var(--b); }
}

div {
  animation: test 1s;
}

The animation would add a value to the cascade that can be illustrated by: calc(var(--a) + t * (var(--b) - var(--a))). In other words, like for em resolution, the var() resolution would be delayed until computed-value time. (Again, Chrome does not currently do this, but it would be the consistent thing to do in this model).

Not sure what I think about this behavior. On the one hand it produces unnecessarily complex animation behavior, but on the other hand it means the concept of "computed values in keyframes" can be mostly avoided, which has a certain simplicity.

Note that getKeyframes in this model should in my opinion not return the keyframes as if "all property values are replaced with their computed values" as the spec currently suggest, as this would misrepresent the endpoints of the interpolation.

---

So at this point I was hoping for some feedback regarding which approach is better, whether or not the author really cares about this at all, how compatible each solution is w.r.t. web-animations, and whether or not there are other approaches that should be considered. Thanks.

cc @birtles @dbaron

PS: I realize this topic has been discussed many times before, and I apologize for not digging up all prior discussion in advance. (Please add links). At the same time, I hope we can avoid treating prior resolutions as automatically holy and immutable, since they apparently failed (so far) to effectuate any real consensus in browser behavior.

[birtles]

PS: I realize this topic has been discussed many times before, and I apologize for not digging up all prior discussion in advance. (Please add links). At the same time, I hope we can avoid treating prior resolutions as automatically holy and immutable, since they apparently failed (so far) to effectuate any real consensus in browser behavior.

This is my fault because I went to make the spec edits but didn't want to make them without adding WPT. When I went to add WPT I realized how poor the existing WPT were and started rewriting them but didn't finish before I left Mozilla.

(Also now that I've lost access to my old Mozilla mail I can't find the different mails where I summarized the discussion so far 😅)

A couple of starter links though:

• Original outline of approaches: https://lists.w3.org/Archives/Public/www-style/2014Aug/0132.html
• Re-visiting and committing to that resolution in Sapporo F2F 2015: https://lists.w3.org/Archives/Public/www-style/2015Nov/0308.html

As for Firefox's behavior, it's definitely a known problem. We have put off fixing it a number of times thinking that we would fix it properly when we implement the Properties and Values API. Unfortunately we've had two incomplete attempts at that.

At this stage, since I don't have experience with Properties and Values and am less involved in implementation these days I suspect @emilio or @hiikezoe or @BorisChiou might be better able to comment.

Note that getKeyframes in this model should in my opinion not return the keyframes as if "all property values are replaced with their computed values" as the spec currently suggest, as this would misrepresent the endpoints of the interpolation.

I'm sorry, I haven't given your proposal proper thought (Friday afternoon here after a tough week) but one comment here is that I suspect we need to stick to returning computed keyframes from getKeyfarmes() for CSS animations.

The reason is that it's simply not possible to represent specified CSS keyframes accurately using Javascript objects. There are all sorts of edge cases (e.g. declaration order matters in CSS, but not in JS; CSS can have duplicate declarations but JS can't) so you end up needing to expand shorthands etc. in order to correctly represent the the CSS markup -- and that in turn means you end up needing to compute values (since you need to know how variables are going to expand).

I really tried hard to preserve specified values in the CSS animation keyframes from getKeyframes() but it proved impossible and when I spoke with @graouts and @flackr they agreed we should just use the computed values.

[andruud]

Thank you @birtles! This means that current state of affairs is that G-β is supposed to be the correct behavior? If I'm not mistaken, this is approximately the same as option 3 here. (Right?)

I really tried hard to preserve specified values in the CSS animation keyframes from getKeyframes() but it proved impossible and when I spoke with @graouts and @flackr they agreed we should just use the computed values.

But "just" using computed values doesn't work either, does it?

Say there's an animation:

@keyframes test {
  from { width: 1em; }
  to { width: 2em; }
}

And at the same time, there's a transition on font-size that moves from 10px to 20px in the same time period. The computed value of font-size is then determined by the value the transition adds to the cascade. Is the idea that getKeyframes should return (an object equivalent to):

@keyframes test {
  from { width: 10px; }
  to { width: 40px; }
}

Because, at the time getKeyframes is called (and at any other time really), there is only one computed value for font-size. The em unit can't resolve to different things (currently, without new concepts).

If the above is what we want, then we're not substituting "the" computed value, but some value that's equal to the would-be computed value at the time indicated by the keyframe.

[birtles]

Thank you @birtles! This means that current state of affairs is that G-β is supposed to be the correct behavior? If I'm not mistaken, this is approximately the same as option 3 here. (Right?)

Right.

But "just" using computed values doesn't work either, does it?

Right, I think we need to distinguish between what the browser internally stores and what getKeyframes() returns.

So it's fine to internally retain the em units but then resolve them to px when generating the result for getKeyframes().

...And at the same time, there's a transition on font-size that moves from 10px to 20px in the same time period. The computed value of font-size is then determined by the value the transition adds to the cascade. Is the idea that getKeyframes should return (an object equivalent to):

@keyframes test {
  from { width: 10px; }
  to { width: 40px; }
}

No, you'd get a different object depending on when you called getKeyframes(). At the start you'd get:

{
  offset: 0,
  width: '10px'
},
{
  offset: 1,
  width: '20px',
}

and at the end you'd get:

{
  offset: 0,
  width: '20px'
},
{
  offset: 1,
  width: '40px',
}

That means that round-tripping the result of getKeyframes() to setKeyframes() loses information but I think it's the best we can do given the mismatch between CSS syntax and the simple JS notation we have. If you want to retain em units (or variable references for that matter), you need to either update the @keyframes rule or set the em units yourself.

Somewhere I wrote a long comment about some of the edge cases that make this expansion necessary but I can't find it now. I think though it's particularly cases like:

  :root {
    --two-values: 10em 20em;
  }
  body {
    font-size: 10px;
  }
  @keyframes a {
    to {
      margin: var(--two-values);
      margin-right: 30em;
      margin-right: 40em;
    }
    to {
      margin-left: 50em;
      margin-inline: 60em;
      margin-inline-start: 70em;
    }
  }

The full-fidelity representation of that using Web animations keyframe objects might be something like:

{
  offset: 1,
  margin: 'var(--two-values)',
  marginInline: '60em',
  marginInlineStart: '70em'
}

but when the writing-mode changes it might be:

{
  offset: 1,
  margin: 'var(--two-values)',
  marginInline: '60em',
  marginInlineStart: '70em',
  marginRight: '40em',
  marginLeft: '50em'
}

(And we can't unconditionally include marginRight and marginLeft since they will clobber the other values.)

Simply trying to keep track of which properties clobber which is difficult in light of overlapping keyframes and shorthands and logical properties so it's easiest if you can expand to physical longhand properties.

But then that becomes complicated because until you expand variables, you don't know where properties will get their values from. Hence why we ended up just expanding everything to physical longhand properties with computed values when generating keyframes from getKeyframes().

[andruud]

Thank you again @birtles for that detailed answer.

No, you'd get a different object depending on when you called getKeyframes()

OK, then it is actually "the" computed value, so it's consistent in that regard. And if we do keep interpolating in specified values but compute the keyframe at getKeyframes()-time, I don't see a technical problem with this.

However, the object we'd return seems like it has nothing to do with the actual animation. The keyframes you get don't actually represent the things we're interpolating between, but instead you get "random" keyframes depending on the current state of other interpolations. This seems not useful, and misleading. It's both losing and adding information.

It would IMO even be preferable to spec that getKeyframes() returns an empty result if the animation is unrepresentable as a web animation vs. returning something which isn't really true.

Ideally we would change how keyframes are represented in JS if we're going for an approach that interpolates in specified value space (e.g. G-β). Otherwise, even if we did manage to stack enough special rules on top of each other to deal with all the difficulties, we'll just keep running into the problem each time CSS/the cascade/interaction-between-properties changes in the future.

Zooming out a bit again, I wonder whether G-β really is very important to the author. It seems to me that reasonable (enough) behavior both for the actual interpolation and getKeyframes() falls out automatically if we pick e.g. (1).

@birtles: How likely is it that we can change the (spec'd/resolved) behavior of:

• How interpolation happens (G-β vs other options)?
• How keyframes are represented in JS?
• What getKeyframes() returns? (Not including the JS representation itself).

[birtles]

However, the object we'd return seems like it has nothing to do with the actual animation. The keyframes you get don't actually represent the things we're interpolating between, but instead you get "random" keyframes depending on the current state of other interpolations. This seems not useful, and misleading. It's both losing and adding information.

I think that might be overstating things a bit. The object returned represents the actual values we are interpolating between at that point. Interpolation happens in computed value space so we have to convert to computed values to interpolate and the object returned accurately represents that.

For some animations, other (typically independent) effects (e.g. from JS or other animations) may cause those interpolation endpoints to change on the next frame, but that doesn't make these values "random" any more than the values from getComputedStyle are "random".

To give some concrete use cases for these values (and these are literally the first two use cases that came to mind):

1. Showing a graph of the animation shape in DevTools. If an animation happened to use em based units and font-size was also being animated, the fact that the result of getKeyframes() changes on each frame would simply mean the graph may need to be re-rendered which seems reasonable. In the case where all values use em units, however, the graph shape would not change.

2. Polyfilling color interpolation using HSV (I really really really hope no "color space" people see this thread! 😅). In this case, we fetch the keyrames using getKeyframes(), synthesize a new set of keyframes, and set them using setKeyframes(). Obviously if one of the original keyframes used currentcolor and that later resolved to a different value, it would no longer be reflected in the animation (since we replaced the keyframes) but that would be true regardless of whether or not returned currentcolor in the first place unless the author specifically added code to watch for changes to the computed value of color.

In both cases, the fact we were able to get the keyframes is of great benefit as otherwise these effect would be impossible without manually fetching and cascading all the CSSKeyframesRule and CSSKeyframeRule objects ourselves.

@birtles: How likely is it that we can change the (spec'd/resolved) behavior of:

• How interpolation happens (G-β vs other options)?

Given that we don't have good interop here, this seems possible depending on the scope of the change.

• How keyframes are represented in JS?

To the extent that these align with what we pass to Element.animate() which has been shipping in Chrome and Firefox for quite a while now, this seems unlikely.

• What getKeyframes() returns? (Not including the JS representation itself).

Given that this has only shipped recently, it might be possible but, depending on the scope, would probably need some use counters / telemetry to be confident we are not breaking content.

[birtles]

I thought about some other alternatives like introducing getComputedKeyframes() but I'm afraid I don't like any of them yet.

[andruud]

The object returned represents the actual values we are interpolating between at that point.

I admit I didn't think about it that way. In isolation that's fair enough.

Interpolation happens in computed value space so we have to convert to computed values to interpolate and the object returned accurately represents that.

Didn't think about it that way either. I was thinking interpolation happens in specified value space, e.g. interpolation from 10em to 20em becomes 15em at t=0.5, and then the em is resolved. Sounds like your mental model is that both 10em and 20em endpoints are dynamically resolved, and then interpolation happens on those values in px. I guess there's no observable difference.

doesn't make these values "random" any more than the values from getComputedStyle are "random"

If you do div { font-size: 10px; width: 10em; }, and then round-trip the getComputedStyle result of width, the width will no longer be responsive to changes in font-size, but you'll at least get the same computed value for width beyond that.

And if you do:

@keyframes test {
  from { font-size: 10px; width: 10em; }
  to { font-size: 20px; width: 20em; }
}

And roundtrip that with getKeyframes()/setKeyframes() at t=0 for example, you get an animation for width that's no longer responsive to changes in font-size, so maybe that's indeed consistent with gCS. But you also get an interpolation for width that's different than the one you would otherwise get. On the one hand this is natural consequence of the interpolation not being responsive to font-size any more, but on the other hand it feels unnecessarily "unstable". If gCS is able to roundtrip a specific value, it seems to fair to at least initially expect that get/setKeyframes should be able to roundtrip a specific interpolation, and not just a specific state of the interpolation.

getComputedKeyframes()

Isn't that basically what this is? 🙂

[andruud]

@birtles In the interest of trying to get this issue to "converge" on something, let's assume that we want to continue with G-β, and that we also want to return "computed keyframes" from getKeyframes() (even though I still think it's weird).

But first I need to clarify the behavior:

Interpolation happens in computed value space

Some implications of G-β + the above are (for example):

(@birtles: please review and confirm the examples).

• em units resolve dynamically against the computed value of font-size:

@keyframes test1 {
  from { font-size: 10px; width: 10em; }
  to { font-size: 20px; width: 20em; }
}
div {
  animation: test1 10s -5s linear paused;
}

At t=0.5, the computed value of font-size is 15px. Hence the computed endpoints of the width interpolation are [150px, 300px], and the computed value of width is therefore 150px + (300px - 150px) * t = 225px.

• var() resolves dynamically against the computed value of the variable:

@keyframes test2 {
  from { --x: 10px; width: calc(10 * var(--x)); }
  to { --x: 20px; width: calc(20 * var(--x)); }
}
div {
  animation: test2 10s -5s linear paused;
}

At t=0.5, the computed value of --x has just flipped to 20px (tokens), hence the computed endpoints for the width interpolation end up being [200px, 400px]. The computed value of width becomes: 200px + (400px - 200px) * t = 300px. Note that at t=0.4999, the computed value of width instead becomes (approximately) 100px + (200px - 100px) * t = 150px, hence the interpolation of width "inherits" the flip-at-50% behavior from --x.

• var() resolves dynamically against the computed value of the variable, registered property version

@property --x {
  syntax: "<length>";
  inherits: true;
  initial-value: 0px;
}
@keyframes test3 {
  from { --x: 10px; width: calc(10 * var(--x)); }
  to { --x: 20px; width: calc(20 * var(--x)); }
}
div {
  animation: test3 10s -5s linear paused;
}

This is the basically the same thing as test1, and the computed value of width becomes 225px.

• Dependencies resolve against the actual computed value of the dependency:

@keyframes test4 {
  from { font-size: 10px; width: 10em; }
  to { font-size: 20px; width: 20em; }
}
div {
  font-size: 30px !important;
  animation: test4 10s -5s linear paused;
}

The interpolated font-size is not relevant for em resolution here, hence the computed value of width is 300px + (600px - 300px) * t = 450px.

What I think we should do now:

• Spec the G-β behavior more clearly (I can try). I think it's worthwhile to mention the em/font-size case, some custom props cases, and possibly a case with transitions, or something else that makes it clear "which" computed value is used for dependency resolution.
• Spec that the UA must behave as if computed keyframes are produced dynamically immediately before interpolation.
• Spec that specified values are retained internally, but getKeyframes() outputs the computed keyframes. (There's a lot of confusion about "when" keyframes are computed at the moment).
• Add WPTs for difficult cases (I can contribute), reviewed by @birtles. Or investigate if they already exist.

Another possible other way of explaining it is that the animation test1 adds values to the cascade at the animations level equivalent to:

font-size: interpolate(10px, 20px, 0.5);
width: interpolate(10em, 20em, 0.5);

Where the params are resolved to absolute values before interpolation occurs.

When it comes to getKeyframes(), is there still any open issue re. "ambiguous clobbering" of properties? Since we absolutize all values in the computed keyframes, there should be no difficult cases with custom properties, since no var() references are present? (You tried to explain this in comment 4, but I can't tell whether or not there's still an issue). I don't immediately see any remaining issues, even if we include custom properties in the result of getKeyframes() (which we should, #5126).

[birtles]

I was thinking interpolation happens in specified value space, e.g. interpolation from 10em to 20em becomes 15em at t=0.5, and then the em is resolved. Sounds like your mental model is that both 10em and 20em endpoints are dynamically resolved, and then interpolation happens on those values in px. I guess there's no observable difference.

At a spec level, we only define interpolation between computed values so we have to convert the endpoints before interpolating.

getComputedKeyframes()

Isn't that basically what this is? 🙂

Yes, I was thinking about having a getKeyframes() / getComputedKeyframes() pair like we have for timing values and then for CSS animations we could have getKeyframes() return an empty list to represent the fact that we can't faithfully represent the specified values while having getComputedKeyframes() produce a computed equivalent so you can still inspect and manipulate the animation.

However, I think that ship has sailed. We already have a computedOffset member in the result from getKeyframes() and we're already shipping getKeyframes() as returning something non-empty in at least Firefox and Safari.

(@birtles: please review and confirm the examples).

• em units resolve dynamically against the computed value of font-size:

@keyframes test1 {
  from { font-size: 10px; width: 10em; }
  to { font-size: 20px; width: 20em; }
}
div {
  animation: test1 10s -5s linear paused;
}

At t=0.5, the computed value of font-size is 15px. Hence the computed endpoints of the width interpolation are [150px, 300px], and the computed value of width is therefore 150px + (300px - 150px) * t = 225px.

Yes, that's right. There is an ordering dependency there. I believe @alancutter worked on this in the context of custom properties in Blink and said it was quite hard.

Those other examples all match my understanding of the expected behavior.

What I think we should do now:

• Spec the G-β behavior more clearly (I can try).

Thank you! 🤩

• Spec that the UA must behave as if computed keyframes are produced dynamically immediately before interpolation.

Sounds good.

• Spec that specified values are retained internally, but getKeyframes() outputs the computed keyframes. (There's a lot of confusion about "when" keyframes are computed at the moment).

Yes, that's right. Obviously this is in the context of CSS animations where the keyframes have not been overridden by script. For script-generated Web Animations or CSSAnimation objects where we call setKeyframes() we return any specified values from getKeyframes().

• Add WPTs for difficult cases (I can contribute), reviewed by @birtles. Or investigate if they already exist.

Yes, that would be very helpful. I expect Firefox will fail a lot of them because we haven't implemented the ordering dependencies or registered custom properties.

When it comes to getKeyframes(), is there still any open issue re. "ambiguous clobbering" of properties?

I'm not aware of any.

Thank you!