[css-animations-2][web-animations-2] (proposal) Add pointer driven animations

[ydaniv]

Context

Goal

Allow users to animate elements based on the position of the pointer, what is sometimes referred to as "mouse parallax". Ideally we should provide a solution for the common features (listed below) that has a coherent model and API with the existing one for scroll-driven animations.

Common features

There are common characteristics to pointer-driven animations:

• The timeline is linked to the position of the pointer relative to (usually),
  and contained within, either the target (animated) element, a container of
  the target, or the entire viewport.
• Some effects are linked to a cartesian position of the pointer, others to
  the polar position.
• Timelines usually have a [1, 0, 1] or [-1, 0, 1] effect progress.
• Timelines are usually centered on the target's center, regardless of the
  element containing the timeline.
• Effects with delayed (transitioned) progress are common.
• Sometimes effects are linked to the velocity of the pointer.

Solution

Add a new non-monotonic, pointer-based timeline, similar to the scroll-based one. This timeline should be linked to the position
of the pointer over an element, or the entire viewport.
The progress of the timeline is linked to the position of the pointer from the start edge to the end edge of the source element or the viewport.

Prior art

This previous proposal by @bramus with more elaborate details, that are also relevant here, which relied on exposing a new pseudo-class for hovered element, plus exposing new environment variables for the position of the pointer.

JS implementations

Some libraries that allow this effect: Parallax.js, Tilt.js, and Atropos.js.

Live examples

• Pokemon Cards
• Mouse Parallax in Wonderland
• NY Times allbirds

---

Concept and terminology

Timeline

A new timeline that's linked to the position of the pointer, relative to an element/viewport - let's call it "source" - on a specific axis, either x or y.
Initially the timeline is defined by the source, starting at its start edge and uniformly increasing to its end edge.

Like ViewTimeline, the PointerTimeline is linked to the un-transformed layout box of the source (so that a timeline on the same element that's animated with transforms doesn't change with the animation).

Attachment range's centering (center shift)

It's common for pointer-driven animations to shift the center of the attachment range to a specific point, so that common animations with an effect progress of [1, 0, 1] or [-1, 0, 1] always reach 0 on that specified point.
Usually that point is relative to the animated element - let's call it "target" - rather than its source. Usually it's the target's center.

To achieve that, we also need to a way for authors to define that shift of the timeline's center to a specified point, either on the source, or on the target. The important thing to note here is that while the range is defined relative to the source, the shift of the range's center may be defined relative to the target.

Ranges

The timeline can then be expanded/contracted or stretched/squeezed using ranges.
These are also controlled in a similar fashion to ranges of ViewTimeline, but with some adjustments. The available ranges are: cover, contain, fill, and fit - building on top of known keywords of object-fit - though it seems having none for a range feels awkward, so currently it's replaced with fit.

All these ranges produce the same identical timeline if the range's center is at the source's center, i.e. center is not shifted.
However, if the range's center is shifted, the ranges behave differently and produce different timelines.

Note: in all the following examples, the outer rectangle (red with black rounded
border) represents the screen, the middle rectangle (green) represents the source,
and the inner rectangle (blue) represents the target. 

Cover

This range acts similar to radial-gradient's farthest-side keyword.
The attachment range reaches either 0% or 100% at the farther edge of the source, and then mirrored to the other side from range's center, so that the attachment range is always covering the source.

Example with center shifted to target's center:

Contain

This range acts similar to radial-gradient's closest-side keyword.
The attachment range reaches either 0% or 100% at the closer edge of the source, and then mirrored to the other side from range's center, so that the attachment range is always contained within the source.

Example with center shifted to target's center:

Fill

This range acts similar to the object-fit's fill keyword.
The attachment range reaches 0% at the start edge of the source, and 100% at the end edge, so that it's stretched to fill the source from its center outwards.
In practice this is equal to automatically set cover to the farthest edge and contain to the closest edge.

Example with center shifted to target's center:

Fit

This range acts similar to the object-fit: none keyword.
The attachment range reaches 0% at the start edge of the source, and 100% at the end edge, and maintains this size even if its center is shifted, so that it's simply displaced according to the center shift.

Example with center shifted to target's center:

Transitioned progress

It's also very common to see pointer-driven animations that have a "lerp" effect or a time-based transition on the effect's progress, so that it slightly lags behind the pointer position. This is usually done with a transition on the animated properties or by an interpolation on every frame between the current progress and the previous one.

This was suggested for scroll-driven animations in #7059, but was deferred to level 2.
Since it's a common pattern for pointer-driven animations, it could be a good opportunity to introduce it here.

Velocity

Some effects are linked to the velocity of the pointer, rather than its position.
This is also common for scroll-driven animations, but was deferred to level 2.
Mouse events already expose the delta between previous and current position via movementX and movementY, so it could be a chance to build on that and introduce that as well.

Polar Axes

Some effects are linked to the polar coordinates of the pointer, rather than its cartesian ones. While it could be very useful to add a "distance" and an "angle" axes to the proposal, they get very complex when trying to solve their progress
and ranges with the proposed model.
So it's probably best to defer them to further iterations, or to level 2 entirely.

---

Proposal

CSS

Add a new property pointer-timeline that takes a dahsed-ident as name and a one of x or y as axis.

For the anonymous timeline, a pointer() function that takes a source keyword and an axis keyword should be added as value for animation-timeline.
Possible values for source are: self for same element, nearest for nearest containing block , and root for viewport.

The animation-range should be extended to include the new range names: fill and fit.

In order to allow the attachment range's center shift, a new property animation-range-center should be added, that takes a <length-percentage> value and an optional keyword target. Without the target keyword, the value is relative to the source, otherwise it's relative to the target.
Inside the animation-range shorthand this value can either be introduced following an at, or a /.

Example:

@keyframes move-x {
  from { translate: -50%; }
  50% { translate: 0; }
  to { translate: 50%; }
}

@keyframes move-y {
  from { translate: 0 -50%; }
  50% { translate: 0 0; }
  to { translate: 0 50%; }
}

.container {
  pointer-timeline: --x x, --y y;
}

.figure {
  animation: move-x linear auto both, move-y linear auto both;
  animation-composition: replace, add;
  animation-timeline: --x, --y;

  /* alternatively with the anonymous timeline */
  animation-timeline: pointer(x nearest), pointer(y nearest);
  animation-range: cover at target 50%, cover at target 50%;
}

Web Animations API

Expose a new interface PointerTimeline:

enum PointerAxis {
  "block",
  "inline",
  "x",
  "y"
};

dictionary PointerTimelineOptions {
  Element? source;
  PointerAxis axis = "inline";
};

[Exposed=Window]
interface PointerTimeline : AnimationTimeline {
  constructor(optional PointerTimelineOptions options = {});
  readonly attribute Element? source;
  readonly attribute PointerAxis axis;
};

Add a new attribute rangeCenter to Animation of the following type:

dictionary TimelineRangeCenter {
  CSSOMString? subject = "normal"; 
  CSSNumericValue offset;  
};

(TimelineRangeCenter or CSSNumericValue or CSSKeywordValue or DOMString) rangeCenter = "normal"

Example:

const source = document.querySelector('.container');
const target = document.querySelector('.figure');

const timelineX = new PointerTimeline({
  source,
  axis: 'x'
});
const timelineY = new PointerTimeline({
  source,
  axis: 'y'
});

const moveX = new KeyframeEffect(
  target,
  { translate: ['-50%', 0, '50%'] },
  { duration: 'auto', fill: 'both' }
);
const moveY = new KeyframeEffect(
  target,
  { translate: ['0 -50%', '0 0', '0 50%'] },
  { duration: 'auto', fill: 'both', composite: 'add' }
);

const animationX = new Animation(moveX, timelineX);
const animationY = new Animation(moveY, timelineY);

animationX.rangeCenter = { offset: CSS.percent(50), subject: 'target' };
animationY.rangeCenter = { offset: CSS.percent(50), subject: 'target' };

animationX.play();
animationY.play();

---

Edit: My examples were off 😖 fixed the transforms.

[noamr]

I love the use cases and the direction!
One thing that struck me is that perhaps the center point can be the 0 of the timeline rather than the 50%? point and everything can expand from there? Then the keyframes could have 2 values (center->out) instead of 3 (out-in-out).

Then the CSS can look like this:

@keyframes move-x {
  to { translate: 50%; }
}

@keyframes move-y {
   to { translate: 0 50%; }
}

.container {
  pointer-timeline: --x x, --y y;
}

.figure {
  animation: move-x linear auto both, move-y linear auto both;
  animation-composition: replace, add;
  animation-timeline: --x, --y;

  /* alternatively with the anonymous timeline */
  animation-timeline: pointer(x nearest), pointer(y nearest);
  animation-range: cover from target 50%, cover from target 50%;
}

[bramus]

One thing that struck me is that perhaps the center point can be the 0 of the timeline rather than the 50%?

To keep a parallel with how ViewTimeline works, keeping the center at 50% makes more sense.

Also, I explored the pros and cons of the various coordinate systems in #6733

In one of the comments there, @tabatkins said:

Since the rest of the CSS uses "0,0 is top-left of box", it would work the same way.

So there’s also a parallel to draw there.

[ydaniv]

One thing that struck me is that perhaps the center point can be the 0 of the timeline rather than the 50%? point and everything can expand from there? Then the keyframes could have 2 values (center->out) instead of 3 (out-in-out).

That's an important point! I recall why that won't work. For some effects, e.g. translation/rotation/skew, a type of [-1, 0, 1] range is desired, and you can't get that with just 2 keyframes and a progress that goes 0->1. Unless we invent some sort of a "flipping" mechanism, but I don't think we want that. Especially if we can relatively easy reuse what we already have.

[ydaniv]

One thing that struck me is that perhaps the center point can be the 0 of the timeline rather than the 50%?

To keep a parallel with how ViewTimeline works, keeping the center at 50% makes more sense.

That's also true.

[noamr]

One thing that struck me is that perhaps the center point can be the 0 of the timeline rather than the 50%?

To keep a parallel with how ViewTimeline works, keeping the center at 50% makes more sense.

That's also true.

Fair points! Thanks for clarifying.

[css-meeting-bot]

The CSS Working Group just discussed [css-animations-2][web-animations-2] (proposal) Add pointer driven animations.

The full IRC log of that discussion

<TabAtkins> ydaniv: New proposal for pointer-driven animations
<TabAtkins> ydaniv: gonna show some examples
<TabAtkins> ydaniv: here's one example, parallax driven by pointer
<TabAtkins> ydaniv: another example, a whole suite of elements parallaxing
<TabAtkins> ydaniv: another example, each element is its own timeline (cards slightly shifting in 3d to avoid the pointer)
<TabAtkins> ydaniv: here's something we built internally, using a polyfill
<astearns> (example links in the issue)
<TabAtkins> ydaniv: (more parallax)
<TabAtkins> ydaniv: (a cat watching the pointer)
<TabAtkins> ydaniv: [slides times]
<TabAtkins> ydaniv: proposal for a pointer-timeline
<TabAtkins> ydaniv: driving an animation based on position of the pointer
<TabAtkins> ydaniv: relative to a box, or to the screen
<TabAtkins> ydaniv: builds on what we already have in scroll-driven animations
<astearns> s/screen/viewport/
<TabAtkins> ydaniv: some commonf eatures, attachment is usually the box of: animations target, a parent container, or the whole viewport
<TabAtkins> ydaniv: effects usually have a focal point, effect range is [1, 0, 1], [-1, 0, 1], or [0, 1, 0]
<TabAtkins> ydaniv: focal point is usually relative to the effect's target, or the timeline's subject
<TabAtkins> ydaniv: more common features: delayed progress
<TabAtkins> ydaniv: progress linked to velocity
<TabAtkins> ydaniv: some effects rely on polar coordinates
<TabAtkins> ydaniv: proposal is fully bikesheddable, some features are deferable
<TabAtkins> ydaniv: new property 'pointer-timeline: <name> <axis>'
<TabAtkins> ydaniv: works like view-timeline
<TabAtkins> ydaniv: new functional notation for anonymous animations: pointer(<axis> [ self | nearest | root ])
<TabAtkins> ydaniv: nearest means containing block of the target
<lea> q?
<TabAtkins> (because it's anonymous)
<TabAtkins> ydaniv: root is the whole viewport. can be a bit tricky, i'll explain later
<lea> q+ to ask can we reuse a referencing mechanism from another part of CSS? E.g. anchor positioning
<fantasai> suggest using container rather than nearest
<fantasai> since nearest parent may not be contianing block
<TabAtkins> ydaniv: new range names, entry and exit dont' really make sense
<TabAtkins> ydaniv: can keep cover, contain
<TabAtkins> ydaniv: propose adding fill and fit, taking from object-fit
<lea> Do we have a link to the slides?
<TabAtkins> ydaniv: by default the ranges aren't different unless you shift the center of the attachment point
<TabAtkins> ydaniv: new property, range-center to set the focal point
<TabAtkins> ydaniv: takes <length-percentage> or center/start/end keywords
<TabAtkins> ydaniv: can also take normal|target keyword, to target the animation target instead
<TabAtkins> ydaniv: have some demos
<TabAtkins> ydaniv: [shows off some illustrations]
<TabAtkins> ydaniv: in this, the red square is the viewport, bluie is the target, green has pointer-timeline
<TabAtkins> ydaniv: non-ancestor relationship, 50% is just in the center of the green
<lea> Material buttons are also another use case: clicking on a button grows a highlight from the pointer to the whole button https://m2.material.io/components/buttons
<TabAtkins> ydaniv: if blue is an ancestor, center is the center of blue, 0-100% is centered on that
<TabAtkins> "contain at target center"
<ydaniv> https://docs.google.com/presentation/d/1QwyeTDK8n6RU1px0-iu0Txi2aJH6zTL6d7X6_z-kk4A/edit#slide=id.g3007a239b93_0_79
<TabAtkins> ydaniv: "cover at target center", blue's center is still used, then it expands so 0-100 covers the green
<TabAtkins> ydaniv: "fill at target center", blue's center is used, but 0-100 exactly fills the green, so 0-50 and 50-100 use different scales
<TabAtkins> fantasai: would you want an easing function for that, since you're compressing one side and expanding the other?
<TabAtkins> ydaniv: I think you already have the potential for that...
<TabAtkins> flackr: I think th e point is your animation keyframes will have a 50% frame, and you can apply easing to work
<TabAtkins> fantasai: right, it's just a sharp change from compressed to expanded
<astearns> q?
<TabAtkins> TabAtkins: but a good easing choice can make that smooth, and making it easier is a separate thing
<TabAtkins> ydaniv: "fit at target center", blue center is used, 0-100 range is the *size* of green, but centered on blue
<TabAtkins> ydaniv: as you scroll you can stretch/squish the timeline
<TabAtkins> ydaniv: already can happen in scroll timelines if fixpos, but here's it's more common
<TabAtkins> ydaniv: the way we handle this fact when you're centering at the target is when you're using cover/contain, and your subject is the viewport....
<astearns> the pointer-driven timeline shifts at every scroll in that case
<TabAtkins> ydaniv: as you scroll, it will change the timeline
<TabAtkins> ydaniv: we shipped this to users with the polyfill
<TabAtkins> ydaniv: we use scroll-end to recalc the timeline on each scroll
<TabAtkins> ydaniv: [shows off "contain at source 200px 200px"] contrived to have a range at a specific point, but possible
<TabAtkins> ydaniv: [demo]
<TabAtkins> ydaniv: [demo shows off the range results as a gradient when various options are used]
<TabAtkins> ydaniv: [recaps the proposal]
<astearns> ack lea
<Zakim> lea, you wanted to ask can we reuse a referencing mechanism from another part of CSS? E.g. anchor positioning
<TabAtkins> lea: is there any way we could harmonize these refs with other parts of CSS?
<TabAtkins> lea: anchor-pos allows you to ref other elements
<kbabbitt> q+
<TabAtkins> lea: also, was wondering, the pointer() function is only available in animations?
<TabAtkins> lea: for second, this is just a way to draw animations, like view-timeline and scroll-timeline
<TabAtkins> s/lea/ydaniv/
<fantasai> s/draw animations/drive animations/
<TabAtkins> q+ to respond
<kbabbitt> q- later
<TabAtkins> ydaniv: the functions are used just for anonymous timelines
<TabAtkins> lea: was thinking this seems like a heavyweight solution if all you want is to get the relative position of a pointer for a background
<TabAtkins> lea: was wondering if we could add a simple function that gets you the pointer location relative to whatever
<TabAtkins> lea: I come across those cases fairly frequently
<TabAtkins> ydaniv: I think what you were asking was asked for by Bramus
<TabAtkins> ydaniv: it didn't go further, there were some issues?
<dbaron> Scribe+
<TabAtkins> flackr: we do the same thing for scroll/view timeline
<TabAtkins> flackr: they're not exposed as variables to use
<TabAtkins> flackr: they can be plugged into animations
<TabAtkins> lea: to resolve cycles?
<TabAtkins> flackr: not quite, if you have a variable it's non-trivial to know how we can respond to that off the main thread
<dbaron> TabAtkins: for now this is intrinsically something that wants to run off main thread, because it's scroll responsive
<dbaron> TabAtkins: we've solved that for animations but not the general case
<TabAtkins> flackr: if your animation *can* be composited, as long as we can determine the progress on the compositor we can accelerate the whole thing
<TabAtkins> flackr: but if it requires animating style we havne't figured that out yet
<kbabbitt> q-
<flackr> TabAtkins: I also think that one of the progress functions in values and units is meant to address handling an animation locally
<dbaron> TabAtkins: I also think one of the progress functions in values and units is meant to address this handling the animation locallly like this. Saying what timeline you want to use, just pops the value right there.
<kizu> q+
<dbaron> TabAtkins: rather than getting a number directly out of the timeline you can directly ???
<astearns> ack TabAtkins
<Zakim> TabAtkins, you wanted to respond
<dbaron> TabAtkins: I think there is something over there -- fantasai wrote that section.
<dbaron> fantasai: are you thinking about progress() and mix() being able to pull a value out of a timeline?
<TabAtkins> fantasai: thinking about progress() and mix() functions that Miriam and I proposed could take a timeline
<TabAtkins> TabAtkins: yes, the grammar for that is currently in the spec
<astearns> ack kizu
<TabAtkins> kizu: +1 to this proposal
<TabAtkins> kizu: this is something authors want
<TabAtkins> kizu: I think animations as a first step is a good idea
<ydaniv> this should work: https://docs.google.com/presentation/d/18P_EX0Vy61Fy6iLUPP6YsNIe2GSHmKtiltEabPXhyo0/edit?usp=sharing
<TabAtkins> kizu: I've played a lot with events
<TabAtkins> kizu: we could do this in a hacky way with view timelines, by getting position as a pixel value
<TabAtkins> kizu: I don't know if the proposal allows doing this rather than using the % directly
<TabAtkins> kizu: if you want to display something you might just want an offset directly; calculating the offset from a % is possible but would like to avoid it
<TabAtkins> (I think you might be able to get that by driving an animation between 0 and anchor-size(), if that's applicable)
<flackr> q+
<TabAtkins> ydaniv: elaborate?
<TabAtkins> kizu: if I want to use this value in other places, like using a % in background. If the % is relative to something *else*, it's not usable directly in backgrounds.
<TabAtkins> kizu: We kinda could do in some hacky ways but it's not easy
<TabAtkins> kizu: so if the container is 300x300, we'd like a way to access a length like "120px" for 40% progress
<TabAtkins> ydaniv: This is using normal animations... basically you just set everything in keyframes
<TabAtkins> kizu: I'll comment with details on the issue
<TabAtkins> flackr: for mouse it's clear that this represents the hovered position
<TabAtkins> flackr: what happens with touch. primary touch point?
<TabAtkins> ydaniv: great question
<TabAtkins> ydaniv: I have libraries that polyfill this behavior, they each do something different
<TabAtkins> ydaniv: one is built to use the gyroscope
<TabAtkins> ydaniv: another uses long-press to trigger the animation, then as you drag your touch it moves with your finger
<TabAtkins> ydaniv: I think the last one just falls back to hover, so if you tap it'll update
<TabAtkins> ydaniv: what we did is just wrap everything in a (hover) MQ, so it's desktop only
<TabAtkins> flackr: so it's default to legacy desktop behavior, if you tap it uses that point
<TabAtkins> ydaniv: yeah
<astearns> ack flackr
<TabAtkins> flackr: does this only update when the cursor is within the observation container?
<fantasai> slides: https://lists.w3.org/Archives/Public/www-archive/2024Sep/att-0011/CSSWG___TPAC_2024___pointer-timeline.pdf
<TabAtkins> flackr: the green boxes
<TabAtkins> flackr: if I move my animation out the box on the left side, circle the box, and reenter on the right, is it tracking my pointer the whole time, or updating when it reenters?
<TabAtkins> ydaniv: the whole time
<TabAtkins> astearns: so I'm assuming this was for socializing the proposal - I think you get a "yes, that's interesting"
<TabAtkins> astearns: did you want anything more?
<lea> +1 to solving the problem, the use cases are common
<kbabbitt> +1 let's do this
<TabAtkins> +1 from me, i'll raise issues
<TabAtkins> ydaniv: demos are also shared in the slides
<TabAtkins> fantasai: do we want to continue just in the issue, or start putting it in a draft?
<TabAtkins> astearns: in the issue for now, it's still early. some point soon can move it to a draft