[css-view-transitions-*] TPAC breakout session 2024

[khushalsagar]

Documenting notes from the TPAC breakout session.

[css-meeting-bot]

The CSS Working Group just discussed [css-view-transitions-*] TPAC breakout session 2024.

The full IRC log of that discussion

<ntim> present++
<vmpstr> noamr: welcome to the future of css view transitions
<vmpstr> noamr: <slide 2> previously on view transitions
<vmpstr> ... <slide 3> we have shared element transitions, things that transition into other things that are not the same element. here we have an icon transitioning to a cover
<vmpstr> ... what we did in the last few years and apple webkit did we did same document and cross document view transitions, SPAs and MPAs work with same experience
<vmpstr> ... we don't want the people to change their architecture for the experience
<vmpstr> ... the constraint is that it's same origin
<vmpstr> ... <slide 4> things we have in the oven right now: current view tranisiont are flat
<vmpstr> ... they create a pseudo element tree with snapshots old and new states. this tree is flat. it's a bunch of things stacked on top of each other. so we're working on nested transitions
<vmpstr> ... the other thing is that they are for the whole page, so we're thinking of scoped transitions that are more modular
<vmpstr> ... that's not the topic for today
<vmpstr> ... <slide 4> the story of navigation is incomplete in two ways
<vmpstr> ... first, it's limited to same origin navigations, and the world is not the same origin
<vmpstr> ... second it's gesture driven
<vmpstr> ... <slide 6> you define the transition by defining an image on both things and we use pseudo elements to style what's happening in between
<vmpstr> ... this happens in the new document
<vmpstr> ... <slide 7> in cross document view transition, all the authors can do is slap a rule and it magically works
<vmpstr> ... and the new document runs the transition
<vmpstr> ... <slide 8> why should this be only cross origin
<vmpstr> ... <slide 9> we have requests for this, you saw album to cover transition. what if it's a recommendation to a music provider. this is about a spacial relationship
<vmpstr> ... that allows you to navigate
<vmpstr> ... what about same owner and different origin
<vmpstr> ... origin is the security barrier on the web, with only a few exceptions
<vmpstr> ilya: commerce has a distinct page, where you want to navigate from ecommerce to a wallet provider
<vmpstr> ... when you finish the wallet, you want to transition back to the site. both of those are janky
<vmpstr> ... cross origin would do wonders here
<vmpstr> noamr: <slide 10> this is about the sense of orientation not specific to the app
<vmpstr> ... <slide 11> this is an example syntax, maybe we can say "to: any" to just allow the transition
<vmpstr> ... <slide 12> so why not then?
<vmpstr> ... <slide 13> because security
<vmpstr> ... this is an easy way to kill projects
<vmpstr> ... we were a bit more curious to understand what this means in practice
<vmpstr> ... <slide 4> so the main security thing is ancillary data leaks. this is data that isn't directly related to the request
<vmpstr> ... for instance if you ask for geolocation and you get it, it's not ancillary
<vmpstr> ... here you ask for a visual effect, but you need geometry style information and you can gather whether the user is logged in etc
<vmpstr> ... even timing whether the page allows the transition and how long it took, all of this is ancillary data and we don't want to expose any of this
<vmpstr> ... is this reasonable to expose something, but let's assume not
<vmpstr> ... <slide 15> second it's about miscoordination, think about stale links on the web
<vmpstr> ... where it leads to information that is no longer there and you can have buggy looking effects
<vmpstr> ... if it's cross ownership the other side may look bad
<vmpstr> ... <slide 16> third is spoofing, you can start a view transition and bind it to a scroll timeline or pause it
<vmpstr> ... the old side can do a bad image like call 1800 givemeallyourmoney and the user will think it's part of the new site
<vmpstr> ... so we can't do it
<vmpstr> ... so we look it at as a tradeoff triangle
<vmpstr> ... first is the expressiveness and then complexity
<vmpstr> ... <Slide 18> with security i think of it as constraints
<vmpstr> ... the first is about the spoofing part, the transition needs to be brief
<vmpstr> ... this transition can't be seconds or scroll timeline, it needs to be 1/2 second or less
<vmpstr> ... it needs to look like an animated transition
<vmpstr> ... the other one is isolated, so unlike same origin transition, here the constraint is that neither document can interrupt or observe the transition
<vmpstr> ... neither document can know if the transition happened
<vmpstr> ... this is a one way street for one document say it wants to transition and the rest is invisible
<vmpstr> ... this is what makes it possible security wise
<vmpstr> ... <slide 19> so let's say we have these principles, now we have 2 dimensions to explore
<vmpstr> ... we have root-to-root element-to-root and element-to-element
<vmpstr> ... <slide 20> this is root to root, the relationship is the whole page not per element
<vmpstr> ... this is still interesting as this is still an opener relationship
<vmpstr> ... this could be something that the page says without each elements
<vmpstr> ... <slide 21> but it's less inspiring than other stuff
<vmpstr> ... like if the opener wants an opacity style animation and the browser can run it by itself
<vmpstr> ... why do browsers control it? maybe there is some relationship there
<vmpstr> ... <slide 22> element-to-root, this is from material design. UXR found that users love this type of transition
<vmpstr> ... from list or from grid open to the whole thing. this is a common usage pattern
<vmpstr> ... this doesn't require coordination since only one page gives us interesting information. the first page gives us geometry etc, but the second page only gives us a surface
<vmpstr> ... <slide 24> we could have a special view transition name that oculd be the new root
<vmpstr> ... and the icon could transition to that name (new-root)
<vmpstr> ... the old page decides what is animation and it needs to brief, but it can do an opacity animations
<vmpstr> fserb: what do you mean by types?
<vmpstr> noamr: it's not important for this
<vmpstr> fserb: and the two is the list of domains?
<vmpstr> noamr: yes
<lea> q+
<vmpstr> noamr: <slide 25> the last is element to element as the last examples
<vmpstr> ... <slide 26> we have an interaction ... you still need to define all your animations in advance, and it allows you shared element
<vmpstr> ... and we need to avoid a way to avoid miscoordinations
<khush> q?
<khush> ack dbaron
<khush> q?
<khush> q?
<khush> ack lea
<flackr> q+
<khush> q?
<lea> oops sorry
<lea> wrong room
<fserb> np :)
<vmpstr> noamr: <slide 28> we have the following implementation options
<vmpstr> ... we can run it on the browser, or run this in a special document
<vmpstr> ... the third one is that we precompile the animation, compute all the keyframes, and then run them one at a time
<vmpstr> ... none of these are simple, this allows interesting UI. security wise it's achievable
<fserb> q+
<vmpstr> ... we're not sure if the expressiveness and complexity trade-off is worth it
<mmocny> q+
<khush> ack flackr
<Adam_Page> q+
<vmpstr> flackr: you mentioned that element to root the style comes from the old page. is that also true for element-to-element? if you're running in the new document you can get the styles
<vmpstr> noamr: <missed>
<vmpstr> ntim: can you go over impl options
<khush> q?
<vmpstr> noamr: one is to run in the browser, one is to do a new document
<vmpstr> ntim: what is browser mean
<vmpstr> khush: same as what the browser UI would do, and make it part of the browser UX
<vmpstr> ntim: like swipe animations?
<aaj> q+ aaj
<khush> q?
<vmpstr> flackr: this is defined by css though right?
<vmpstr> noamr: yes
<vmpstr> noamr: another is to have an anonymous document, and that's what's running the animation
<vmpstr> noamr: the third is precompile the animation all of the keyframes in advance to something like display lists and send them over to play
<khush> ack fserb
<vmpstr> fserb: first, because i don't know how it works. for the same origin, how does javascript interact with this. does it execute on the new page before the transition starts
<vmpstr> noamr: when the transition starts, the new page is already active
<vmpstr> fserb: doesn't it break the assumption to be as fast as possible
<vmpstr> noamr: it would be in a different renderer process and only deals with the surface
<vmpstr> fserb: so kind of like loads int he backhground
<vmpstr> khush: what you're saying is an existing problem when you're navigation across the two pages
<vmpstr> ... we just don't want to add more bad things like allowing an inifnite bad animations
<vmpstr> fserb: follow-up -- in the case where we're doing root-to-root or element-to-root, then a lot of time when you use motion to hide loading time
<vmpstr> ... as in you start animating before it's ready so that it looks instance
<vmpstr> ... in the element to element you can'yt do that, but in the element-to-root you could do that
<vmpstr> noamr: yeah, you can animate to an empty surface before it's ready
<vmpstr> ... it's an option vs render blocking
<vmpstr> fserb: both cases exist
<vmpstr> noamr: we'll need to research this
<khush> q?
<vmpstr> fserb: and the only other question: do you have because it needs a coordination that is hard
<vmpstr> noamr: we have requests for same origin and there is an ask for it, and it's a question we need to answer
<vmpstr> khush: the use case is you share a link and you get an image in there but when you click that image expands to an image on the new site
<vmpstr> fserb: but those htings don't track where that image was
<vmpstr> noamr: yeah we'd need to do something with opengraph maybe, and not rely on viewtreansition names
<khush> ack mmocny
<khush> q?
<vmpstr> mmocny: for element-to-element, i envision that the coordination has to happen ahead of time. would it help if ifrmes are involved like if you have an embed and then it's a same origin navigation
<vmpstr> noamr: this is portals
<flackr> q+
<vmpstr> ... this would be a different architecture that would be a different set of use cases, we didn't want to go there
<vmpstr> mmocny: can i ask for shoppify, when you do check out do you see the preview of the card are there iframes
<vmpstr> ilya: no
<vmpstr> mmocny: but you might only need element-to-root or root-to-root
<vmpstr> ilya: yes, element-to-root is a better experience
<khush> ack Adam_Page
<khush> q?
<vmpstr> Adam_Page: i'm on the aria working group, and only know css superficially, but i have two accessibility related questions: could there be anything that can disruptive to the mechanics of unloading a page
<vmpstr> ... when you're having something across the new page
<vmpstr> khush: this would be the same with cross document transition
<vmpstr> ... all of this animation happens on the new dom
<khush> q?
<vmpstr> Adam_Page: the second question is that accessibility was sensitive to the motion
<khush> q?
<vmpstr> noamr: all of the things we are there in css to do prefers-reduced-motion
<vmpstr> noamr: we might have different defaults
<khush> ack aaj
<khush> q?
<vmpstr> aaj: i had a question about what the perceived benefit is, since there is increasing complexity. does the root-to-root use-case, can we estimate how much of the problem that solves
<vmpstr> ... is that enough to be happy or do we go beyond that
<vmpstr> ntim: it would be weird an dinconsistent if we only did one of the use cases
<khush> q?
<vmpstr> noamr: it would be something we need to take a look <missed>
<vmpstr> aaj: you start with a simple implementation and go beyond that
<vmpstr> ntim: for developers it's confusing if you only cover cross origin but only have same origin it's confusing
<vmpstr> noamr: you can have a long animation or something else
<vmpstr> noamr: none of these are all shared element transitions
<vmpstr> khush: we haven't decided what page we are running this on
<khush> q?
<khush> ack flackr
<vmpstr> flackr: if we have element-to-root then going back would be weird that you can't do the opposite (shrink page)
<vmpstr> s/shrink page/shrink back/
<vmpstr> noamr: it's an open question
<vmpstr> fserb: good point
<vmpstr> khush: the other thing we want to talk about what's in the works, and talking about in view transitions next
<vmpstr> ... most users use gestyures to navigate and view transitions aren't supported there
<vmpstr> ... <slide 2> this is an example of cross document view transition and you click and you see the animation, when you do the back arrow it navigates you back
<vmpstr> ... <slide 3> this is something we're working on, and this is in other browser. the interaction when you clicked a link is the same, but when you did the swipe gesture and we're doing a animation when we do the swipe
<vmpstr> ... so we can't run the view transition there
<vmpstr> ... so we gave the users an ability to customize and took it away
<vmpstr> ... i wanted to mention that the way this is done is we capture a screenshot of the page, so when you swipe back, you can get the peek of where you're going and choose to go there or not
<vmpstr> ... <slide 4> here, we have a couple of example of other transition where customixation might be helpful where the browser can't do it automatically
<vmpstr> ... one is grid of items where you want the animaiton of detailed item to collapse to where it came from
<vmpstr> ... the browser can't do it, but the browser can't do it
<vmpstr> ntim: how is it related
<vmpstr> khush: if we didn't have gesture then the browser can run the transition but if you're running it with a gesture, then we can't do the view transition for that customization
<vmpstr> ... the browser isn't doing a job as good as the author could
<vmpstr> ... the second example is we're dealing with two tabs
<vmpstr> fserb: we're talking about gestures, but you're only mentioning back gestures
<vmpstr> khush: yes, we're starting with back gestures
<vmpstr> noamr: thinking of dismissing dialogs, there are other gestures that exist and we were discussing whether to call it swipe to back or a gesture problem in general
<vmpstr> flackr: the tab switch is going forward as well
<vmpstr> khush: <slide 5> so i'm diving deep into how to solve this problem
<vmpstr> ... the red and green part is showing you whether you're on the old or new part of the navigation
<vmpstr> ... most of the interesting stuff happens on the green side
<vmpstr> ... on the old state all we do is do the navigation and do captures, but everything else including animation happens on the new page
<vmpstr> ... <slide 6> this is what the box tree/pseudo tree looks like you have old pseudo that is static images, and the new pseudo is content that is coming from the current document and that's why it's live
<vmpstr> ... <slide 7> this model doesn't work, because by design you would need to run the animation before you've navigated
<vmpstr> ... just to mention why that's bad: there's a lot of state we can undo accidentally, like if you swipe a page and that cancels a payment page without committing to the back gesture
<vmpstr> ... this is why we need to do it on the old pahge
<vmpstr> ... <slide 8> the model we're proposing, we construct the pseudo tree on the old page, and now the line is red since all of the interesting stuff is happening on the old page
<vmpstr> ... if the user decides to cancel and we snap back, nothing happens
<vmpstr> ... if they decide to invoke we flip to the new state
<vmpstr> ... <slide 9> the pseudo tree looks the same, but now the old pseudo is live content from the old page and new pseudo is static content
<vmpstr> ... <slide 10> so where is that content coming from, since we haven't navigated there
<vmpstr> ... if it's a new navigation we don't know what we can do other than prerender
<vmpstr> ... for the history navigation, we have a few options. first is the UA screenshot, which is what we're doing for browser ui
<vmpstr> ... let's wrap it up as a pseudo and give it to the page
<vmpstr> ... the other option is when you went from page A to page B, we broke down the page and captured all of that state, we currently discard it. maybe we can persist it which gives you more interesting information
<vmpstr> ... the last one is back-forward cache. if it's already there, maybe we can ask it to produce one frame and that gives you a high fidelity content
<fserb> q+
<vmpstr> ... we're proposing starting with UA screenshot
<vmpstr> ... <slide 11> mostly this is what we already have but changing how we do this, but something that is missing is that we don't have a notion of gestures that navigate
<vmpstr> ... for this feature we need this
<vmpstr> ... we're thinking of introducing these concepts
<vmpstr> ... one nice thing is that we're not eh first ones to do this
<vmpstr> ... scroll driven animations solved that problem
<vmpstr> ... <slide 12> so the idea as far as CSS is concerned, this gesture can be introduces as a timeline
<vmpstr> ... <slide 13> the one we're htinking of is similar: the first block customize-ua-gestures, is the author saying that i want to run a custom animation, so suppress your own (browser's)
<vmpstr> ... and the second part says this animation is driven by that timeline
<vmpstr> ... so far this is nothing view transition-y going on
<vmpstr> ... for same document navigations, you don't have use view transitions so maybe you can do something simpler
<vmpstr> ... the goal is modularity
<vmpstr> ... the other thing is important: we're not giving developers power of what the gesture does, back gesture still navigates back
<vmpstr> ... all you get to do is customize the back gesture
<vmpstr> ... <slide 14> this is how you would glue it into view trnasiitons
<vmpstr> ... you give it a navigation component that says gesture(--back-timeline) and how UA css will automatically use that timeline
<vmpstr> ... the ideal goal is you defined a transition and it doesn't matter if the user clicked a button or swiped
<flackr> q+
<vmpstr> ... i'm happy to hear question
<vmpstr> ack fserb
<vmpstr> fserb: there are two things: gestures to the timeline, and automatically creating back transitions depending on the forward transitions
<vmpstr> khush: so distinction if you go back, then you load the last page
<vmpstr> fserb: this. might be nice to have without gestures
<vmpstr> ack flackr
<ntim> q+
<vmpstr> flackr: one thing you mentioned it has to run on the old page.. if we have a bfcache, you might unload the page and bfcache it and then reload it
<vmpstr> khush: yes and no, since we still run pagehide and things like that, and the page can cleanup the state
<vmpstr> flackr: it may not be that bad, but setting that aside: at hte point when you're capturing the screenshots, you can capture all of the info that you want to execute targeted transitions with captures and named elements
<vmpstr> khush: it has some intersection state
<vmpstr> ack ntim
<vmpstr> ntim: what if the website puts a vierw transition that is completely incompatible with the gesture
<vmpstr> ntim: where it doesn't makes sense with it
<vmpstr> khush: the site can do weird things with click based animations as well, it's all withint hte context of that
<vmpstr> ??: the site can scroll up when you scroll down
<vmpstr> fserb: the back one is decided by the ua
<vmpstr> khush: the action decided by the UA but the visuals are up to the author
<vmpstr> ntim: what if the page decides that we swipe from the right instead of from the left
<vmpstr> khush: if the site decides, and some of the semantics we have to decide
<vmpstr> ??: does that change if the page is RTL
<vmpstr> khush: yes, as a browser we change the direction of the UX
<vmpstr> noamr: you'd have the coordinates that are relative to the gesture and relative to the history
<vmpstr> fserb: have you thought of the next step, you're presenting as a linear timeline, but that doesn't work in 2d gestures
<vmpstr> noamr: you can have several coordinate systems, you can have two additional ones that are relative to teh gesture and to the historuy
<vmpstr> ... are you going in the direction of the browser history or finger direction.
<vmpstr> ... this is the type of thing to figure out
<vmpstr> khush: we went into a lot of thinking in browser UI so authors don't have to think about it
<vmpstr> fserb: the expressiveness is a complex problem
<khush> q?
<vmpstr> ntim: i'm concerned that you have a different platform that you want a different gesture to go back
<vmpstr> khush: you can start without exposing this gesture to this platform
<vmpstr> noamr: it's a progressive enhancement, if you don't support it nothign breaks
<vmpstr> fserb: have you thought about naming the gestures and then having those
<khush> q?
<vmpstr> tim: i was going to say that there is a case either way, being able to hook into the intent of the gesture is likely more good (but can make arguments for both)
<vmpstr> ... if you're hooked into the gesture itself
<vmpstr> ... history back is history back
<vmpstr> ntim: if you're hook into intents and you assume a certain visual effect, the gesture doesn't make sense
<vmpstr> khush: yeah you need a combination of both
<vmpstr> noamr: these are all the great questions

[khushalsagar]

This issue is just to archive the breakout notes.