[css-fonts-5] Text Fitting: Scaling of things based on font-size

[kizu]

My main point of disagreement with the Chrome team's feedback is the proposed method of scaling the font.

A few quotes from the feedback where they argue about not scaling any dependent properties or elements based on the change of the font-size, and proposing that it should potentially be implementation-dependent and up to browsers.

Several quotes from the feedback

• Other <length> values should not be scaled even if they are font-relative.
  e.g. padding of inline elements
• Non-text parts such as inline images and inline-blocks should not be scaled.

The fit-width text feature needs to work after line breaking because it needs original widths of lines. Adding another font shaping step in the fit-width text processing would be possible, but it's hard to add another style computation in it or to restart from style computation.

• Font-size and scale methods: If we drop the <fit-method> option entirely, the scaling method should be implementation-dependent, and probably it would be either font-size or scale.
  These methods produce very similar results, and their differences are likely not significant to most users. In the prototype, the font-size method didn't scale letter-spacing and word-spacing, and the scale method scaled them. In order to minimize differences between two scaling methods, latter-spacing and word-spacing should always be scaled because it's difficult to implement non-scaled letter-spacing and word-spacing with the scale method.

• Existing properties with <length>: Scaling font-relative <length> values is not practical due to implementation complexity. In Chromium, we only have a computed <length> value at the "layout" stage, as a numeric value without unit information. Changing this to retain unit information would increase memory consumption even on pages not using this feature.
  The line-height property is an exception, as we retain its unit information and can resolve font-relative <length> at the "layout" stage.

And probably semi-related

• Performance Considerations: Concerns were raised about the performance of the font-size method, as repeatedly reshaping fonts to find the best fit is impractical. The prototype addressed this with a combination of one-shot reshaping and a scaling transformation.

---

If I were to sum things up as I understand it: current Chromium implementation converts font-relative values at the Style computation step, so we lose that information when we need to apply the fitting algorithm after line breaking. We can still do so for line-height as it retains the necessary information.

I very strongly believe that it will be wrong to do scaling that does not affect the actual font-size and is implementation-dependent with a possibility of just scaling the glyphs and letter-/word-spacing alongside them. For many typographic purposes this will be not enough.

Many fonts have optical sizing variable axis, which is used to adjust various aspects of glyphs based on the applied font-size, improving the readability of the text.

Many elements will need to be scaled alongside the text: things like SVG icons with dimensions defined with em come to mind, alongside all the various paddings and margins that use font-relative units and need to be scaled alongside the text.

I have an idea of a proposal, which on the surface, seems to address the issues raised by the Chrome team.

The idea: what if had a new font-relative unit that would not be converted to its absolute value, or could retain the original value in some way before arriving to the text-fitting algorithm?

Then, I see two options:

• For any element with the text-fitting feature enabled, when we calculate styles, instead of converting the font-relative units to their absolute values, we would replace them with this special unit. Then any existent code will continue working as is, without any potential perf regressions, and only code that would add text-fitting will be impacted. Or — could this be done automatically somewhere inside the engine, without exposing this unit to authors at all?
• Don't do the above, and go with what Chrome team proposes by default, at the same time suggesting authors to use this new unit if they want to opt-in some of the places in their code to be scaled alongside the font.

I need to think more about it, but I can see pros/cons for both ways. On one hand, I can see cases where you'd want to keep using the absolute values of original fonts, so having both would be nice. However, I think in most cases authors would want things to scale alongside the change, and we already have tools like registered custom properties that allow retaining the original font-size's absolute value, so, ideally, if possible, this process should be transparent for authors, and “things should just work”.

If the feature were to be implemented as implementation-dependent and without correct scaling of things that depend on the change of the font-size, the feature will be unusable for many cases. Yes, it will cover some of them, but for anything else authors will have to rely on much more cumbersome and potentially less accessible hacks.

And not scaling things like paddings and margins could potentially lead to other accessibility violations for authors, where the spacing between words could become too small, for example.

If we were to make this a standard CSS feature, it should be robust. It is possible to recreate this feature today in a finite number of steps through text duplication — and it does not affect performance of anything else. A native implementation could be only faster, and if it seems that it is impossible now in some of the current engines, we should look for a solution, not discard the feature altogether or trim its abilities to be unusable.

[tkent-google]

Our feedback was that the text scaling method was implementation-dependent (reshaping or scaling transform), while other <length> handling should be standardized as "never scale".

I understand that ideally all font-relative values should be scaled, however I'm afraid it's implementation complexity is too high and some browser vendors give up implementing it.

calc() with font-relative values would be problematic. If an element in a line box has something like margin-left: calc(1em - 5px), it's difficult to compute the scaling factor of the line.

An alternative option is to give up on reshaping, and just apply scaling transform to the entire line box at once, including any inline images, margins, padding, etc. It means absolute <length> values are scaled too.

[kizu]

calc() with font-relative values would be problematic. If an element in a line box has something like margin-left: calc(1em - 5px), it's difficult to compute the scaling factor of the line.

The 1em - 5px case (when used for not font-size) is relatively simple to calculate, as this is just a static contribution that can be calculated for all elements in just one extra step (best method can vary: we can either calculate what will happen with font-size: 0, which will allow us to get all the static contributions, or use original font size vs, say, 200% font-size while freezing the optical sizing axis, which will allow us to see if there is anything that adds non-relative values and accommodate the available space based on them.)

The more complex cases, like with reverse relative units 5px - 1em or when regular calculations like 1em - 5px are used on nested elements could break the calculation of the available space, but I consider these edge cases which would be ok to discard. Because if that's the price we pay for all the regular use cases to be available, that is much preferable to throwing all use cases away.

I will come up with some examples that demonstrate real use cases and how not scaling font-relative units in them will make the feature not produce the expected result.

[tkent-google]

I think we need to agree on whether the scaling of the font-size by this feature should affect the computed value or the used value. As a browser vendor, we lean toward adopting the used value approach.

Used Value Approach

If we choose the used value:

• It would be difficult to calculate the scaling factor if we need to scale only font-relative values, and it would be difficult to scale only the font-relative values of various properties because we have only computed values during the layout step. It would likely be a choice between two outcomes:
  • "Nothing, including font-relative values, is scaled."
  • "Everything, including absolute values, is scaled."
• The result of window.getComputedStyle() will not be affected.
• The implementation complexity should not be high.

Computed Value Approach

If we choose the computed value:

• Due to the precedent set by container queries, it may be possible to recompute styles during the layout step.
  • It would be easy to update the font-relative values of various properties.
• Calculating the scaling factor during the layout step would be difficult because we have only computed values during the layout step.
• The result of window.getComputedStyle() will be affected. When a single inline element is split across multiple lines, applying a per-line scaling could result in a different computed value for each line. We need to define getComputedStyle() behavior in such case.
• The implementation complexity is expected to be quite high.

[kizu]

The way I was thinking it could be done, it should not affect the font-size/line-height on the element with the text-fit at all, similar to how changes to those on a ::first-line do not affect them.

Anything on the element itself will not scale based on the adjustments, but any nested elements will need to be scaled as if they inherited all the new values.

[tkent-google]

Ok, I agree that the feature won't affect computed values.

similar to how changes to those on a ::first-line do not affect them.

Currently an element in ::first-line has the original set of computed values and another set for ::first-line, and the former is exposed via getComputedStyle(). I think It would be possible to apply it to inline boxes in the fit-width container, however I'm not sure how much difficult to apply it to nested formatting contexts in the fit-width container.

I still don't have a good idea how to compute scaling factors with arbitrary font-relative values at a reasonable cost. Chromium doesn't have a way to get a calc expression from its resultant computed value.

---

Anyway, I still prefer a simpler behavior like "scale only text" or "scale everything".

[tabatkins]

I also agree this all needs to be used-value time. We're in the middle of layout; computed values have already happened and we're doing used-value-time stuff now. I also don't think we should, by default, be scaling different units differently anyway. We have a long line of evidence that when units change their relative ratios in unexpected ways, pages tend to break because authors didn't expect that.

Also, to be a little clearer about what knobs are actually available for us to reasonably/performantly twiddle, here's (roughly) how Chrome's text-shaping and line-breaking works (and I believe other impls are roughly similar):

1. We have a paragraph-worth of text, and a list of fonts that can be used on that text.
2. We just toss both of these at Harfbuzz, who then lays out one big long line accordingly (shaping the text, using different fonts as needed when glyphs dont' exist, etc.
3. Harfbuzz returns an annotated list of (letter, position) pairs.
4. We identify the linebreak opportunities, and find the furthest-along one that is less than or equal to the available space we are laying a line into.
5. We ask Harfbuzz to lay out just the text up to that point. (It can reuse a lot of its previous shaping work, only redoing the last few glyphs usually.)
6. It tells us the new positions. If they're the same or smaller, great! Line finished, move on to the next one. If they're larger, we back up to the previous line-breaking opportunity and ask it again. Repeat until it fits or we're at the smallest unit between linebreak opportunities.

Notably, in this process, exactly what Harfbuzz is doing with the text, and what the fonts allow it to do, are more or less opaque to us. For example, whether a font can do optical sizing adjustments is none of our business, generally.

Another notable bit is that font shaping is entirely non-linear. With common, well-behaved fonts it's usually nearly linear - lay out with twice the font-size, get text approximately twice the length - but absolutely nothing stops it from doing weird and wacky things. We force line layout to eventually hit a base-case by only ever going backwards in the break opportunities; even if breaking at a later point would alter shaping to allow it to fit, we won't discover that (as it would, in general, require laying out at all possible linebreak opportunities, which is ridiculously costly). Fonts are turing-complete programs and can do whatever they want. Even just a simple font-size change can result in having to use a different font for rendering, if it's a bitmap font, resulting in a completely unrelated line length.

---

The upshot of this is that we really don't have much we can do to offer authors control over this, without an enormous amount of engineering effort to rip open the shaping process and expose a lot more of it to the broader browser process. This is why @tkent-google is trying to limit what we do. In particular, I think we've got three reasonable choices we can offer authors that don't require massive engineering efforts:

1. Do a literal scale(), just in the inline axis. No reshaping at all, just a visual transform. (This is done today in CJK text a lot. It's not great, but it doesn't change the line height, so sometimes it's worth it.)
2. Do a literal scale(), but the same in both axises. Still no reshaping at all, just a visual transform.
3. Change the font-size. Because fonts can do arbitrary stuff in response, we just make a guess at the target font-size, rerender with that, then do a scale (like option 2) to finish fitting it exactly.

In all these cases, there are a few other options we could offer, which are probably worth letting authors choose between:

1. Adjust each line individually, or adjust an entire IFC with a single scale so all lines are consistent, using the smallest factor among the lines. (Default: each individually, probably.)
2. Whether atomic inlines get scaled or left alone. (Default: scaled, probably.) (We don't really want to redo layout inside the atomic, because that's another case where the results can be completely non-linear; line-breaking inside the atomic might change, for instance. So by "scale" I literally mean transform-type scaling.)
3. Whether layout-effecting decorations, like borders and margins on the inlines, are scaled or not. (Default: not scaled, probably?)

(2) and (3) just change how many things participate in the scaling size determination, so they're not a big complexity hit.

---

Any further controls would just be more shots in the dark that require correspondingly more whole-line/paragraph relayouts. For example, if we ask to do optical-sizing scaling, only some of the fonts on the line might support that. We don't know that, tho - only Harfbuzz does, currently. So it might result in our guess being way off, when only part of the text on a line ends up being scaled in this way, and so we either have a much larger transform-scale to get a perfect fit, or do another complete round of sizing with different values.

We certainly can ensure the syntax allows for more controls in the future, but I don't think we should offer them for now. Just the three scaling strategies, plus the options I listed, should be enough for now imo.

[tkent-google]

We certainly can ensure the syntax allows for more controls in the future, but I don't think we should offer them for now. Just the three scaling strategies, plus the options I listed, should be enough for now imo.

I agree with this comment.

The first version of this feature should be as simple as possible to accelerate its standardization and implementation, allowing web authors to use it sooner.

Recomputing font-relative values is still of unknown feasibility, and it would be better to defer this to a later stage to leave room for future feature extension.