Images with layout information

[litherum]

There are quite a few examples of images which are associated with additional layout information. When I say "layout information" I'm talking about situations where the images are more than just simple width × height boxes.

MathML

From the minutes of the Math on Web Pages Joint Meeting at the Lyon F2F:

arno: Moving on to another challenge
arno: Baseline alignment
...
arno: Needed for lining up equations

Consider a math equation like this:

If it's presented inline in the middle of a paragraph, proper typesetting would align the baseline of the line it's on with the baseline of the 𝑓(t)dπ(t) part. However, that isn't the lowest part of the content; with naive typesetting, the bottom of the content would lie upon the baseline and the whole formula would be too high on the line of the paragraph.

Logos

Consider the Apple Pay logo:

Similar to above, part of this logo should descend below the baseline. If you treat this logo as a simple image, the whole thing would be moved up too high, and wouldn't appear correctly on the line.

Currently, web authors often use icon fonts to work around this, as glyphs in fonts are allowed to have descenders. However, using icon fonts is an anti-pattern on the web. Instead, there should be a way to associate this baseline information with the logo itself.

Icons & Symbols

Consider the Play icon in the Music app on iOS. It is a little triangle, but when you have your finger pressed on it, there's a little circle surrounding the triangle.

If you get your ruler out and measure, it turns out that the Play button is not mathematically centered in the circle. Instead, it's optically centered, using the overshoot concept from typography. Since the right point of the triangle is much more narrow than the left edge of the triangle, the right point gets to extend farther outward from the center.

The facilities for this are present in the "Symbol Images" concept natively available on macOS and iOS. An artist can associate a symbol with hand-authored baseline information. This allows the system to place these icons properly.

Kanbun

Not all CJK characters are encoded in Unicode. These characters are usually represented with images. Even though, technically, CJK doesn’t have descent, in practice, CJK characters end up having descent in real fonts because this is the only way to have proper vertical alignment on lines with mixed scripts. When a kanbun image is used, it cannot currently have a descent, meaning it cannot be placed vertically correctly. This is particularly bad in vertical writing mode (which is common in CJK), where the baseline is in the middle of the line, meaning the kanbun image would naturally be in a completely incorrect place.

Conclusion

CSS should have facilities to allow for this kind of layout information associated with images. Layout engines should allow designers to place or tweak layout information such as baselines or vertical baselines for their image artwork.

There is currently no file format for associating layout information with images. So, there are a few ways this could possibly work today:

1. SVG is modified to allow an extra attribute on the root-most SVG element, which can indicate to the containing layout content additional layout information
2. System assets, such as Symbols on macOS and iOS, could be exposed to the web. These assets would have layout information built-in. E.g. background-image: system-ui(play-button); background-position: auto;
3. An additional CSS property that could modify the baseline of an image. E.g. image-baseline: 20% 50%; (for horizontal and vertical baseline). This would do more than background-position; it would actually move the layout box of the image.

Regardless of which method is chosen, CSS layout would have to be modified to include affordances for this additional layout information.

[litherum]

cc @fantasai @tabatkins @jensimmons @AmeliaBR @dirkschulze @dbaron

[emilio]

If the main use-case for this is vertical-alignment, you can do that already (for inline images or SVGs), can't you?

data:text/html,Foo <img width=10 height=10 style='background: green; vertical-align: 75px'> bar

[emilio]

Oh though I guess you're saying the <img> should contribute its own baseline somehow?

[litherum]

Yes. The image should be able to contribute its own baseline, and thereby be able to affect the size of the line box.

[othermaciej]

It would be good if the baseline info could be contained in the image without needing external style info (which might put this partly beyond the realm of CSS, but CSS could describe the semantic for such a thing, and could have a style-level override).

[AmeliaBR]

I do like the idea of exposing baselines as metadata on an SVG, and then using that for the default inline alignment when the SVG is a replaced object in CSS (e.g., an , or included via CSS content or list-style-image). For inline SVG text, I often make the SVG layout box only extend down to the baseline, and then use overflow: visible to get the descenders. But that's not an option for replaced content.

One concern is that this could be a new bit of information exposed from inside the file to the outside layout, and therefore we'd need to consider whether crossorigin restrictions should apply.

I'm not sure I understand how you're proposing this would interact with background position.

Do you know of any other image formats which have alignment metadata baked in? Is the Apple "symbol image" format standardized in any way that allows it to be used by other applications? (I notice they recommend importing from SVG for any customizations.)

As far as having access to system icons from CSS, I think that is an interesting idea that should be discussed in a separate issue. We'd need to look at whether there is a common set of icons available to apps in different platforms, and consider how it integrates into fallbacks (e.g., the image() function).

[AmeliaBR]

CSS could describe the semantic for such a thing, and could have a style-level override

The style override already exists, in the vertical-align property. So it really comes down to CSS saying that replaced inline boxes can have baselines if the content inside them defines them to exist. Then the layout would behave like an inline-block CSS box.

Also relevant: CSS Inline 3 has a note about a possible baseline-table property that could define multiple baselines on an <img> or similar element, separate from the vertical-align property that defines which one to use. For SVG, we could use the same property on the root element, and then define that the auto behavior of baseline-table is to use the values from the embedded file if they exist.

[othermaciej]

A property specifically for defining baselines seems like the right idea (although baseline or baselines seems like a more appropriate name than baseline-table).

[Crissov]

@frivoal considered something similar for images of rare (East Asian) characters that are supposed to blend into text seamlessly. #4013 “gaiji”

[dbaron]

This seems like reasonable thing to me. It seems like there are two things being discussed:

• image formats with optional baseline information, which CSS would then honor by default (when the information is present) instead of assuming that the (alphabetic?) baseline is at the bottom margin edge
• a CSS mechanism for specifying where the baseline(s) of an image are

It's not clear to me which would be preferable (or whether it would be best to have both).

[faceless2]

As it happens we've been considering this problem for some time as well.

Our goal was to improve layout for SVG embedded within HTML - for example, to insert text following a path in SVG as an inline element within HTML, and have the baselines match up:

<!DOCTYPE html>
<html>
<head>
<style>
p, text { font-size: 30px }
svg { display: inline; width:80px; height: 40px }
path { display: none }
</style>
<body>
<p>
Text on a
<svg>
<path id="curve" d="M 10 40 C 30 20 40 20 70 40"></path>
<text id="curveText"><textPath href="#curve">curve</textPath></text>
</svg>
and back to normal
</p>
</body>
</html>

Our plan was for an additional property on the SVG element which is the id of its descendant with the baseline to align on externally. So, in the example above we could have something like <svg baselineElement="curveText"> (syntax, property name etc all just for illustration). This would work for MathML or other embedded XML syntaxes where internal text is laid out with CSS, and for elements with a shadow DOM. We hadn't considered bitmaps.

(The second part of the problem was an auto-size property for SVG, something like <svg style="width:content-fit; height:content-fit>", to size the element to its visible boundaries. Not relevant for baselines though so I'll come back to that in another issue at some point).