[css-images] Should EXIF image orientation data be ignored if it comes after the image data in the encoded image?

[smfr]

There was some discussion in #4165 about whether, in CSS images, EXIF image orientation should be ignored based on where it occurs in the encoded image file. Browsers seem, to some extent, to only respect EXIF if it occurs before the image data. If it comes at the end of the file, respecting it there might result in layout shifts as progressively loaded images rotate after loading.

Spec text is here.

[smfr]

Some complications:

• What if some future image format puts EXIF in the middle of the file?
• EXIF is per-frame, so for an animated (or multi-representation) image, the spec needs to be more nuanced.
• Implementations may not know, from their underlying image decoders, where the data is located in the file.

[smfr]

@svgeesus

[svgeesus]

Related:

• Testing eXIf, and CSS image-orientation png#310

[smfr]

I confirmed that Core Graphics/ImageIO APIs on macOS/iOS don't tell us the ordering of the EXIF metadata in the file. Behavior differences here may be happenstance based on when we try to access the data.

[jfkthame]

FWIW, it seems fundamentally wrong to me that we should get a different result depending where in an image file the EXIF metadata occurs (unless, of course, the image file format specifies that it's only valid in a certain place).

I understand there's a concern that

If it comes at the end of the file, respecting it there might result in layout shifts as progressively loaded images rotate after loading

but to address that, authors should be advised to web-optimize their images by ensuring the EXIF metadata is at the beginning. (Surely there must be tools available that can do this?)

[schenney-chromium]

There are WPT tests: css/css-images/image-orientation/image-orientation-exif-png.html and friends, where browsers agree on all but one case where Safari must process the chunks differently.

[svgeesus]

browsers agree on all but one case where Safari must process the chunks differently.

Yes. Note that the current spec text allows this so the test is incorrectly reporting a fail for Safari here:

If a user agent cannot ignore the metadata based its location in the image (for example, if the decoder being used does not report where in the image the metadata was located), it must use the metadata in all cases. (In particular, it is not valid to use the metadata only when the image is "small" and the entire image is downloaded quickly, but to ignore it if the image is large and the metadata isn’t downloaded until well after the image starts being displayed.)

(The parenthetical statement about small vs. large images does not seem to be tested on WPT).