https://drafts.csswg.org/css-images-3/#the-object-position
Percentages: refer to width and height of element itself
That's wrong. It should be analogous to background-positition:
Percentages: refer to size of background positioning area minus size of background image; see text
A percentage for the horizontal offset is relative to (width of background positioning area - width of background image). A percentage for the vertical offset is relative to (height of background positioning area - height of background image), where the size of the image is the size given by background-size.
Otherwise, the initial object-position: 50% 50% wouldn't center the image!