Behavior of createImageBitmap with { imageOrientation: "flipY" } is unclear

[junov]

Spec: https://html.spec.whatwg.org/multipage/imagebitmap-and-animations.html#cropped-to-the-source-rectangle-with-formatting

Currently, imageOrientation is applied by createImageBItmap after cropping and resizing. The problem with this is that "flipY" causes the source image's EXIF orientation metadata to be ignored. In this case, it is unclear whether the coordinate space used for cropping and resizing should ignore the orientation metadata. It would make sense for it to be so, but it is weird for a step's behavior to be modified by a step that comes after it. In any case, the spec should be more explicit about how the options interact with each other.

IMHO, the imageOrientation option should be applied earlier (before step 2). This would make the behavior easier to reason about. It would break backwards compatibility though. But perhaps we should seize the opportunity while usage is still low.

[junov]

Also, "flipY" should have been "flip-y".

[zcorpan]

But perhaps we should seize the opportunity while usage is still low.

I think so!

[annevk]

cc @whatwg/canvas

[kdashg]

flipY is I think is mostly intended for dealing with uploads to webgl (for which textures are kinda y-up, but It's Complicated for uploads), and so is best applied at the very end of all transforms.

EXIF orientation feels to me like it should be the very first thing applied, and done universally ideally?
For example, a wide 3000x1000 landscape might be stored as 1000x3000-with-exif, but the fact that it's 1000x3000 should probably be hidden, and we should probably effectively operate on this as 3000x1000, right?

That flipY causes EXIF to be ignored is probably wrong, and we'd prefer EXIF to apply, I should think!

@kenrussell, @annevk: thoughts? Do you remember why it's like this? It feels like it's likely due to merging of similar logical codepaths/spec-steps?

[kenrussell]

Agreed that the EXIF orientation should be applied first, and that the flipY option should not override it, but be applied later.

Feedback from early WebGPU users indicates that the flipY option is still needed in that API, so it turns out it's not just a WebGL-specific functionality but one needed in all of the imperative graphics APIs.

[annevk]

I traced "disregard EXIF" back through #741 to https://wiki.whatwg.org/wiki/ImageBitmap_Options which suggests it originates with @junov. Might well have been the output of a team discussion of course. (I agree that not disregarding EXIF makes more sense.)

[kenrussell]

@junov mentioned in #7210 - as well as above - that he thinks the EXIF orientation should be applied first, then the flipY operation, if requested. I think this issue could be duplicated into #7210 and that the spec should be updated because Firefox, WebKit, Edge, and Chromium are all in agreement that that's the semantic that should be implemented.

[junov]

I traced "disregard EXIF" back through #741 to https://wiki.whatwg.org/wiki/ImageBitmap_Options which suggests it originates with @junov. Might well have been the output of a team discussion of course. (I agree that not disregarding EXIF makes more sense.)

Yes, I recall writing this. It was the outcome of an exchange that kbr@ and I had with some WebGL developers. I can no longer find the old thread, but It seems obvious to me now that the use case we considered must have been either contrived or misunderstood.