Adding support for High Dynamic Range (HDR) imagery to HTML Canvas

[palemieux]

こんにちは TAG-さん!

The Color on the Web CG is kindly requesting a TAG review of its proposed modification to the Web Platform to support High Dynamic Range (HDR) imagery in HTML Canvas.

The complete proposal is at https://github.com/w3c/ColorWeb-CG/blob/hdr_canvas_r2/hdr_html_canvas_element.md

Today HTML Canvas supports only 8 bit per color channel and two PredefinedColorSpace color spaces (srgb and display-p3). This is insufficient for High-Dynamic Range (HDR) imagery, which is in widespread use today. Accordingly, the following API modifications are needed to manipulate HDR images in HTML Canvas:

• add BT.2100 color spaces to PredefinedColorSpace
• add higher bit depth capabilities to CanvasRenderingContext2DSettings
• add higher bit depth capabilities to ImageDataSettings
• add image color volume information to ImageDataSettings and CanvasRenderingContext2DSettings
• add display color volume information to the Screen interface of the CSS Object Model, to determine the characteristics of the display on which the image is being reproduced

Please note that the proposed screenColorInfo property adds a fingerprinting surface. This feature is however intended primarily for advanced color volume mapping applications and is not believed to be generally necessary.

Contacts: @palemieux and @svgeesus (Color on the Web CG co-chairs)

Feedback, issues and questions are welcome at mailto:public-colorweb@w3.org and https://github.com/w3c/ColorWeb-CG/issues.

[svgeesus]

Hello TAG,

On 19 Feb 2024 in Tokyo there is a cross-SDO meeting HDR Stills Experts Day, with the International Color Consortium, W3C, and ISO TC 42/WG 23 (Photography) on the subject of HDR imagery. It would be super-useful to get any TAG input from this HDR in Canvas review ahead of that meeting

[palemieux]

@svgeesus Where is the meeting announced in the context of W3C? Any requirements for participation?

[svgeesus]

@palemieux It is on the ICC site in the Member-only area. It isn't a public meeting. I will be there, and Chris Cameron also.

[palemieux]

What is the process if myself and other W3C members wish to attend?

[palemieux]

To be clear, my concern is with the output of the meeting being described as including an opinion from the W3C and/or input provided in the meeting be presented as W3C input, without such input having been discussed/reviewed by W3C participants.

[svgeesus]

I think the proposed modification to the Web Platform to support High Dynamic Range (HDR) imagery in HTML Canvas has been adequately discussed that it can be presented at the meeting, no?

[palemieux]

Ok. Which document(s) are you planning on presenting -- beyond https://github.com/w3c/ColorWeb-CG/blob/hdr_canvas_r2/hdr_html_canvas_element.md? It would be good to inform their respective groups.

[jyasskin]

This doesn't seem like the right repository for this side discussion.

[LeaVerou]

Hi there,

@plinss @ylafon and I looked at this today during a breakout.
We had a few thoughts and questions:

1. The part of the API around checking whether a color is in within the volume of the output device and mapping it manually by the author seemed both too close to the hardware to be future-compatible, and too low level to have good DX. We think a higher level method to check whether a color is within the range of the output device would be preferable.
2. The Canvas APIs do not seem like the right place to specify primitives for querying the volume of the output device, which are useful in APIs across the entire web platform.
3. While we appreciated the discussion on volume mapping, we still had questions about the specifics. We see several cases where gamut/volume/tone mapping come into play, and we’d love some more detail on how these are handled (actually, the same questions apply to wider/smaller gamuts too):
   1. HDR image painted on a non-HDR canvas
   2. HDR canvas displayed non-HDR screen
   3. Author painting an HDR CSS color on a non-HDR canvas
4. The used color space is a property of CanvasRenderingContext2D, but HTMLCanvasElement provides methods for exporting the canvas contents as an image (toBlob(), toDataURL()). Is the color space preserved during these operations? (again this also applies to wide gamut)

And some more minor comments:

• Since CSS Color HDR has been approved as a workstream by the CSS WG, this work should probably reference it in the same way it references css-color-4.
• screen.screenColorInfo seems a little repetitive, why not screen.colorInfo?

[palemieux]

The part of the API around checking whether a color is in within the volume of the output device

@LeaVerou Do you mean this part of the API.

If so, how is it too close to the hardware? Note the API does not require the returned values to match the hardware exactly, and may reflect user preferences/OS limitations/environmental conditions/etc.

That part of the API is intended for advanced applications. The vast majority of applications are expected to merely set the Canvas to an HDR colorspace and write pixels in that colorspace, letting the UA/OS/Device render the HDR pixels as appropriate based on current conditions.

[palemieux]

FYI. Comments above have been entered at https://github.com/w3c/ColorWeb-CG/issues?q=is%3Aopen+is%3Aissue+label%3Atag-comment.

[jyasskin]

A piece of this is shipping in https://groups.google.com/a/chromium.org/g/blink-dev/c/rBQIRHUEAe8/m/JvL612SvBQAJ

[svgeesus]

@palemieux wrote:

FYI. Comments above have been entered at https://github.com/w3c/ColorWeb-CG/issues?q=is%3Aopen+is%3Aissue+label%3Atag-comment.

Just to clarify that the Color CG is awaiting answers to the clarifying questions in those issues. It would also be helpful if the TAG could update the assignees for this issue (given the recent elections)

[martinthomson]

We are happy to see that CSS and Canvas and other spaces are moving toward getting better color support (w3ctag/gaps#7). It appears as though there is a strong effort to make this consistent from a usage perspective, which is good. We aren't experts in this area, but we are happy to resolve this one.

@LeaVerou's questions include a few things that we're not sure about (including whether they have been answered satisfactorily). The remaining members of the TAG don't know enough to pursue that line of questioning, and we think the members of the CG are probably the best placed to decide whether the spec is good enough in those areas. If there are any specific questions that you think we need to address, please let us know.