Add CanvasColorType, with unorm8 (the default) and float16 support

[ccameron-chromium]

The default behavior is unorm8, which matches the existing behavior of
all user agents.

The name unorm8 is chosen to match modern APIs which distinguish between
8-bit unsigned normalized and 8-bit unsigned integer (e.g,
VK_FORMAT_R8G8B8A8_UNORM vs VK_FORMAT_R8G8B8A8_UINT in Vulkan,
MTLPixelFormatRGBA8Unorm vs MTLPixelFormatRGBA8Uint in Metal, and
"rgba8unorm" vs "rgba8uint" in WebGPU).

The use of colorType (instead of pixelFormat) is used to avoid the
complexity of selecting channel ordering (BGRA vs RGBA) and interactions
with the alpha parameter (RGB, RGBX, or BGRX potentially being required if
alpha is false).

• At least two implementers are interested (and none opposed):
  • Chromium (author)
  • Canvas Floating Point Color Types mozilla/standards-positions#708
  • Canvas Floating Point Color Types WebKit/standards-positions#91
• Tests are written and can be reviewed and commented upon at:
  • https://chromium-review.googlesource.com/c/chromium/src/+/6235086
• Implementation bugs are filed:
  • Chromium: crbug.com/40277213
  • Gecko: https://bugzilla.mozilla.org/show_bug.cgi?id=1946355
  • WebKit: https://bugs.webkit.org/show_bug.cgi?id=287139
• MDN issue is filed:
  • Add colorType documentation to CanvasRenderingContext2DSettings mdn/content#38087
• The top of this comment includes a clear commit message to use.

---

/canvas.html ( diff )

[annevk]

I guess this can in theory be isolated from the ImageData changes, but is that useful or even observable (apart from this API itself)?

[ccameron-chromium]

I guess this can in theory be isolated from the ImageData changes, but is that useful or even observable (apart from this API itself)?

This actually hits lots of testable paths before adding ImageData support. Canvases that are float16 should be able to round-trip data from a different color space without a loss of precision (including round-tripping through PNG encoding).

This patch is the proto-WPT test that I'm working against. E.g, if you write an sRGB image to a float16-P3 canvas, then read it back as sRGB, you shouldn't lose any code points. But in our implementation you do... BUT ONLY if you read back as 8-bit (if you read back as float16, then the bug is side-stepped). The fact that there's so much to be gotten right in this area alone is part of why I chopped it into a separate commit.

[annevk]

cc @whatwg/canvas

[ccameron-chromium]

Are there any further issues with the the PR itself?

If this is all in line, then I'll file implementation and MDN bugs.

[Kaiido]

Nit: The end section about alpha-premult might need an update. I suppose the following sentence doesn't entirely hold anymore:

[...] and division and multiplication on 8-bit integers (which is how canvas's colors are currently stored) [...]

[ccameron-chromium]

Nit: The end section about alpha-premult might need an update. I suppose the following sentence doesn't entirely hold anymore:

[...] and division and multiplication on 8-bit integers (which is how canvas's colors are currently stored) entails a loss of precision [...]

Thanks for finding that! I've change it to read "division and multiplication using finite precision entails a loss of accuracy". This also fixes a nit where what is lost is accuracy (not precision).

[foolip]

I've done editorial review and this all looks to be in good order. I understand there's some additional bits to come, but this chunk looks testable and more importantly usable+useful.

[foolip]

Note to other reviewers. The use of the variable type here outside of an algorithm looks wrong at first, but all of the paragraphs in this section are specifying requirements on "a serialization of the bitmap as a file" and there are a few other uses of type. It seems OK to leave this style alone I think.

[annevk]

This needs to update getContextAttributes() as well.

I don't really like that we don't really have anything about a processing model here. It's fairly unusual to introduce enum members in this way as well. Normally enum members are defined in IDL and the processing model makes it clear what kind of impact they have. I guess this matches the colorSpace prior art, but at some point we'll have to refactor all of this to instead have proper processing models and that will be quite some work.

[ccameron-chromium]

This needs to update getContextAttributes() as well.

Thanks, updated this (and will include it in the WPT tests).

[annevk]

Can you please file the MDN issue?

I'm not quite sure how to deal with the WebKit standards-positions issue as this is only a subset of the whole idea, but I confirmed with colleagues that we are okay with this specific PR so in terms of implementer interest this should be fine.

[ccameron-chromium]

Can you please file the MDN issue?

Done, updated the issue description.

I'm not quite sure how to deal with the WebKit standards-positions issue as this is only a subset of the whole idea, but I confirmed with colleagues that we are okay with this specific PR so in terms of implementer interest this should be fine.

Thanks!