CMYK jpgs are not decoded correctly

[bohning]

Actual behaviour

Use e.g. this file as a cover, compare how it looks in the browser to how it is displayed in USDX (colors are wrong). Of course, a workaround is to convert to RGB jpg, but I thought this may be easily fixed.

Expected behaviour

USDX should handle color information correctly.

Details

Probably any version of USDX, but this was noticed in 2024.1.0.

[s09bQ5]

If SDL_image sees that a JPEG has four color components, it requests the data to be decoded as CMYK. But since there is no SDL pixel format for CMYK, it allocates an RGBA surface.

https://github.com/libsdl-org/SDL_image/blob/main/src/IMG_jpg.c#L382

[bohning]

So if I understand correctly, the issue is upstream in SDL_image?

[hoehermann]

There is no support for CMYK images in SDL, see https://wiki.libsdl.org/SDL2/SDL_PixelFormatEnum. The decoder does a "best effort" by presenting the data, leaving the CMYK → RGB conversion up to the application. Since CMYK is designed for printing and not screen presentation, I think implementing the conversion is out of scope for USDX (or SDL, for that matter). Also, CMYK JPEGs are quite rare, so we are better off if users manually convert the occasional CMYK JPEG to RGB for use with USDX. :)

[s09bQ5]

The decoder does a "best effort" by presenting the data, leaving the CMYK → RGB conversion up to the application.

SDL_image explicitly tells libjpeg to not convert the CMYK to RGB.

[s09bQ5]

Libjpeg has no code to convert 4 component images to RGB. But that is actually not necessary since CMYK JPEGs store the image with inverted color components, which makes them RGBA. A 255/0/0/255 pixel has no cyan filtering paint, lots of magenta filtering paint, lots of yellow filtering paint and no black, which combines to red. The problem we have is that SDL2_image uses a BGRA32 surface instead of an RGBA32 surface.

libsdl-org/SDL_image#410

There is also a YCCK JPEG format and SDL2_image tells libjpeg to convert it to CMYK during decoding. The conversion is done correctly, but libjpeg will not use the inverted CMYK format. A 0/0/0/0 output pixel is white. Edit: So far the theory. In practice all YCCK JPEGs I found decode as inverted CMYK.