Olympus white level

[paolodepetrillo]

I think there is a problem with the white level for Olympus raws.

It gets set here

rawspeed/src/librawspeed/decoders/OrfDecoder.cpp

Line 278 in e5cbf27

mRaw->whitePoint -= (mRaw->blackLevel - mRaw->blackLevelSeparate[0]);

which was originally added in this commit from 2011 98420bf

It starts with the black and white levels from cameras.xml, replaces the black level with the values from Exif.OlympusIp.BlackLevel tag, and then adjusts the white level so that the difference between the black and white level is the same as it was in cameras.xml. The problem is that this often sets the white level greater than 4094, so for a 12 bit raw it appears to darktable as if there are no clipped pixels at all.

There is a proprietary tag Exif.OlympusIp.0x0805 which may contain the white level:
https://github.com/exiftool/exiftool/blob/5c2467fa6cdb38233793884e80cee9abf4da48e6/lib/Image/ExifTool/Olympus.pm#L2907
I haven't investigated extensively but checked some images from my E-P5 and other orfs on RPU and it always seems to have a reasonable value, never greater than 4094. The value does vary based on camera settings.

[pitbuster]

Oooh, I have some old ORF files that I've noticed have some weird stuff happening with their white point.

What would be the best course of action? I don't think RPU samples have clipped highlights and probably the white point varies by ISO. Should we ask for samples for every camera/ISO combination?

[LebedevRI]

That compensation does look fishy, The commit did not state from which camera the sample came from,
so it's a bit impossible to re-investigate (i assume there was more than one supported Olympus camera at the time?).

I'd guess someone needs to contribute such a long-exposure sample from some Olympus camera to RPU,
i don't recall seeing one there. And then i guess we should just try if reading that tag (Exif.OlympusIp.0x0805)
always produces viable white level.

Thanks.

[paolodepetrillo]

LibRaw uses that 0x0805 tag for the white level: https://github.com/LibRaw/LibRaw/blob/6fffd414bfda63dfef2276ae07f7ca36660b8888/src/metadata/olympus.cpp#L404

Checking the Olympus samples from RPU, it doesn't ever seem to have an obviously way off value. Sometimes it is lower than the brightest pixel in the image, for example in Olympus - E-P2 - 16bit (4:3).ORF the 0x0805 tag is "3643 324" and the max pixel in the raw image is 4091. In most cases the difference is not that large though.

What kind of sample images would be needed to confirm that it really is a valid white level?

[LebedevRI]

What kind of sample images would be needed to confirm that it really is a valid white level?

Perhaps a somewhat overexposed, with blown highlights, long exposure one?

[paolodepetrillo]

For the E-M1 II, the 0x0805 tag is always "4095 206" in all cases - I can't find any mode or setting that has any effect on it. For the E-P5 it seems to depend on ISO - changes from 4094 or 4095 to 4036 at lower ISO.

Adobe DNG Converter interprets some other Olympus proprietary tags (like distortion and CA) but it seems to ignore the 0x0805 tag. For every image from my E-M1 II and E-P5 in any mode, and for all the Olympus samples on RPU from cameras made around 2012 or 2013 and newer, it always uses 4000 as the white level - must be hardcoded. It uses a different value for some older cameras, not sure where those values are coming from. It always uses the black levels from the Exif.Olympus.BlackLevel tag.