Codec for LogScale

[gauteh]

Hi,

We're thinking about how to improve compression of values of ocean models with large value spans. We would like to encode them using a log-scale/ln-scale. Would it be interesting to add official support for a LogScale(base, offset,etc..) codec?

Regards, Gaute

[ehgus]

I don't know the details of the ocean model, but the floating-point number expression seems reasonable in your case.
Floating point separately saves exponents and fractions according to IEEE 754.
By doing so, significant digits are saved even for large numbers.
I think you can achieve the similar thing using less precise floating point expression.

[gauteh]

That's a very good point. The next question is whether float16 is supported?

[ehgus]

Numpy support np.float16 and try using AsType.

import numpy as np
from numcodecs import AsType
x = np.arange(100,120,2,dtype = np.float32)
f = AsType(encode_dtype=np.float16, decode_dtype=np.float32)
y = f.encode(x)
print(y, y.dtype)
# [100. 102. 104. 106. 108. 110. 112. 114. 116. 118.] float16

[ehgus]

By the way, a similar issue was raised in #237.