'Storing a positive floating point number in a known range in two bytes
I know that a number x lies between n and f (f > n > 0). So my idea is to bring that range to [0, 0.65535] by 0.65535 * (x - n) / (f - n).
Then I just could multiply by 10000, round and store integer in two bytes.
Is it going to be an effective use of storage in terms of precision?
I'm doing it for a WebGL1.0 shader, so I'd like to have simple encoding/decoding math, I don't have access to bitwise operations.
Solution 1:[1]
Why multiply by 0.65535 and then by 10000.0? That introduces a second rounding with an unnecessary loss of precision.
The data will be represented well if it has equal likelihood over the entire range (f,n). But this is not always a reasonable assumption. What you're doing is similar to creating a fixed-point representation (fixed step size, just not starting at 0 or with steps that are negative powers of 2).
Floating-point numbers use bigger step sizes for bigger numbers. You could do the same by calculating log(x/f) / log(n/f) * 65535
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | MSalters |
