Underflow bugfix & fix off-by-one error in the texture height calculation

[hannes-vernooij]

It should be max index / max size because 4096 / 4096 + 1 = 2, which means that two rows are allocated when a LUT1D with a size of 4096 fits in a 1D texture of 4096 pixels.

EDIT:
In the initial commit I made the mistake of not taking into account that the last value of a row is repeated in the first value of the next row, this is now fixed. I validated the result by running a shaper LUT with a size of 4096, 4097, 8191 and 8192. I then applied the transform LUT on syntheticChart.01.exr and validated a 0 pixel difference.

Underflow bugfix

This also seems to solve an existing underflow bug that appeared with a 1D LUT with a size of 8191. When the height is calculated the value is not ceiled, which results in 8191 / 4096 = 2, while it should be 3.

This caused an underflow later in the code in:
unsigned long missingEntries = width * height - (unsigned long)paddedChannel.size();

where width * height is less than (unsigned long)paddedChannel.size(); causing an underflow that results in a very high value that is iterated.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: hannes-vernooij / name: Hannes Vernooij (1cc7582, 9005a12)