Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add color_curve RGBGFX test #1554

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Add color_curve RGBGFX test #1554

wants to merge 3 commits into from

Conversation

Rangi42
Copy link
Contributor

@Rangi42 Rangi42 commented Oct 27, 2024

Follow-up to #1553. Adds a test for -C/--color-curve support, which is breaking somehow on 32-bit MinGW.

color_curve.out.pal result.pal differ: char 1, line 1
00:0000 
< 00:0000: ec1f 0516 0748 0000 ff03 e03b 0c7f 0516  .....H.....;....
< 00:0010: ff03 ec1f e075 0000 ff03 ec1f 0748 0000  .....u.......H..
< 00:0020: ff03 1f57 0c7f e075 ff03 3a77 1f57 0c7f  ...W...u..:w.W..
< 00:0030: 9f02 e075 2e09 0000 9f02 2e09 0748 0000  ...u.........H..
---
00:0000 
> 00:0000: ff03 ec1f e075 0000 ff03 e03b 0c7f 0516  .....u.....;....
> 00:0010: ff03 3a77 1f57 0c7f ff03 1f57 0c7f e075  ..:w.W.....W...u
> 00:0020: 9f02 2e09 0748 0000 9f02 e075 2e09 0000  .....H.....u....
> 00:0030: ff03 ec1f 0748 0000 ec1f 0516 0748 0000  .....H.......H..

(The palette order changes from 0 1 2 3 4 5 6 7 to 2 1 5 4 7 6 3 0.)

@Rangi42 Rangi42 added bug Unexpected behavior / crashes; to be fixed ASAP! tests This affects the test suite rgbgfx This affects RGBGFX labels Oct 27, 2024
@Rangi42 Rangi42 added this to the v0.9.0 milestone Oct 27, 2024
@Rangi42 Rangi42 requested a review from ISSOtm October 27, 2024 15:35
@Rangi42 Rangi42 changed the title Add more RGBGFX tests Add color_curve RGBGFX test Oct 28, 2024
@Rangi42 Rangi42 mentioned this pull request Nov 3, 2024
2 tasks
@Rangi42
Copy link
Contributor Author

Rangi42 commented Nov 23, 2024

Verbose logging (-vvvv) shows mismatched "Rel size" values between 32-bit MinGW and every other platform.

Correct log (most platforms): https://pastebin.com/EffXYRg9
Incorrect log (32-bit MinGW): https://pastebin.com/WFMW4S0X

	/*
	 * Computes the "relative size" of a proto-palette on this palette
	 */
	double relSizeOf(ProtoPalette const &protoPal) const {
		// NOTE: this function must not call `uniqueColors`, or one of its callers will break!
		double relSize = 0.;
		for (uint16_t color : protoPal) {
			auto n = std::count_if(RANGE(*this), [this, &color](ProtoPalAttrs const &attrs) {
				ProtoPalette const &pal = (*_protoPals)[attrs.protoPalIndex];
				return std::find(RANGE(pal), color) != pal.end();
			});
			// NOTE: The paper and the associated code disagree on this: the code has
			// this `1 +`, whereas the paper does not; its lack causes a division by 0
			// if the symbol is not found anywhere, so I'm assuming the paper is wrong.
			relSize += 1. / (1 + n);
		}
		return relSize;
	}

Note that I have previously rewritten this function to not depend on std::count_if or std::find, without affecting either logged output.

@ISSOtm
Copy link
Member

ISSOtm commented Nov 26, 2024

I added some more logging, and it seems like the difference stems from std::minmax_element breaking a tie between two equal-efficiency proto-palettes differently. (Strangely, it appears to use a more efficient algorithm than glibc..? It does fewer comparisons in this case, at least.)

@Rangi42
Copy link
Contributor Author

Rangi42 commented Nov 26, 2024

it seems like the difference stems from std::minmax_element breaking a tie between two equal-efficiency proto-palettes differently

Odd, according to cppreference.com "If several elements are equivalent to the smallest element, the iterator to the first such element is returned. If several elements are equivalent to the largest element, the iterator to the last such element is returned." So this may be a bug with mingw32's implementation.

(I also could have sword I reimplemented this without minmax_element already and didn't fix the bug... oh well, will try again.)

@ISSOtm
Copy link
Member

ISSOtm commented Nov 26, 2024

🗡️

Turns out that efficiency is computed slightly differently; like, post-millionth discrepancy. I haven't disassembled the code to figure out why.

Cc @aaaaaa123456789 as the resident floats expert. Possibly we'd make an equivalent comparison using only int ops..? (Multiplying by the respective denominators instead of employing division)

@ISSOtm
Copy link
Member

ISSOtm commented Nov 26, 2024

I turned a pair of divisions into a pair of multiplications, and this test is now consistent across all platforms. (Amusingly, it does not yield the same order as before! But result.pal is consistently a palette permutation of color_curve.out.pal.)

There is nonetheless still some discrepancy across platforms:

Screenshot of differences between platforms' generated logs

This could be fixed by eliminating double entirely from the process; it's currently still emitted by relSizeOf, but ax6 pointed out that this can be scaled up by the LCM of all its expected denominator values (for 2bpp: lcm(1..5) = 60, for 4bpp: lcm(1..17) = 12252240; keeping in mind that the denominator is n + 1, not n), and the resulting products (a×d < c×b) computed in u64's.

This is a little more involved, and more obscure. Should we implement this?

@Rangi42
Copy link
Contributor Author

Rangi42 commented Nov 26, 2024

Thanks so much! ❤️ I'll clean up this PR and merge (and maybe try out the integer-only approach).

@Rangi42
Copy link
Contributor Author

Rangi42 commented Nov 27, 2024

Amusingly, it does not yield the same order as before!

Turns out this was a bug with the < .001 threshold comparison; fixing that makes Mingw32's result.pal agree with what the other platforms already had. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Unexpected behavior / crashes; to be fixed ASAP! rgbgfx This affects RGBGFX tests This affects the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants