Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

do not change cursed wins to draws #5805

Conversation

robertnurnberg
Copy link
Contributor

This fixes an oversight introduced in #5414.

The problem is that is_draw() has no knowledge of the UCI option Syzygy50MoveRule, and so its calls have to be guarded in syzygy_extend_pv().

An example position where the bug triggers is 7b/8/6b1/8/5Nk1/4K3/8/8 b - - 0 1.

A review by @vondele would be good.

No functional change.

Copy link

clang-format 18 needs to be run on this PR.
If you do not have clang-format installed, the maintainer will run it when merging.
For the exact version please see https://packages.ubuntu.com/noble/clang-format-18.

(execution 12855041177 / attempt 1)

@robertnurnberg
Copy link
Contributor Author

Master:

setoption name SyzygyPath value /disk1/syzygy/3-4-5-6/WDL:/disk2/syzygy/3-4-5-6/DTZ
setoption name Syzygy50MoveRule value false
position fen 7b/8/6b1/8/5Nk1/4K3/8/8 b - - 0 1
go depth 1
info string Found 510 WDL and 510 DTZ tablebase files (up to 6-man).
info string Available processors: 0-23
info string Using 1 thread
info string info string NNUE evaluation using nn-1c0000000000.nnue (133MiB, (22528, 3072, 15, 32, 1))
info string info string NNUE evaluation using nn-37f18f62d772.nnue (6MiB, (22528, 128, 15, 32, 1))
info depth 1 seldepth 2 multipv 1 score cp 0 nodes 2 nps 90 hashfull 0 tbhits 20 time 22 pv g6h7 f4d5 h8e5 d5b4 e5d6 b4a6 d6h2 e3d4 g4f5 a6c5 h2g1 d4d5 h7g8 d5d6 g1h2 d6c6 f5e5 c5d3 e5d4 d3c5 g8d5 c6b6 d4c4 c5b7 h2g1 b6c7 g1f2 b7d6 c4b4 d6e8 f2g3 c7d7 g3e5 d7e7 d5a2 e8f6 e5b2 f6d7 b2c3 e7d6 b4b5 d7e5 c3b4 d6d7 a2d5 e5d3 b4a3 d3f4 d5c4 f4g6 b5c5 g6e7 a3b2 e7g6 b2c3 g6f4 c5d4 f4e6 d4e5 e6c7 c4f7 c7b5 c3b4 b5c7 f7h5 c7b5 b4f8 b5c7 f8h6 c7b5 h6e3 b5c7 h5d1 c7a6 d1a4 d7c7 a4e8 a6b4 e3d2 b4a6 d2a5 c7b7 e5d6 a6b8 a5c7 b8a6 e8c6 b7a7 c7d8 a7b8 d8b6 a6b4 c6e4 b4a2 b6e3 a2c3 e4g2 c3b5 d6c5 b5a3
bestmove g6h7 ponder f4d5

Patch:

setoption name SyzygyPath value /disk1/syzygy/3-4-5-6/WDL:/disk2/syzygy/3-4-5-6/DTZ
setoption name Syzygy50MoveRule value false
position fen 7b/8/6b1/8/5Nk1/4K3/8/8 b - - 0 1
go depth 1
info string Found 510 WDL and 510 DTZ tablebase files (up to 6-man).
info string Available processors: 0-23
info string Using 1 thread
info string info string NNUE evaluation using nn-1c0000000000.nnue (133MiB, (22528, 3072, 15, 32, 1))
info string info string NNUE evaluation using nn-37f18f62d772.nnue (6MiB, (22528, 128, 15, 32, 1))
info depth 1 seldepth 2 multipv 1 score cp 20000 nodes 2 nps 86 hashfull 0 tbhits 20 time 23 pv g6h7 f4d5 h8e5 d5b4 e5d6 b4a6 d6h2 e3d4 g4f5 a6c5 h2g1 d4d5 h7g8 d5d6 g1h2 d6c6 f5e5 c5d3 e5d4 d3c5 g8d5 c6b6 d4c4 c5b7 h2g1 b6c7 g1f2 b7d6 c4b4 d6e8 f2g3 c7d7 g3e5 d7e7 d5a2 e8f6 e5b2 f6d7 b2c3 e7d6 b4b5 d7e5 c3b4 d6d7 a2d5 e5d3 b4a3 d3f4 d5c4 f4g6 b5c5 g6e7 a3b2 e7g6 b2c3 g6f4 c5d4 f4e6 d4e5 e6c7 c4f7 c7b5 c3b4 b5c7 f7h5 c7b5 b4f8 b5c7 f8h6 c7b5 h6e3 b5c7 h5d1 c7a6 d1a4 d7c7 a4e8 a6b4 e3d2 b4a6 d2a5 c7b7 e5d6 a6b8 a5c7 b8a6 e8c6 b7a7 c7d8 a7b8 d8b6 a6b4 c6e4 b4a2 b6e3 a2c3 e4g2 c3b5 d6c5 b5a3 e3f4 b8a7 g2h1 a3c2 c5b5 c2a3 b5a5 a3c4 a5b5 c4a3
bestmove g6h7 ponder f4d5

@robertnurnberg
Copy link
Contributor Author

Passed the matetrack PV verification cleanly:

> python matecheck.py --epdFile matetrack.epd ../matetools/mate-in-2.epd --engine ../stockfish/src/patch --nodes 10000 --syzygyPath /disk1/syzygy/3-4-5-6/WDL:/disk2/syzygy/3-4-5-6/DTZ
Loaded 12869 FENs, with max(|bm|) = 126.

Matetrack started for ../stockfish/src/patch on matetrack.epd ../matetools/mate-in-2.epd with --nodes 10000 --syzygyPath /disk1/syzygy/3-4-5-6/WDL:/disk2/syzygy/3-4-5-6/DTZ ...
100%|###########################################| 97/97 [10:46<00:00,  6.67s/it]

Found 510 tablebases. Checking 10560 TB win PVs. This may take some time ...

Using ../stockfish/src/patch on matetrack.epd ../matetools/mate-in-2.epd with --nodes 10000 --syzygyPath /disk1/syzygy/3-4-5-6/WDL:/disk2/syzygy/3-4-5-6/DTZ
Engine ID:     Stockfish dev-20250119-5b0482ba
Total FENs:    12869
Found mates:   6102
Best mates:    5976
Found TB wins: 957

@robertnurnberg
Copy link
Contributor Author

As reported in vondele/matetrack#130 (comment) this patch seems to uncover a subtle issue with the pv extension in the absence of the 50mr. For example:

Found TB score -20000 with PV status "draw wdl = 0 at ply = 100 for 8/8/8/3K4/8/1k6/8/nB2B3 b - -" for FEN "K7/8/7B/8/4k3/8/6n1/7B b - -" with bm #-1.
PV: e4f3 h6g7 f3g3 g7e5 g3f3 e5h2 f3f2 a8a7 f2f1 a7b6 f1f2 b6c5 f2f1 c5d5 g2e3 d5e6 e3g2 e6f5 f1f2 f5g4 g2e3 g4h3 e3c4 h1g2 c4e3 g2e4 e3d1 e4b7 d1e3 h2e5 e3d1 e5f6 f2e2 h3g3 e2d3 g3f4 d3c4 b7a6 c4b3 a6d3 d1f2 d3e2 b3c2 e2h5 f2d3 f4e3 d3b2 h5g6 c2b3 f6d4 b2c4 e3e2 c4a5 g6f7 b3b4 f7d5 b4b5 d4g1 a5c6 d5g2 c6b4 g2f3 b5c4 e2d2 b4d5 f3e2 c4b4 g1d4 d5f4 e2f1 f4e6 d4e3 e6c7 d2d3 c7b5 f1e2 b5c7 e2f3 c7e6 d3e4 e6c5 e4d5 c5b3 e3b6 b3d2 f3e2 d2b3 b6d8 b3d2 e2a6 d2b3 a6c8 b3d2 c8f5 d2b3 d8h4 b3a1 h4e1 b4b3 f5b1 b3a4 d5c4 a1b3 b1c2 a4a3 c2b3 a3b2 e1d2 b2b1 c4c3 b1a1 d2e3 a1b1 b3c4 b1a1 c3b3 a1b1 c4d3 b1a1 e3d4

So a TBloss score at root comes with a PV that after 50 moves gives a drawn position.

I believe this may have to do with the rank calculation in Tablebases::root_probe(), but I would prefer not to touch that code a part of this PR.

So I would suggest to merge this PR as a bug-fix, and then deal with the incorrect PVs in a follow-up.

@@ -1989,7 +1990,7 @@ void syzygy_extend_pv(const OptionsMap& options,
pos.do_move(pvMove, st);

// Do not allow for repetitions or drawing moves along the PV in TB regime
if (config.rootInTB && pos.is_draw(ply))
if (config.rootInTB && ((rule50 && pos.is_draw(ply)) || pos.has_repeated()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pos.is_draw(ply) and pos.has_repeated() are different in how they flag 2 and 3 fold repetitions. However, it might be that pos.has_repeated() but that needs some motivation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I tried minimal changes at first, and did not want to add a new function to the code base at this stage. We may eventually have to, but I do not know.

@@ -2008,7 +2009,7 @@ void syzygy_extend_pv(const OptionsMap& options,
// Step 2, now extend the PV to mate, as if the user explored syzygy-tables.info
// using top ranked moves (minimal DTZ), which gives optimal mates only for simple
// endgames e.g. KRvK.
while (!pos.is_draw(0))
while (!((rule50 && pos.is_draw(0)) || pos.has_repeated()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could be similar.

@@ -2061,7 +2062,7 @@ void syzygy_extend_pv(const OptionsMap& options,
// We adjust the score to match the found PV. Note that a TB loss score can be
// displayed if the engine did not find a drawing move yet, but eventually search
// will figure it out (e.g. 1kq5/q2r4/5K2/8/8/8/8/7Q w - - 96 1 )
if (pos.is_draw(0))
if (rule50 && pos.is_draw(0))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also here not necessarily equivalent with 3-folds, not sure we can have that case.

Copy link
Contributor Author

@robertnurnberg robertnurnberg Jan 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since pos.is_draw(0) would flag any position after 50moves without capture or pawn move as draw, we have to guard this here by rule50 to avoid cursed wins to be changed to draws. I believe this is the only place that is strictly necessary to fix the bug. The other changes try to maintain the nice PV extension also for !rule50, without success so far it seems.

@vondele
Copy link
Member

vondele commented Jan 20, 2025

It would be good to get the correct PVs as well of course. Might need some additional thinking about what is causing the issue.

@robertnurnberg
Copy link
Contributor Author

robertnurnberg commented Jan 20, 2025

It would be good to get the correct PVs as well of course. Might need some additional thinking about what is causing the issue.

Agreed. It seems to me by now that this PR only deals with a symptom, whereas the cause are the incorrect PVs. I.e. somehow when 50mr is off, the extension includes positions in the PV that are TB draws. And those incorrect extensions then lead to the score change at root.
Edit: In master we actually falsely flag any PV going over the 50mr as draw, so that bug is real, and is fixed by this PR.

I suspect the reason that the extended PVs can have drawn positions is somehow hidden in here:

        // Better moves are ranked higher. Certain wins are ranked equally.
        // Losing moves are ranked equally unless a 50-move draw is in sight.
        int r    = dtz > 0 ? (dtz + cnt50 <= 99 && !rep ? MAX_DTZ - (rankDTZ ? dtz : 0)
                                                        : MAX_DTZ / 2 - (dtz + cnt50))
                 : dtz < 0 ? (-dtz * 2 + cnt50 < 100 ? -MAX_DTZ - (rankDTZ ? dtz : 0)
                                                     : -MAX_DTZ / 2 + (-dtz + cnt50))
                           : 0;

But I do not see it. :)

@robertnurnberg
Copy link
Contributor Author

Closing in favour of #5814.

@robertnurnberg robertnurnberg deleted the cursed-win-fix branch January 25, 2025 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants