-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Contempt 20 #1366
Contempt 20 #1366
Conversation
Set the default contempt value of Stockfish to 20 centipawns. The contempt feature of Stockfish tries to prevent the engine from simplifying the position too quickly when it feels that it is very slightly behind, instead keeping the tension a little bit longer. Various tests in November 2017 have proved that our current imple- mentation works well against SF7 (which is about 130 Elo weaker than current master) and than the Elo gain is an increasing function of contempt, going (against SF7) from +0 Elo when contempt is set at zero centipawns, to +30 Elo when contempt is 40 centipawns. See pull request 1325 for details: official-stockfish/Stockfish#1325 This november discussion left open the decision of which "default" value for contempt we should use for Stockfish, taking into account the various uses ofStockfish (opening preparation for humans, computer online tournaments,analysis tool for web pages, human/computer play, etc). This pull request proposes to set the default contempt value of SF to twenty centipawns, which turns out to be the highest value which is not a regression against current master, as this seemed to be a good compromise between risk and safety. A couple of SPRT[-3..1] tests were done to bisect this value: Contempt 10: http://tests.stockfishchess.org/tests/view/5a5d42d20ebc5902977e2901 (PASSED) Contempt 15: http://tests.stockfishchess.org/tests/view/5a5d41740ebc5902977e28fa (PASSED) Contempt 20: http://tests.stockfishchess.org/tests/view/5a5d42060ebc5902977e28fc (PASSED) Contempt 25: http://tests.stockfishchess.org/tests/view/5a5d433f0ebc5902977e2904 (FAILED) Surprisingly, a test at "very long time control" hinted that using contempt 20 is not only be non-regressive against contempt 0, but may actually exhibit some small Elo gain, giving a likehood of superio- rity of 88.7% after 8500 games: VLTC: ELO: 2.28 +-3.7 (95%) LOS: 88.7% Total: 8521 W: 1096 L: 1040 D: 6385 http://tests.stockfishchess.org/tests/view/5a60b2820ebc590297b9b7e0 Finally, there was some concerns that a contempt value of 20 would be worse than a value of 7, but a test with 20000 games at STC was neutral: STC: ELO: 0.45 +-3.1 (95%) LOS: 61.2% Total: 20000 W: 4222 L: 4196 D: 11582 http://tests.stockfishchess.org/tests/view/5a64d2fd0ebc590297903868 See the comments in pull request 1361 for the long, nice discussion (180 entries :-)) leading to the decision to propose contempt 20 as the default value: official-stockfish/Stockfish#1361 Whether Stockfish should strictly adhere to the Komodo and Houdini semantics and add the UCI commands to force the contempt to be White in the so-called "analysis mode" is still under discussion, and may be or may not be the object of a future commit. Bench: 5571216
Very nice and well put. Hope SF9's release is near. |
Merged with 254d995 Congratulations! Impressive work and impressive log message. Very high quality stuff here :-) |
Thank you Marco..looks like my testings shows again right ;) |
@mcostalba, it was too early to commit this patch. @syzygy1 has correctly pointed out that the current implementation of contempt isn't suitable for analysis. |
@atumanian the correct line is that user should set contempt zero for analysis. Please note that this is true for any engine at any time, and (to me) it is acceptable to think this fact is in the knwoledge of users that usually do chess analysis. The new fact here is that before contempt was at zero by default now it should be explicitly set. |
@mcostalba, thank you for your reply. I've recently created a pull request #1369. We can discuss the issue there. |
Set the default contempt value of Stockfish to 20 centipawns.
The so-called contempt feature of Stockfish tries to prevent the
engine from simplifying the position too quickly when it feels that it
is very slightly behind, instead keeping the tension a little bit longer.
Various tests in November 2017 have proved that our current imple-
mentation works well against SF7 (which is about 130 Elo weaker than
current master) and that the Elo gain is an increasing function of
contempt, going (against SF7) from +0 Elo when contempt is set at
zero centipawns, to +32 Elo when contempt is 40 centipawns.
See pull request 1325 for details:
#1325
This November discussion left open the decision of which "default"
value for contempt we should use for Stockfish, taking into account
the various uses of Stockfish (opening preparation for humans, computer
online tournaments, analysis tool for web pages, human/computer play,
etc).
This pull request proposes to set the default contempt value of SF
to twenty centipawns, which turns out to be the highest value which
is not a regression against current master, as this seemed to be a
good compromise between risk and safety. A couple of SPRT[-3..1]
tests at long time control were done to bisect this value:
LTC:
Contempt 10: http://tests.stockfishchess.org/tests/view/5a5d42d20ebc5902977e2901 (PASSED)
Contempt 15: http://tests.stockfishchess.org/tests/view/5a5d41740ebc5902977e28fa (PASSED)
Contempt 20: http://tests.stockfishchess.org/tests/view/5a5d42060ebc5902977e28fc (PASSED)
Contempt 25: http://tests.stockfishchess.org/tests/view/5a5d433f0ebc5902977e2904 (FAILED)
Surprisingly, a test at "very long time control" hinted that using
contempt 20 would not only be non-regressive against contempt 0, but
may actually exhibit some small Elo gain (+2.4 Elo), giving a likelihood
of superiority of 90% after 8500 games:
VLTC:
ELO: 2.44 +-3.7 (95%) LOS: 90.2%
Total: 8546 W: 1104 L: 1044 D: 6398
http://tests.stockfishchess.org/tests/view/5a60b2820ebc590297b9b7e0
Finally, there were some concerns that a contempt value of 20 would
be worse than a value of 7, but a test with 20000 games at STC was
neutral:
STC:
ELO: 0.45 +-3.1 (95%) LOS: 61.2%
Total: 20000 W: 4222 L: 4196 D: 11582
http://tests.stockfishchess.org/tests/view/5a64d2fd0ebc590297903868
See the comments in pull request 1361 for the long, nice discussion
(180 entries :-)) leading to the decision to propose contempt 20 as
the default value:
#1361
Whether Stockfish should strictly adhere to the Komodo and Houdini
semantics, adding the UCI commands to force the contempt to be White
in the so-called "analysis mode" is still under discussion, and may be or
may not be the object of a future commit.
Bench: 5571216