NNUE merge #2823

vondele · 2020-07-25T08:03:35Z

The NNUE branch maintained by @nodchip has demonstrated strong results and offers great potential, and we will proceed to merge it into master. This will assure that Stockfish remains a reference engine based on the top computer chess technology for the foreseeable future. This merge will introduce machine learning based coding to the engine, thus enlarging the community of developers, bringing in new skills. We are eager to keep everybody on board, including all developers and users of diverse hardware, aiming to be an inclusive community. We will keep and evolve the handcrafted evaluation, not only for the beauty and insight it offers, but for additional reasons, including its value on diverse hardware platforms, and for the potential of hybridization with NNUE, and use in search. The initial goal for the merge is the NNUE playing capability only, this focus makes sense given the complexity of the project, and the aim of Stockfish as a production quality chess engine. We will try to make changes such that the @nodchip repo and ours can be kept in sync as easily as possible. Written by knowledgeable people, NNUE is already of good quality, we will make small modifications needed to make it pass our standard CI, and integrate nicely in the existing engine. Rigorous testing will continue to be a cornerstone of our development, and accordingly integration in fishtest needs to be properly implemented. Afterwards, we will guarantee continuous progression in playing strength testing core engine and networks with our usual SPRT process. Some initial steps of the merge will be outlined below, the precise steps needed will become clearer as we proceed, I look forward to working with the community to make this happen!

vondele · 2020-07-25T08:06:38Z

some tasks needed for integration:

create network server with download/upload api
- merge NN upload/download support fishtest#736
- delete now unused repo (https://github.com/official-stockfish/networks)
- make readme documenting process for NNUE testing (https://github.com/glinscott/fishtest/wiki/Creating-my-first-test#nnue-net-tests)
- all nets use a nn-sha256sum[12 digits].bin name for reference from engine
- upload API (https://tests.stockfishchess.org/upload)
- add first net (and more: https://tests.stockfishchess.org/nns)
create nnue-player-wip branch in Stockfish repo, work in this branch till ready for a merge.
- create branch (https://github.com/official-stockfish/Stockfish/tree/nnue-player-wip)
- review code comments throughout
- go through open issues
- remove non-player related code
- properly adjust readme
- clamp abs(NNUE eval) < VALUE_TB_WIN_IN_MAX_PLY
- fix eval / evalnn output
- no compiler warning generated
- fix debug=yes build
- passes the usual CI tests (sanitizers, valgrind, etc) when run by hand
  - verify bench
  - verify perft
  - verify reprosearch
  - instrumented valgrind
  - instrumented valgrind-thread (very slow! (multiple hours))
  - instrumented sanitizer-undefined
  - instrumented sanitizer-thread
- verify equal bench on all (18!) architectures supported (done except for armv7, armv8, ppc-32)
- adjust bench to cover both handcrafted and NNUE evaluation (needs solution for network download as part of CI)
- consider random / synthetic network to perform bench / pgo.
- decide when to init_nnue() (startup, EvalFile, Use NNUE, is_ready, cmdline, ucinewgame, ....), add assert.
- changes in core engine
  - minimize
  - coding style
  - minimize performance impact on handcrafted running style
- UCI default net name matching a net in networks repo.
- complete/maintain AUTHORS file
- integrate in CI
  - deal with std::aligned_alloc
  - perform NNUE testing in CI (needs net download solution)
  - update appveyor / MSCV to use C++17
  - update travis environment to support C++17
  - unfreed memory (fixed with newer toolchain)
  - clang build (needs at least clang 7.0)
  - gcc build (needs at least gcc 7.4)
  - valgrind uninit memory on loading TB
  - valgrind uninit memory
  - memset
  - memcpy
- figure out reason for bench change wrt to nodchip repo (we are up-to-date with master on the branch).
- test performance locally.
- Figure out if we need to update Position objects on a 'Use NNUE' change ([NNUE] Position state / Use NNUE #2859)
- SPRT test classical eval impacted less than 5 Elo on fishtest (looking good after [NNUE] StateInfo handling speed improvement #2873, STC: https://tests.stockfishchess.org/tests/view/5f2906a2a5abc164f05e4c5b LTC: https://tests.stockfishchess.org/tests/view/5f2906cfa5abc164f05e4c5d)
- SPRT test NNUE more than 30 Elo gain on fishtest (skipping SPRT. See >80Elo gain: https://tests.stockfishchess.org/tests/view/5f28fe6ea5abc164f05e4c4c and https://tests.stockfishchess.org/tests/view/5f290229a5abc164f05e4c58)
- Merge master
- squash commits ?
- create PR
update fishtest to support NNUE testing
- communicate / document / update compiler requirements to C++17
- communicate / document removal of custom_make.txt
- download networks from network server as required by binary.
- build code with proper architecture (avx2, bmi2, ?)
- instruct cutechess to do the right thing (i.e. Use NNUE, find network)
- adjust minimal nps for running ?
- decide how to deal with hardware that is slower for nnue.
contact abrok, possible new targets for build (but see https://abrok.eu/stockfish-nnue/)

vondele · 2020-07-25T08:10:45Z

Let's use this issue specifically for synchronizing work on these or related tasks. General discussion use #2728

vondele · 2020-07-25T13:30:28Z

debug=yes build is broken it seems.

vondele · 2020-07-25T14:42:45Z

A quick verification / benchmark (single bench run) on the targets I can test easily (on zen2 architecture), gives a feeling for the order of magnitudes. Good thing is all benches match. If people can test x86-64-avx512 armv7 armv8 ppc-32 ppc-64 that would be appreciated.

target	nodes	nodes NNUE	nps	nps NNUE
general-32	4578298	3377227	1725705	134486
x86-32-old	4578298	3377227	1783520	133497
x86-32	4578298	3377227	1758178	133635
general-64	4578298	3377227	2423662	308000
x86-64	4578298	3377227	2485503	436108
x86-64-modern	4578298	3377227	2611693	424648
x86-64-sse3	4578298	3377227	2470749	1131778
x86-64-sse3-popcnt	4578298	3377227	2675802	1199299
x86-64-ssse3	4578298	3377227	2577870	1166975
x86-64-sse41	4578298	3377227	2445672	1243914
x86-64-sse42	4578298	3377227	2549163	1113126
x86-64-avx2	4578298	3377227	2678933	1635461
x86-64-bmi2	4578298	3377227	1906829	1411884

Edit: since 'x86-64-modern' on master enables both sse3 and popcnt already, we can easily adjust that one (actually, incorrect, needs ssse3 not sse3, the target x86-64-sse3 enables both, increased modern to require ssse3).

dorzechowski · 2020-07-25T15:15:26Z

Fix for debug build: PR #2827.

vondele · 2020-07-25T15:41:56Z

Some more tasks as seen from the CI (patches welcome):

[DONE] appveyor testing needs to add '/std:c++17' somewhere in the right place (needs solution for aligned_alloc with MSCV)
clang: nnue/evaluate_nnue.cpp:60:45: error: no member named 'aligned_alloc' in namespace 'std' (on osx)
[DONE] position.cpp:199:39: error: ‘void* memset(void*, int, size_t)’ clearing an object of non-trivial type ‘struct StateInfo’; use assignment or value-initialization instead [-Werror=class-memaccess]
[DONE] position.cpp:714:51: error: ‘void* memcpy(void*, const void*, size_t)’ writing to an object of a non-trivial type ‘struct StateInfo’ leaves 1272 bytes unchanged [-Werror=class-memaccess]
[DONE] unfreed memory: in use at exit: 72,704 bytes in 1 blocks (possibly only with older version of toolchain)
[DONE] valgrind: uninit memory access: at 0x426526: Position::set(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, bool, bool, StateInfo*, Thread*) (position.cpp:222)

sf-x · 2020-07-25T16:16:40Z

* clang: nnue/evaluate_nnue.cpp:60:45: error: no member named 'aligned_alloc' in namespace 'std'

Compiler bug. std::aligned_alloc is standard C++17.

vondele · 2020-07-25T16:18:57Z

or header missing?

sf-x · 2020-07-25T16:57:06Z

hmm. std::aligned_alloc should exist after #include <cstdlib> which exists in "types.h", which is included from "position.h", which is included from nnue/evaluate_nnue.cpp...

vondele · 2020-07-25T17:05:36Z

OK, needs clang 7.0 (https://godbolt.org/z/rE3joM), CI is still based on 6.0. .travis.yml needs updating
gcc needs to be >= 7.4

vondele · 2020-07-25T17:24:39Z

I have add a commit to upgrade the travis settings on Linux... probably somebody needs to come with a similar fix for the osx of our CI.

Edit: seems like this upgrade didn't work. I'll revert that patch and we'll address the issue later

JavaMast · 2020-07-25T19:04:39Z

@vondele

I'm afraid that assemblies for the x86_64 and x86_32 architecture (without sse3 or popcnt) are useless for testing in the framework, since they are much slower than the classic Stockfish assemblies for the same architecture and are much weaker. Testing them can significantly affect the quality of the tests.

vondele · 2020-07-25T19:14:37Z

almost certainly all machines on fishtest support x86-64-modern (which is right now equivalent to x86-64-sse3-popcnt) on the branch. The majority of the machines support avx2 or bmi2. We'll have to deal with that eventually, but right now it is not too much of a concern.

daylen · 2020-07-25T20:53:17Z

On compiling for macOS

On macOS 10.15.6 (Catalina) make build ARCH=x86-64-bmi2 COMP=clang has two issues:

The first one: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/malloc/_malloc.h:50:10: note: 'aligned_alloc' has been marked as being introduced in macOS 10.15 here, but the deployment target is macOS 10.9.0

So we need to change -mmacosx-version-min=10.9 in the Makefile to -mmacosx-version-min=10.15 but according to statcounter only ~52% of Mac users are on 10.15.

If we do change the min version, second issue:
nnue/evaluate_nnue.cpp:60:40: error: no member named 'aligned_alloc' in namespace 'std'; did you mean simply 'aligned_alloc'?
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/malloc/_malloc.h:50:10: note: 'aligned_alloc' declared here

It seems like the C++ library that ships with Xcode dev tools has aligned_alloc in the global namespace. Could maybe get around this with a #if __APPLE__.

dorzechowski · 2020-07-25T23:16:07Z

Fix for memset/memcpy warnings and valgrind issue: PR #2830.

I can't see unfreed memory issue. Could you check again after applying these fixes?

MichaelB7 · 2020-07-26T02:02:57Z

error on build

$ make  build ARCH=x86-64 COMP=mingw

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-18363'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'no'
sse: 'yes'
sse3: 'no'
ssse3: 'no'
sse41: 'no'
sse42: 'no'
avx2: 'no'
pext: 'no'
avx512: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2
LDFLAGS:  -static

Testing config sanity. If this fails, try 'make help' ...

C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64 COMP=mingw all
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o position.o position.cpp
position.cpp: In member function 'Position& Position::set(const string&, bool, bool, StateInfo*, Thread*)':
position.cpp:199:39: warning: 'void* memset(void*, int, size_t)' clearing an object of non-trivial type 'struct StateInfo'; use assignment or value-initialization instead [-Wclass-memaccess]
  199 |   std::memset(si, 0, sizeof(StateInfo));
      |                                       ^
In file included from position.cpp:31:
position.h:40:8: note: 'struct StateInfo' declared here
   40 | struct StateInfo {
      |        ^~~~~~~~~
position.cpp: In member function 'void Position::do_move(Move, StateInfo&, bool)':
position.cpp:714:51: warning: 'void* memcpy(void*, const void*, size_t)' writing to an object of a non-trivial type 'struct StateInfo' leaves 1272 bytes unchanged [-Wclass-memaccess]
  714 |   std::memcpy(&newSt, st, offsetof(StateInfo, key));
      |                                                   ^
In file included from position.cpp:31:
position.h:40:8: note: 'struct StateInfo' declared here
   40 | struct StateInfo {
      |        ^~~~~~~~~
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o syzygy/tbprobe.o syzygy/tbprobe.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o nnue/evaluate_nnue.o nnue/evaluate_nnue.cpp
nnue/evaluate_nnue.cpp: In function 'void Eval::NNUE::Detail::Initialize(Eval::NNUE::AlignedPtr<T>&)':
nnue/evaluate_nnue.cpp:60:45: error: 'aligned_alloc' is not a member of 'std'; did you mean 'aligned_union'?
   60 |     pointer.reset(reinterpret_cast<T*>(std::aligned_alloc(alignof(T), sizeof(T))));
      |                                             ^~~~~~~~~~~~~
      |                                             aligned_union
make[1]: *** [<builtin>: nnue/evaluate_nnue.o] Error 1
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make: *** [Makefile:595: build] Error 2

MichaelB7@VM-894787 MINGW64 ~/home/Github/Stockfish/src (nnue-player-wip)

daylen · 2020-07-26T02:17:23Z

@MichaelB7 Please provide more details, such as what version of g++ you have and your OS/platform. Also, you could try #2831 and see if that fixes your issue—I think the PR makes the aligned_alloc issue more compatible in general for platforms other than macOS.

MichaelB7 · 2020-07-26T03:41:56Z

$ stockfish
Stockfish 250720 64 POPCNT by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
compiler

Compiled by g++ (GNUC) 10.1.0 on MinGW64
VERSION macro expands to: 10.1.0

Windows 10 Pro
Version 10.0.18363 Build 18363
will try your patch

MichaelB7 · 2020-07-26T03:53:45Z

Update: The patch did not resolve the issue:

$ make  build ARCH=x86-64 COMP=mingw

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-18363'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'no'
sse: 'yes'
sse3: 'no'
ssse3: 'no'
sse41: 'no'
sse42: 'no'
avx2: 'no'
pext: 'no'
avx512: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2
LDFLAGS:  -static

Testing config sanity. If this fails, try 'make help' ...

C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64 COMP=mingw all
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++17  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -DUSE_SSE2   -c -o nnue/evaluate_nnue.o nnue/evaluate_nnue.cpp
nnue/evaluate_nnue.cpp: In function 'void Eval::NNUE::Detail::Initialize(Eval::NNUE::AlignedPtr<T>&)':
nnue/evaluate_nnue.cpp:60:40: error: there are no arguments to 'aligned_malloc' that depend on a template parameter, so a declaration of 'aligned_malloc' must be available [-fpermissive]
   60 |     pointer.reset(reinterpret_cast<T*>(aligned_malloc(alignof(T), sizeof(T))));
      |                                        ^~~~~~~~~~~~~~
nnue/evaluate_nnue.cpp:60:40: note: (if you use '-fpermissive', G++ will accept your code, but allowing the use of an undeclared name is deprecated)
nnue/evaluate_nnue.cpp: In instantiation of 'void Eval::NNUE::Detail::Initialize(Eval::NNUE::AlignedPtr<T>&) [with T = Eval::NNUE::FeatureTransformer; Eval::NNUE::AlignedPtr<T> = std::unique_ptr<Eval::NNUE::FeatureTransformer, Eval::NNUE::AlignedDeleter<Eval::NNUE::FeatureTransformer> >]':
nnue/evaluate_nnue.cpp:79:43:   required from here
nnue/evaluate_nnue.cpp:60:54: error: 'aligned_malloc' was not declared in this scope; did you mean '_aligned_malloc'?
   60 |     pointer.reset(reinterpret_cast<T*>(aligned_malloc(alignof(T), sizeof(T))));
      |                                        ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
      |                                        _aligned_malloc
nnue/evaluate_nnue.cpp: In instantiation of 'void Eval::NNUE::Detail::Initialize(Eval::NNUE::AlignedPtr<T>&) [with T = Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::ClippedReLU<Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::ClippedReLU<Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::InputSlice<512>, 32> >, 32> >, 1>; Eval::NNUE::AlignedPtr<T> = std::unique_ptr<Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::ClippedReLU<Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::ClippedReLU<Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::InputSlice<512>, 32> >, 32> >, 1>, Eval::NNUE::AlignedDeleter<Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::ClippedReLU<Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::ClippedReLU<Eval::NNUE::Layers::AffineTransform<Eval::NNUE::Layers::InputSlice<512>, 32> >, 32> >, 1> > >]':
nnue/evaluate_nnue.cpp:80:31:   required from here
nnue/evaluate_nnue.cpp:60:54: error: 'aligned_malloc' was not declared in this scope; did you mean '_aligned_malloc'?
   60 |     pointer.reset(reinterpret_cast<T*>(aligned_malloc(alignof(T), sizeof(T))));
      |                                        ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
      |                                        _aligned_malloc
make[1]: *** [<builtin>: nnue/evaluate_nnue.o] Error 1
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make: *** [Makefile:595: build] Error 2

But if I make the suggested change from the error message above , it worked.

replacing this on line 60:

  pointer.reset(reinterpret_cast<T*>(aligned_alloc(alignof(T), sizeof(T))));

with this:

  pointer.reset(reinterpret_cast<T*>(_aligned_malloc(alignof(T), sizeof(T))));

vondele · 2020-07-26T07:43:51Z

@dorzechowski one more valgrind warning triggers when loading TB files:

uciok
setoption name SyzygyPath value ../tests/syzygy/
==122921== Conditional jump or move depends on uninitialised value(s)
==122921==    at 0x13DF43: Position::set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool, StateInfo*, Thread*) (position.cpp:222)
==122921==    by 0x13EFDE: Position::set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Color, StateInfo*) (position.cpp:407)
==122921==    by 0x16D412: (anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>::TBTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (tbprobe.cpp:366)
==122921==    by 0x173AB9: void __gnu_cxx::new_allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> >::construct<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>((anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (new_allocator.h:147)
==122921==    by 0x171CFC: void std::allocator_traits<std::allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> > >::construct<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> >&, (anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (alloc_traits.h:484)
==122921==    by 0x171D97: void std::deque<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>, std::allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> > >::_M_push_back_aux<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (deque.tcc:496)
==122921==    by 0x170C2F: (anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>& std::deque<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>, std::allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> > >::emplace_back<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (deque.tcc:174)
==122921==    by 0x16DAFB: (anonymous namespace)::TBTables::add(std::vector<PieceType, std::allocator<PieceType> > const&) (tbprobe.cpp:489)
==122921==    by 0x16F2A6: Tablebases::init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (tbprobe.cpp:1367)
==122921==    by 0x167A8C: UCI::on_tb_path(UCI::Option const&) (ucioption.cpp:44)
==122921==    by 0x1699F0: UCI::Option::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (ucioption.cpp:202)
==122921==    by 0x163837: (anonymous namespace)::setoption(std::__cxx11::basic_istringstream<char, std::char_traits<char>, std::allocator<char> >&) (uci.cpp:100)
==122921== 
==122921== Conditional jump or move depends on uninitialised value(s)
==122921==    at 0x13DF43: Position::set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool, StateInfo*, Thread*) (position.cpp:222)
==122921==    by 0x13EFDE: Position::set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Color, StateInfo*) (position.cpp:407)
==122921==    by 0x16D62F: (anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>::TBTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (tbprobe.cpp:385)
==122921==    by 0x173AB9: void __gnu_cxx::new_allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> >::construct<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>((anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (new_allocator.h:147)
==122921==    by 0x171CFC: void std::allocator_traits<std::allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> > >::construct<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> >&, (anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (alloc_traits.h:484)
==122921==    by 0x171D97: void std::deque<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>, std::allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> > >::_M_push_back_aux<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (deque.tcc:496)
==122921==    by 0x170C2F: (anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>& std::deque<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0>, std::allocator<(anonymous namespace)::TBTable<((anonymous namespace)::TBType)0> > >::emplace_back<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (deque.tcc:174)
==122921==    by 0x16DAFB: (anonymous namespace)::TBTables::add(std::vector<PieceType, std::allocator<PieceType> > const&) (tbprobe.cpp:489)
==122921==    by 0x16F2A6: Tablebases::init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (tbprobe.cpp:1367)
==122921==    by 0x167A8C: UCI::on_tb_path(UCI::Option const&) (ucioption.cpp:44)
==122921==    by 0x1699F0: UCI::Option::operator=(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (ucioption.cpp:202)
==122921==    by 0x163837: (anonymous namespace)::setoption(std::__cxx11::basic_istringstream<char, std::char_traits<char>, std::allocator<char> >&) (uci.cpp:100)
==122921== 
info string Found 35 tablebases

Edit: this should be fixed by:

diff --git a/src/position.cpp b/src/position.cpp
index 52ba710ae..8f2c4b029 100644
--- a/src/position.cpp
+++ b/src/position.cpp
@@ -404,7 +404,7 @@ Position& Position::set(const string& code, Color c, StateInfo* si) {
   string fenStr = "8/" + sides[0] + char(8 - sides[0].length() + '0') + "/8/8/8/8/"
                        + sides[1] + char(8 - sides[1].length() + '0') + "/8 w - - 0 10";
 
-  return set(fenStr, false, use_nnue(), si, nullptr);
+  return set(fenStr, false, false, si, nullptr);
 }

This set() method is basically just there to create a materials key, and starts from a freshly created Position() object.

mstembera · 2020-07-26T11:25:13Z

@MichaelB7 Thank you. I can now at least compile. However, I believe the parameters for aligned_alloc and _aligned_malloc should be reversed. Also we should probably match it with _aligned_free(ptr) instead of std::free(ptr).

vondele · 2020-07-26T13:38:48Z

first green CI badge on Linux/gcc
https://travis-ci.org/github/official-stockfish/Stockfish/jobs/711914367

dorzechowski · 2020-07-26T17:23:16Z

@vondele Another thing to ascertain before we can use NNUE in SF is a license to use it. As nodchip's implementation doesn't have any license attached, I'm not sure we can even use it legally now. We assume so but it's not enough I think. This should be solved together with @nodchip and those files should have either GPLv3 or compatible license attached.

vondele · 2020-07-26T17:34:22Z

@dorzechowski would be great @nodchip confirms this, but as the code (binaries and sources) are being distributed already, the license is GPL, this is also what the repository Copying file in the repository mentions.

vondele · 2020-07-26T17:35:30Z

Meanwhile also green on Linux/clang : https://travis-ci.org/github/official-stockfish/Stockfish/jobs/711961757

mstembera · 2020-08-04T01:21:36Z

@noobpwnftw I have this PR #2879 to address profiling of both code paths in one PGO compile. However, you are right there has to be a net available to load.

vondele · 2020-08-04T05:35:43Z

yes, PGO builds / default bench is an open issue, but I might as well postpone that one. It does require to have a net available at all times, and maybe it is better to wait a little till we get a feel for that requirement. However, it is quite easy on most systems with the new Makefile target make net. It would require somebody to come up with something similar in Microsoft PowerShell to do this for our appveyor CI (any experts?)

I haven't seen measurements yet of the performance benefit of PGO for NNUE.

vondele · 2020-08-04T07:06:05Z

Meanwhile merged master and updated to a new default net by Sergio Vieri, with impressive testing results:

https://tests.stockfishchess.org/tests/view/5f28fe6ea5abc164f05e4c4c
https://tests.stockfishchess.org/tests/view/5f290229a5abc164f05e4c58

gekkehenker · 2020-08-04T07:14:33Z

Stockfish 12 will surely become the Rybka 3 of Stockfish releases.

nguyenpham · 2020-08-04T08:56:29Z

@vondele: BanksiaGUi displays info string in a special panel, not a popup as the bellow image, thus users won't be interrupted. If they don't open the panel, they don't see that info either.

However, BanksiaGUi keeps displaying all info string until the engine is reloaded. Thus it is not nice if some strings are displayed repeatedly (such as info about Syzygy loading) thus users may see them all the time.

Below is another confusion for users when SF NNUE load data twice when starting.

vondele · 2020-08-04T09:01:52Z

@nguyenpham that's presumably not with the latest branch (at least for the found & loaded, which is not on the branch anymore).

cameron-shahmirzadi · 2020-08-04T11:05:19Z

Is there currently a plan for how testing will work for patches that may have different impacts on different nets? Is the current "best" net going to be used for testing new patches to search?

vondele · 2020-08-04T11:07:46Z

Is the current "best" net going to be used for testing new patches to search?

IMO, yes.

nguyenpham · 2020-08-04T11:22:48Z

I have updated the code and done a quick check, there is no redundant info string when starting. Well done!

nguyenpham · 2020-08-04T12:09:44Z

A few thoughts about EvalFile: setting up EvalFile seems to be one of the new steps which may confuse users since they have to set it correctly before they can play: aware to set up, know where to find eval files, try-error with short or full path forms...

We can help users by doing some auto works to setup EvalFile. To do that we just scan all data files (.nnue) in the SF folder as well as subfolders to get the list of all data files. Now there are two ways to show the list:

EvalFile, type as a string, the default value is . If users use the default value, we will use the latest file from the list. Note that when implementing this way, we may not need the real list. Just compare files and keep the latest one. Advantages: users always have a network to run, they can also set manually to use any networks they want. Disadvantages: if users want to set up manually, they have to find/select it from the disk as before
EvalFile, type as a combo: sort the list by descent time and put their file names only (for being short) as the combo choices. The first item of the combo as well as the default value are to auto pick the latest network. Advantages: users can simply select any networks from the combo without worrying about files/folders/paths. Disadvantages: they can’t select networks outside the main folder.

Your thoughts?

rooklift · 2020-08-04T14:05:00Z

How about this (which is similar to how Lc0 solves this problem, but adapted to the possibility of not needing a net):

Scan the Stockfish folder for .nnue files.
If there aren't any:
- Use NNUE defaults to false
- EvalFile defaults to <empty>
else:
- Use NNUE defaults to true
- EvalFile defaults to the most recent .nnue file present.

This would have to be done before Stockfish responds to the initial uci command, so that the defaults sent to the GUI (via option UCI lines) are correct.

(I don't speak C++ so I can't offer to actually write this, sorry...)

rooklift · 2020-08-04T14:14:52Z

Having said all that, the fact that 2 options are needed is potentially confusing, and will probably lead to some users using classical eval by accident. How about just removing the Use NNUE option and:

Scan the Stockfish folder for .nnue files.
If there aren't any:
- EvalFile defaults to <empty>
else:
- EvalFile defaults to the most recent .nnue file present.

(Again, doing all this before replying to the initial uci command.)

The GUI can still override the choice with an explicit setoption command, of course. When the time comes to actually evaluate positions, use classical eval only if EvalFile is <empty>.

vondele · 2020-08-04T16:02:13Z

I feel it is better to clearly indicate which net is default/recommended with the given binary, and would encourage against running the binary with a non-default net, except for experts/enthusiasts. I'm hopeful that we will have process that basically makes sure that the best net available will be the default.

Adding too much logic to the code is not good IMO, and if so best done by the GUI. If the GUI makes sure the engines working directory contains the default net, there user just needs to click the 'Use NNUE' option. The way we have a unique name for the net (with the sha) makes finding this net relatively easy for the GUI.

It is clear that .nnue files will not remain of the same format. They will evolve in features, content, and what exactly is part of the evaluation. Unless the user is an expert, picking non-default nets will lead to surprising behavior.

vondele · 2020-08-04T17:10:59Z

I haven't seen measurements yet of the performance benefit of PGO for NNUE.

On my build (x86-64-avx2), the effect is actually quite significant, around 10% speedup. Anyway... not urgent IMO.

vondele · 2020-08-04T18:27:50Z

I have decided to postpone those items that require running with a net during build / CI to after a merge, it is better to get some feeling for the download/build process first. The step is likely not very difficult, but if done incorrectly would be annoying.

Similarly we don't really need to worry on older hardware on fishtest, performance is good even if included.

The mostly leaves the Makefile issue for fixing.

vondele · 2020-08-05T06:25:34Z

Makefile issues fixed. SPRT NNUE vs NNUE testing OK.

Small note for classical vs NNUE tests (e.g. regression tests, FYI @Alayan-stk-2), since the precise result of such a match depends quite strongly on the hardware used, we might expect some variability in future RT (SF11 vs SFdev). It is not worth trying to fix this with time odds. It does however mean that also our statistical tests to decide if a worker is bad will fail for RT, so it is important to switch 'auto-purge' to off for those tests. (see also https://tests.stockfishchess.org/tests/view/5f28fe6ea5abc164f05e4c4c)

mstembera · 2020-08-05T10:25:15Z

@vondele I see you changed the default net in the code for this test
https://tests.stockfishchess.org/tests/view/5f2a4d35a5abc164f05e4cf6
Are the options on the test page not working properly if you use
EvalFile=nn-c157e0a5755b.nnue
?

vondele · 2020-08-05T11:01:29Z

@mstembera indeed, on fishtest using EvalFile as an option on test submission is not supported, at least not right now. It would need some code to download this net, and translate it to the proper cutechess option (add path). Not impossible, but not there. There is some WIP documentation on testing nets:
https://github.com/glinscott/fishtest/wiki/Creating-my-first-test#nnue-net-tests

domschl · 2020-08-05T16:49:27Z

Some bench results for armv8 (Raspberry Pi 4, 64bit), armv7 (Raspberry Pi 3, 32bit) and x86, node count consistency check with apple silicon:

target	nodes	nodes NNUE	nps	nps NNUE
X86-64-bmi2	4746616	4254913	2139078	1444301
apple-silicon	4746616	4254913
armv8	4746616	4254913	648444	64236
armv8-neon	4746616	4254913	648444	183916
armv7	4746616	4254913	174456	16878

edit: armv7 added.

vondele · 2020-08-05T17:43:33Z

@domschl see #2899 (comment)

This patch ports the efficiently updatable neural network (NNUE) evaluation to Stockfish. Both the NNUE and the classical evaluations are available, and can be used to assign a value to a position that is later used in alpha-beta (PVS) search to find the best move. The classical evaluation computes this value as a function of various chess concepts, handcrafted by experts, tested and tuned using fishtest. The NNUE evaluation computes this value with a neural network based on basic inputs. The network is optimized and trained on the evalutions of millions of positions at moderate search depth. The NNUE evaluation was first introduced in shogi, and ported to Stockfish afterward. It can be evaluated efficiently on CPUs, and exploits the fact that only parts of the neural network need to be updated after a typical chess move. [The nodchip repository](https://github.com/nodchip/Stockfish) provides additional tools to train and develop the NNUE networks. This patch is the result of contributions of various authors, from various communities, including: nodchip, ynasu87, yaneurao (initial port and NNUE authors), domschl, FireFather, rqs, xXH4CKST3RXx, tttak, zz4032, joergoster, mstembera, nguyenpham, erbsenzaehler, dorzechowski, and vondele. This new evaluation needed various changes to fishtest and the corresponding infrastructure, for which tomtor, ppigazzini, noobpwnftw, daylen, and vondele are gratefully acknowledged. The first networks have been provided by gekkehenker and sergiovieri, with the latter net (nn-97f742aaefcd.nnue) being the current default. The evaluation function can be selected at run time with the `Use NNUE` (true/false) UCI option, provided the `EvalFile` option points the the network file (depending on the GUI, with full path). The performance of the NNUE evaluation relative to the classical evaluation depends somewhat on the hardware, and is expected to improve quickly, but is currently on > 80 Elo on fishtest: 60000 @ 10+0.1 th 1 https://tests.stockfishchess.org/tests/view/5f28fe6ea5abc164f05e4c4c ELO: 92.77 +-2.1 (95%) LOS: 100.0% Total: 60000 W: 24193 L: 8543 D: 27264 Ptnml(0-2): 609, 3850, 9708, 10948, 4885 40000 @ 20+0.2 th 8 https://tests.stockfishchess.org/tests/view/5f290229a5abc164f05e4c58 ELO: 89.47 +-2.0 (95%) LOS: 100.0% Total: 40000 W: 12756 L: 2677 D: 24567 Ptnml(0-2): 74, 1583, 8550, 7776, 2017 At the same time, the impact on the classical evaluation remains minimal, causing no significant regression: sprt @ 10+0.1 th 1 https://tests.stockfishchess.org/tests/view/5f2906a2a5abc164f05e4c5b LLR: 2.94 (-2.94,2.94) {-6.00,-4.00} Total: 34936 W: 6502 L: 6825 D: 21609 Ptnml(0-2): 571, 4082, 8434, 3861, 520 sprt @ 60+0.6 th 1 https://tests.stockfishchess.org/tests/view/5f2906cfa5abc164f05e4c5d LLR: 2.93 (-2.94,2.94) {-6.00,-4.00} Total: 10088 W: 1232 L: 1265 D: 7591 Ptnml(0-2): 49, 914, 3170, 843, 68 The needed networks can be found at https://tests.stockfishchess.org/nns It is recommended to use the default one as indicated by the `EvalFile` UCI option. Guidelines for testing new nets can be found at https://github.com/glinscott/fishtest/wiki/Creating-my-first-test#nnue-net-tests Integration has been discussed in various issues: official-stockfish#2823 official-stockfish#2728 The integration branch will be closed after the merge: official-stockfish#2825 https://github.com/official-stockfish/Stockfish/tree/nnue-player-wip This will be an exciting time for computer chess, looking forward to seeing the evolution of this approach. Bench: 4746616

This patch ports the efficiently updatable neural network (NNUE) evaluation to Stockfish. Both the NNUE and the classical evaluations are available, and can be used to assign a value to a position that is later used in alpha-beta (PVS) search to find the best move. The classical evaluation computes this value as a function of various chess concepts, handcrafted by experts, tested and tuned using fishtest. The NNUE evaluation computes this value with a neural network based on basic inputs. The network is optimized and trained on the evalutions of millions of positions at moderate search depth. The NNUE evaluation was first introduced in shogi, and ported to Stockfish afterward. It can be evaluated efficiently on CPUs, and exploits the fact that only parts of the neural network need to be updated after a typical chess move. [The nodchip repository](https://github.com/nodchip/Stockfish) provides additional tools to train and develop the NNUE networks. This patch is the result of contributions of various authors, from various communities, including: nodchip, ynasu87, yaneurao (initial port and NNUE authors), domschl, FireFather, rqs, xXH4CKST3RXx, tttak, zz4032, joergoster, mstembera, nguyenpham, erbsenzaehler, dorzechowski, and vondele. This new evaluation needed various changes to fishtest and the corresponding infrastructure, for which tomtor, ppigazzini, noobpwnftw, daylen, and vondele are gratefully acknowledged. The first networks have been provided by gekkehenker and sergiovieri, with the latter net (nn-97f742aaefcd.nnue) being the current default. The evaluation function can be selected at run time with the `Use NNUE` (true/false) UCI option, provided the `EvalFile` option points the the network file (depending on the GUI, with full path). The performance of the NNUE evaluation relative to the classical evaluation depends somewhat on the hardware, and is expected to improve quickly, but is currently on > 80 Elo on fishtest: 60000 @ 10+0.1 th 1 https://tests.stockfishchess.org/tests/view/5f28fe6ea5abc164f05e4c4c ELO: 92.77 +-2.1 (95%) LOS: 100.0% Total: 60000 W: 24193 L: 8543 D: 27264 Ptnml(0-2): 609, 3850, 9708, 10948, 4885 40000 @ 20+0.2 th 8 https://tests.stockfishchess.org/tests/view/5f290229a5abc164f05e4c58 ELO: 89.47 +-2.0 (95%) LOS: 100.0% Total: 40000 W: 12756 L: 2677 D: 24567 Ptnml(0-2): 74, 1583, 8550, 7776, 2017 At the same time, the impact on the classical evaluation remains minimal, causing no significant regression: sprt @ 10+0.1 th 1 https://tests.stockfishchess.org/tests/view/5f2906a2a5abc164f05e4c5b LLR: 2.94 (-2.94,2.94) {-6.00,-4.00} Total: 34936 W: 6502 L: 6825 D: 21609 Ptnml(0-2): 571, 4082, 8434, 3861, 520 sprt @ 60+0.6 th 1 https://tests.stockfishchess.org/tests/view/5f2906cfa5abc164f05e4c5d LLR: 2.93 (-2.94,2.94) {-6.00,-4.00} Total: 10088 W: 1232 L: 1265 D: 7591 Ptnml(0-2): 49, 914, 3170, 843, 68 The needed networks can be found at https://tests.stockfishchess.org/nns It is recommended to use the default one as indicated by the `EvalFile` UCI option. Guidelines for testing new nets can be found at https://github.com/glinscott/fishtest/wiki/Creating-my-first-test#nnue-net-tests Integration has been discussed in various issues: #2823 #2728 The integration branch will be closed after the merge: #2825 https://github.com/official-stockfish/Stockfish/tree/nnue-player-wip closes #2912 This will be an exciting time for computer chess, looking forward to seeing the evolution of this approach. Bench: 4746616

vondele · 2020-08-06T15:11:54Z

The code has been merged, I'll close this issue. Thanks for the contributions!

This patch ports the efficiently updatable neural network (NNUE) evaluation to Stockfish. Both the NNUE and the classical evaluations are available, and can be used to assign a value to a position that is later used in alpha-beta (PVS) search to find the best move. The classical evaluation computes this value as a function of various chess concepts, handcrafted by experts, tested and tuned using fishtest. The NNUE evaluation computes this value with a neural network based on basic inputs. The network is optimized and trained on the evalutions of millions of positions at moderate search depth. The NNUE evaluation was first introduced in shogi, and ported to Stockfish afterward. It can be evaluated efficiently on CPUs, and exploits the fact that only parts of the neural network need to be updated after a typical chess move. [The nodchip repository](https://github.com/nodchip/Stockfish) provides additional tools to train and develop the NNUE networks. This patch is the result of contributions of various authors, from various communities, including: nodchip, ynasu87, yaneurao (initial port and NNUE authors), domschl, FireFather, rqs, xXH4CKST3RXx, tttak, zz4032, joergoster, mstembera, nguyenpham, erbsenzaehler, dorzechowski, and vondele. This new evaluation needed various changes to fishtest and the corresponding infrastructure, for which tomtor, ppigazzini, noobpwnftw, daylen, and vondele are gratefully acknowledged. The first networks have been provided by gekkehenker and sergiovieri, with the latter net (nn-97f742aaefcd.nnue) being the current default. The evaluation function can be selected at run time with the `Use NNUE` (true/false) UCI option, provided the `EvalFile` option points the the network file (depending on the GUI, with full path). The performance of the NNUE evaluation relative to the classical evaluation depends somewhat on the hardware, and is expected to improve quickly, but is currently on > 80 Elo on fishtest: 60000 @ 10+0.1 th 1 https://tests.stockfishchess.org/tests/view/5f28fe6ea5abc164f05e4c4c ELO: 92.77 +-2.1 (95%) LOS: 100.0% Total: 60000 W: 24193 L: 8543 D: 27264 Ptnml(0-2): 609, 3850, 9708, 10948, 4885 40000 @ 20+0.2 th 8 https://tests.stockfishchess.org/tests/view/5f290229a5abc164f05e4c58 ELO: 89.47 +-2.0 (95%) LOS: 100.0% Total: 40000 W: 12756 L: 2677 D: 24567 Ptnml(0-2): 74, 1583, 8550, 7776, 2017 At the same time, the impact on the classical evaluation remains minimal, causing no significant regression: sprt @ 10+0.1 th 1 https://tests.stockfishchess.org/tests/view/5f2906a2a5abc164f05e4c5b LLR: 2.94 (-2.94,2.94) {-6.00,-4.00} Total: 34936 W: 6502 L: 6825 D: 21609 Ptnml(0-2): 571, 4082, 8434, 3861, 520 sprt @ 60+0.6 th 1 https://tests.stockfishchess.org/tests/view/5f2906cfa5abc164f05e4c5d LLR: 2.93 (-2.94,2.94) {-6.00,-4.00} Total: 10088 W: 1232 L: 1265 D: 7591 Ptnml(0-2): 49, 914, 3170, 843, 68 The needed networks can be found at https://tests.stockfishchess.org/nns It is recommended to use the default one as indicated by the `EvalFile` UCI option. Guidelines for testing new nets can be found at https://github.com/glinscott/fishtest/wiki/Creating-my-first-test#nnue-net-tests Integration has been discussed in various issues: official-stockfish#2823 official-stockfish#2728 The integration branch will be closed after the merge: official-stockfish#2825 https://github.com/official-stockfish/Stockfish/tree/nnue-player-wip closes official-stockfish#2912 This will be an exciting time for computer chess, looking forward to seeing the evolution of this approach. Bench: 4746616

vondele mentioned this issue Jul 25, 2020

SF NNUE #2728

Closed

This was referenced Jul 25, 2020

NNUE integration work. official-stockfish/fishtest#732

Closed

[WIP] NNUE player #2825

Closed

daylen mentioned this issue Jul 26, 2020

Fix NNUE build on macOS #2831

Closed

vondele mentioned this issue Jul 26, 2020

[NNUE] use_nue=false for getting the material key #2835

Merged

vondele mentioned this issue Aug 4, 2020

Set option Use NNUE to true #2899

Closed

vondele mentioned this issue Aug 6, 2020

Add NNUE evaluation #2912

Closed

vondele closed this as completed Aug 6, 2020

vondele mentioned this issue Aug 13, 2020

Clean Stockfish UCI options [RFC] #2999

Closed

5 tasks

NNUE merge #2823

NNUE merge #2823

Comments

vondele commented Jul 25, 2020

vondele commented Jul 25, 2020 • edited Loading

vondele commented Jul 25, 2020

vondele commented Jul 25, 2020

vondele commented Jul 25, 2020 • edited Loading

dorzechowski commented Jul 25, 2020

vondele commented Jul 25, 2020 • edited Loading

sf-x commented Jul 25, 2020

vondele commented Jul 25, 2020

sf-x commented Jul 25, 2020

vondele commented Jul 25, 2020

vondele commented Jul 25, 2020 • edited Loading

JavaMast commented Jul 25, 2020

vondele commented Jul 25, 2020

daylen commented Jul 25, 2020

dorzechowski commented Jul 25, 2020 • edited Loading

MichaelB7 commented Jul 26, 2020

daylen commented Jul 26, 2020

MichaelB7 commented Jul 26, 2020

MichaelB7 commented Jul 26, 2020

vondele commented Jul 26, 2020 • edited Loading

mstembera commented Jul 26, 2020

vondele commented Jul 26, 2020

dorzechowski commented Jul 26, 2020

vondele commented Jul 26, 2020

vondele commented Jul 26, 2020

mstembera commented Aug 4, 2020 • edited Loading

vondele commented Aug 4, 2020

vondele commented Aug 4, 2020

gekkehenker commented Aug 4, 2020

nguyenpham commented Aug 4, 2020

vondele commented Aug 4, 2020

cameron-shahmirzadi commented Aug 4, 2020

vondele commented Aug 4, 2020

nguyenpham commented Aug 4, 2020

nguyenpham commented Aug 4, 2020

rooklift commented Aug 4, 2020 • edited Loading

rooklift commented Aug 4, 2020 • edited Loading

vondele commented Aug 4, 2020

vondele commented Aug 4, 2020

vondele commented Aug 4, 2020

vondele commented Aug 5, 2020

mstembera commented Aug 5, 2020

vondele commented Aug 5, 2020

domschl commented Aug 5, 2020 • edited Loading

vondele commented Aug 5, 2020

vondele commented Aug 6, 2020

vondele commented Jul 25, 2020 •

edited

Loading

vondele commented Jul 25, 2020 •

edited

Loading

vondele commented Jul 25, 2020 •

edited

Loading

vondele commented Jul 25, 2020 •

edited

Loading

dorzechowski commented Jul 25, 2020 •

edited

Loading

vondele commented Jul 26, 2020 •

edited

Loading

mstembera commented Aug 4, 2020 •

edited

Loading

rooklift commented Aug 4, 2020 •

edited

Loading

rooklift commented Aug 4, 2020 •

edited

Loading

domschl commented Aug 5, 2020 •

edited

Loading