fail to explore

[terasakisatoshi]

Thank you for your hard work. I'm trying to run Connect Four example according to the manual:

• https://jonathan-laurent.github.io/AlphaZero.jl/dev/tutorial/connect_four/

git clone --branch v0.2.1 https://github.com/jonathan-laurent/AlphaZero.jl.git
cd AlphaZero.jl
julia --project -e "import Pkg; Pkg.instantiate()"
julia --project --color=yes scripts/alphazero.jl --game connect-four train

Training sequence worked for me. On the other hand, the following command fails which is described in Examining the current agent section

julia --project --color=yes scripts/alphazero.jl --game connect-four explore

with the error message below:

CuArrays.jl SplittingPool statistics:
 - 0 pool allocations: 0 bytes in 0.0s
 - 0 CUDA allocations: 0 bytes in 0.0s
CuArrays.jl SplittingPool statistics:
 - 0 pool allocations: 0 bytes in 0.0s
 - 0 CUDA allocations: 0 bytes in 0.0s

Loading environment

  Loading network from: sessions/connect-four/bestnn.data
  Loading network from: sessions/connect-four/curnn.data
  Loading memory from: sessions/connect-four/mem.data
  Loaded iteration counter from: sessions/connect-four/iter.txt

Starting interactive exploration

Red plays:

1 2 3 4 5 6 7
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .

ERROR: LoadError: MethodError: no method matching think(::MctsPlayer{Game,AlphaZero.MCTS.Env{Game,StaticArrays.SArray{Tuple{7,6},UInt8,2,42},ResNet{Game}}}, ::Game, ::Int64)
Closest candidates are:
  think(::MctsPlayer, ::Any) at /home/terasaki/tmp/AlphaZero.jl/src/play.jl:214
  think(::AbstractPlayer, ::Any) at /home/terasaki/tmp/AlphaZero.jl/src/play.jl:20
  think(::RandomPlayer, ::Any) at /home/terasaki/tmp/AlphaZero.jl/src/play.jl:71
  ...
Stacktrace:
 [1] state_statistics(::Game, ::MctsPlayer{Game,AlphaZero.MCTS.Env{Game,StaticArrays.SArray{Tuple{7,6},UInt8,2,42},ResNet{Game}}}, ::Int64, ::AlphaZero.MemoryBuffer{StaticArrays.SArray{Tuple{7,6},UInt8,2,42}}) at /home/terasaki/tmp/AlphaZero.jl/src/ui/explorer.jl:62
 [2] compute_and_print_state_statistics(::Explorer{Game}) at /home/terasaki/tmp/AlphaZero.jl/src/ui/explorer.jl:151
 [3] start_explorer(::Explorer{Game}) at /home/terasaki/tmp/AlphaZero.jl/src/ui/explorer.jl:294
 [4] start_explorer(::Session{Env{Game,ResNet{Game},StaticArrays.SArray{Tuple{7,6},UInt8,2,42}}}) at /home/terasaki/tmp/AlphaZero.jl/src/ui/session.jl:440
 [5] top-level scope at /home/terasaki/tmp/AlphaZero.jl/scripts/alphazero.jl:77
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] exec_options(::Base.JLOptions) at ./client.jl:288
 [8] _start() at ./client.jl:484
in expression starting at /home/terasaki/tmp/AlphaZero.jl/scripts/alphazero.jl:74
CuArrays.jl SplittingPool statistics:
 - 0 pool allocations: 0 bytes in 0.0s
 - 0 CUDA allocations: 0 bytes in 0.0s

Here is my environment with GPU 1080Ti:

julia> versioninfo()
Julia Version 1.4.2
Commit 44fa15b150* (2020-05-23 18:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Genuine Intel(R) CPU 0000 @ 2.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, broadwell)
Environment:
  JULIA_NUM_THREADS = 40

Any ideas ?

[jonathan-laurent]

You can patch the explorer in v0.2.1 by removing the turn argument in the think function call at the error site.
You can also use master, which also fixes a significant MCTS bug, making training of the connect four agent much faster (it reaches 100% win rate against both baselines after running overnight).

[terasakisatoshi]

• Thank you for your advise. I'm on master branch such that:

$ git rev-parse HEAD 605dd84e8685ad0c8cd7c6885236b5884c9d557e)

• Looks like it works. 👍

Loading environment

  Loading network from: sessions/connect-four/bestnn.data
  Loading network from: sessions/connect-four/curnn.data
  Loading memory from: sessions/connect-four/mem.data
  Loaded iteration counter from: sessions/connect-four/iter.txt

Starting interactive exploration

Red plays:

1 2 3 4 5 6 7
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .

Nmcts    Nmem   Vmem   Vnet
  599  12,000  +0.05  -0.04

          Pmcts   Pnet    UCT   Pmem  Qmcts   Qnet
4  29.0%  29.0%  18.5%  +0.17  24.6%  +0.08  +0.22
6  17.5%  17.5%  15.0%  +0.17  12.0%  +0.07  +0.08
7  14.2%  14.2%  15.6%  +0.16  11.9%  +0.02  +0.02
3  10.5%  10.5%  11.9%  +0.18  14.1%  +0.06  +0.06
5  10.5%  10.5%  10.5%  +0.22  16.5%  +0.11  +0.10
2  10.4%  10.4%  14.9%  +0.18   8.9%  +0.01  +0.07
1   7.8%   7.8%  13.6%  +0.16  12.0%  -0.04  +0.09

>

[terasakisatoshi]

I hope you release v0.2.2

[jonathan-laurent]

I'm about to release v0.3.0. :-)