Stateful Game-structs throw errors

[oqpvc]

I've started to play around with AlphaZero.jl over the last few days by implementing Othello and Hex. In the process I've run into similar issues a few times: As soon as I let the Game-struct be more stateful, training throws weird errors (e.g. "MCTS.explore! must be called before MCTS.policy", but also others that were more verbose and even less helpful). I expect that is the reason why these lines of code exists in Connect4:

function Base.copy(g::Game)
  history = isnothing(g.history) ? nothing : copy(g.history)
  Game(g.board, g.curplayer, g.finished, g.winner, copy(g.amask), history)
end

I am, generally speaking, at a loss when it comes to debugging this. I think I want to write a few tests to catch these kinds of errors, but I have no idea where to start. Any help would be appreciated.

[jonathan-laurent]

First of all, there are some tests available to catch errors in an implementation of GameInterface.
You can look at tests/test_game.jl, which can be executed on your game by running scripts/alphazero.jl with the check-game sub-command.

Also, what probably happens in your case is that your states do not appear as persistent. This seems to be a pretty common mistake and so maybe I should change the API to make it less common.

Indeed, MCTS calls current_state and stores the resulting state in a hash table. Therefore, you will encounter weird bugs if your states are mutable and you do not copy them properly. If you are struggling with this, my advice is to add copy operations conservatively and only remove the unnecessary ones once everything works.