suggest begin to train the 20*192 size net

[l1t1]

since 20224 20256 are all tested by others, and 12*128 stalled

[Vandertic]

Thank you for asking. Current structure is in fact 12x256, not 12x128. We are going to train 20x256 next, but we are having some problems we want to solve before, just to be sure that we are not stalling because of them.

[l1t1]

when will you swich to 20x256 net? stronger net will get more contributors

[Vandertic]

Thank you as always for you interest, @l1t1. We are trying to train the 20x256 network, but there are technical difficulties which are slowing down the procedure.

First of all, "RAM" minibatch had to be reduced from 128 to 64 positions, because otherwise the training doesn't fit in the GPU memory (which, by the way is a RTX 2080ti with 11Gb).
This in turn changed something in the training, and the learning wouldn't even start with the same hyperparameters we used for the smaller structures. We found that by decreasing the learning rate this could be corrected, but this in turn gave overfitting after some generations.

So it's a tricky exercise, but we are spending all time on it at the moment. I hope we will be able to give good news in a few days and have a 20x256 network ready in three weeks.

[l1t1]

rtx 3090 has 24GB

[Vandertic]

Awesome. It would be great if we could find the money to buy one, but I don't see it coming for now...