Warmup Stable Decay seems to fit the training pipeline better than the previous linear scheduler based on metrics. Does this hold in self play?

Get an understanding of how far apart the based increased size net is compared to the current version. Fixed nodes in this case as system load would otherwise distort results.

The larger 24x64 net architecture is failing STC so we instead try 16x96. This has far fewer parameters but still has a 5% slowdown we need to overcome. Based on training metrics this lies squarely between the baseline and 24x64.

Having passed the fixed node test as expected, we try an STC test with [-5, 0] bounds, as the slowdown is expected to hurt more than in actual play.

Larger net, hopefully a clear improvement at fixed nodes. Around 16% less Nps.

Trying to better understand the hybrid loss. Both nets are not fully trained, but are at a similar point in training.

Trying to better understand the hybrid loss. Both nets are not fully trained, but are at a similar point in training. 311n removes the CE loss component completely. Even if worth Elo, we may not want this.

Final net with less CE loss component.

Reduced the weighting of the draw prediction loss component.

Draw calibration handled with MSE loss directly on the draw probability should be less sensitive to outliers that are common in the high entropy STC time controls the training data stems from.

Fully trained model with identical hyperparameters to prior gen, but extra data. This should clearly beat the master branch.

Testing net with same hyperparameters but extra self play data a bit prematurely to get a sense of where things lie.

Trained with new data. The new training data has a very different distribution and has neutral effect on validation metrics.

Trained with new data. The new training data has a very different distribution and has neutral effect on validation metrics.

Net trained with some extra data I had laying around and not yet utilized.

Net trained with some extra data I had laying around and not yet utilized.

Confirmed 2 year old net reproduced and slightly improved