Regression Test vs v5.0

This is also trained with WSD but more data. I think I need to train with a longer cooldown phase and have a run with the old scheduler which will hopefully finish this afternoon.

Comparison of old scheduler vs WSD. Based on my understanding the new scheduler is not properly tuned at the moment, so this should pass.

This is also trained with WSD but more data. I think I need to train with a longer cooldown phase and have a run with the old scheduler which will hopefully finish this afternoon.

Slight reduction in dataset size, but hopefully higher quality for training. Using WSD scheduler instead of linear. Hoping for small rating gain!

New approach to training net. Hopefully at least Elo neutral but less blunder prone.

New approach to training net. Hopefully at least Elo neutral but less blunder prone.

A much more reasonable middle ground.

Regression testing net training idea.

Non-regression sanity check.

Sanity test of net training

Warmup Stable Decay seems to fit the training pipeline better than the previous linear scheduler based on metrics. Does this hold in self play?

Get an understanding of how far apart the based increased size net is compared to the current version. Fixed nodes in this case as system load would otherwise distort results.

The larger 24x64 net architecture is failing STC so we instead try 16x96. This has far fewer parameters but still has a 5% slowdown we need to overcome. Based on training metrics this lies squarely between the baseline and 24x64.

Having passed the fixed node test as expected, we try an STC test with [-5, 0] bounds, as the slowdown is expected to hurt more than in actual play.

Larger net, hopefully a clear improvement at fixed nodes. Around 16% less Nps.

Trying to better understand the hybrid loss. Both nets are not fully trained, but are at a similar point in training.

Trying to better understand the hybrid loss. Both nets are not fully trained, but are at a similar point in training. 311n removes the CE loss component completely. Even if worth Elo, we may not want this.

Final net with less CE loss component.

Reduced the weighting of the draw prediction loss component.

Draw calibration handled with MSE loss directly on the draw probability should be less sensitive to outliers that are common in the high entropy STC time controls the training data stems from.

Fully trained model with identical hyperparameters to prior gen, but extra data. This should clearly beat the master branch.

Testing net with same hyperparameters but extra self play data a bit prematurely to get a sense of where things lie.

Trained with new data. The new training data has a very different distribution and has neutral effect on validation metrics.

Trained with new data. The new training data has a very different distribution and has neutral effect on validation metrics.