Quantized network. Should be faster, at the cost of a little accuracy.

Added net definition

Quantized network. Should be faster, at the cost of a little accuracy.

Net trained with more uniform data. If this is not negative we are happy because it simplifies our life. If this is negative it is fine as it means we were doing something reasonably fine.

Another day another net.

Preliminary fixed node test with a potential larger net.

Trained with more data.

Trained with more data.

Regression Test vs v5.0

This is also trained with WSD but more data. I think I need to train with a longer cooldown phase and have a run with the old scheduler which will hopefully finish this afternoon.

Comparison of old scheduler vs WSD. Based on my understanding the new scheduler is not properly tuned at the moment, so this should pass.

This is also trained with WSD but more data. I think I need to train with a longer cooldown phase and have a run with the old scheduler which will hopefully finish this afternoon.

Slight reduction in dataset size, but hopefully higher quality for training. Using WSD scheduler instead of linear. Hoping for small rating gain!

New approach to training net. Hopefully at least Elo neutral but less blunder prone.

New approach to training net. Hopefully at least Elo neutral but less blunder prone.

A much more reasonable middle ground.

Regression testing net training idea.

Non-regression sanity check.

Sanity test of net training

Warmup Stable Decay seems to fit the training pipeline better than the previous linear scheduler based on metrics. Does this hold in self play?

Get an understanding of how far apart the based increased size net is compared to the current version. Fixed nodes in this case as system load would otherwise distort results.

The larger 24x64 net architecture is failing STC so we instead try 16x96. This has far fewer parameters but still has a 5% slowdown we need to overcome. Based on training metrics this lies squarely between the baseline and 24x64.

Having passed the fixed node test as expected, we try an STC test with [-5, 0] bounds, as the slowdown is expected to hurt more than in actual play.

Larger net, hopefully a clear improvement at fixed nodes. Around 16% less Nps.

Trying to better understand the hybrid loss. Both nets are not fully trained, but are at a similar point in training.

Trying to better understand the hybrid loss. Both nets are not fully trained, but are at a similar point in training. 311n removes the CE loss component completely. Even if worth Elo, we may not want this.

Final net with less CE loss component.