Subject: Latest abrok Development Version May 14th * Including Update NNUE architecture to SFNNv5 * (SF The gift that keeps on giving Sat May 14, 2022 4:23 pm
and should have nearly no influence at STC as depth 27 is rarely reached. It was noticed that initializing the threshold with MAX_PLY, had an adverse effect, possibly because the first move is sensitive to this.
Author: Tomasz Sobczyk Date: Sat May 14 12:47:22 2022 +0200 Timestamp: 1652525242
Update NNUE architecture to SFNNv5. Update network to nn-3c0aa92af1da.nnue.
Architecture changes:
Duplicated activation after the 1024->15 layer with squared crelu (so 15->15*2). As proposed by vondele.
Trainer changes:
Added bias to L1 factorization, which was previously missing (no measurable improvement but at least neutral in principle) For retraining linearly reduce lambda parameter from 1.0 at epoch 0 to 0.75 at epoch 800. reduce max_skipping_rate from 15 to 10 (compared to vondele's outstanding PR)
Note: This network was trained with a ~0.8% error in quantization regarding the newly added activation function. This will be fixed in the released trainer version. Expect a trainer PR tomorrow.
Note: The inference implementation cuts a corner to merge results from two activation functions. This could possibly be resolved nicer in the future. AVX2 implementation likely not necessary, but NEON is missing.
Update NNUE architecture to SFNNv5. Update network to nn-3c0aa92af1da.nnue. 
