Subject: Latest abrok Development Version May 14th * Including Update NNUE architecture to SFNNv5 * (SF The gift that keeps on giving Sat May 14, 2022 4:23 pm
and should have nearly no influence at STC as depth 27 is rarely reached. It was noticed that initializing the threshold with MAX_PLY, had an adverse effect, possibly because the first move is sensitive to this.
Author: Tomasz Sobczyk Date: Sat May 14 12:47:22 2022 +0200 Timestamp: 1652525242
Update NNUE architecture to SFNNv5. Update network to nn-3c0aa92af1da.nnue.
Architecture changes:
Duplicated activation after the 1024->15 layer with squared crelu (so 15->15*2). As proposed by vondele.
Trainer changes:
Added bias to L1 factorization, which was previously missing (no measurable improvement but at least neutral in principle) For retraining linearly reduce lambda parameter from 1.0 at epoch 0 to 0.75 at epoch 800. reduce max_skipping_rate from 15 to 10 (compared to vondele's outstanding PR)
Note: This network was trained with a ~0.8% error in quantization regarding the newly added activation function. This will be fixed in the released trainer version. Expect a trainer PR tomorrow.
Note: The inference implementation cuts a corner to merge results from two activation functions. This could possibly be resolved nicer in the future. AVX2 implementation likely not necessary, but NEON is missing.