Stockfish testing Playing conditions: Hardware: Since 20/07/21
AMD Ryzen 3900 12-core (24 threads) notebook with 32GB RAM. Now, 20 games are played simultaneously (!), so from now, each testrun will have 6000 or 7000 games (instead of 5000 before) and will take only 2 days, not 6-7 days as before! From now, all engine-binaries are popcount/avx2, of course, because bmi2-compiles are extremly slow on AMD. To keep the rating-list engine-names consistent, the "bmi2"- or "pext"-extension in the engine-name is still in use for older engines - otherwise ORDO will not calculate all played games by this engine as one engine...
Speed: (singlethread, TurboBoost-mode switched off, chess starting position) Stockfish: 1.3 mn/s, Komodo: 1.1 mn/s
Hash: 256MB per engine
GUI: Cutechess-cli (GUI ends game, when a 5-piece endgame is on the board)
Tablebases: None for engines, 5 Syzygy for cutechess-cli
Openings: HERT_500 testset (by Thomas Zipproth) (download the file at the "Download & Links"-section or
[You must be registered and logged in to see this link.])
Ponder, Large Memory Pages & learning: Off
Thinking time:
180''+1000ms (
= 3'+1'') per game/engine (average game-duration:
around 7.5 minutes). One 7000 games-testrun takes about 2 days.The version-numbers of the Stockfish engines are the date of the latest patch, which was included in the Stockfish sourcecode,
not the release-date of the engine-file, written backwards (year,month,day))(example: 200807 = August, 7, 2020). The used SF compile is the AVX2-compile, which is the fastest on my AMD Ryzen CPU. SF binaries are taken from
[You must be registered and logged in to see this link.] (except the official SF-release versions, which are taken form the official Stockfish website).
Download BrainFish (and the Cerebellum-Libraries):
[You must be registered and logged in to see this link.] To avoid distortions in the Ordo Elo-calculation, from now, only 3x Stockfish (latest official release + the latest 2 dev-versions) and 1x Brainfish are stored in the gamebase (all older engine-versions games will be deleted, every time, when a new version was tested). Stockfish and BrainFish older Elo-results can still be seen in the Elo-diagrams below. BrainFish plays always with the latest Cerebellum-Libraries of course, because otherwise BrainFish = Stockfish. Latest update: 2020/12/16: Stockfish 201214 avx2 (+2 Elo to Stockfish 201205) (Ordo-calculation fixed to Stockfish 12 = 3684 Elo)
See the individual statistics of engine-results [You must be registered and logged in to see this link.]Download the current gamebase [You must be registered and logged in to see this link.] Program Elo + - Games Score Av.Op. Draws
1 CFish 12 3xCerebellum : 3726 8 8 7000 86.1 % 3389 27.3 %
2 Stockfish 201214 avx2 : 3725 8 8 7000 78.4 % 3472 41.2 %
3 Stockfish 201205 avx2 : 3723 8 8 7000 78.2 % 3472 41.5 %
4 CFish 12 avx2 : 3704 8 8 7000 84.6 % 3389 29.1 %
5 Stockfish 12 200902 : 3684 5 5 22000 78.0 % 3436 39.9 %
6 KomodoDragon 1.0 avx2 : 3653 6 6 10000 70.7 % 3471 45.7 %
7 SF 200910 miniNNue avx2 : 3616 7 7 7000 72.1 % 3437 43.2 %
8 Stockfish 200731 popc : 3602 8 8 7000 80.5 % 3345 36.2 %
9 Stockfish 11 200118 : 3565 5 5 17000 69.5 % 3403 41.6 %
10 Stockfish 10 181129 : 3525 5 5 15000 78.5 % 3288 37.7 %
11 KomodoDragon 1.0 MCTS : 3479 6 6 7000 57.7 % 3424 56.9 %
12 Stockfish 9 180201 : 3475 8 8 5000 74.9 % 3273 41.7 %
13 Komodo 14.1 x64 : 3454 6 6 8000 56.3 % 3411 55.6 %
14 Komodo 14 bmi2 : 3445 4 4 20000 52.1 % 3433 51.4 %
15 Houdini 6 pext : 3440 2 2 48000 57.2 % 3388 46.8 %
16 Nemorino 6.00 avx2 : 3440 4 4 17000 47.3 % 3467 50.7 %
17 Komodo 13.3 bmi2 : 3439 6 6 8000 62.8 % 3343 49.9 %
18 Komodo 13.1 bmi2 : 3425 5 5 11000 62.0 % 3334 48.8 %
19 Komodo 12.3 bmi2 : 3413 7 7 7000 62.7 % 3314 49.4 %
20 Ethereal 12.75 avx2 : 3399 4 4 16000 41.5 % 3474 49.9 %
21 Ethereal 12.62 avx2 : 3390 6 6 8000 49.1 % 3402 54.6 %
22 Slow Chess 2.4 popc : 3371 5 5 12000 35.9 % 3493 46.8 %
23 Ethereal 12.50 popc : 3356 6 6 8000 46.9 % 3387 55.5 %
24 Slow Chess 2.3 popc : 3345 4 4 15000 43.2 % 3402 52.4 %
25 Komodo 14 MCTS : 3340 8 8 5000 44.4 % 3386 53.4 %
26 Ethereal 12.25 pext : 3338 5 5 12000 35.2 % 3471 46.4 %
27 Slow Chess 2.2 popc : 3329 5 5 11000 32.9 % 3483 42.7 %
28 RubiChess 1.9dev nnue : 3320 6 6 10000 27.2 % 3522 40.0 %
29 Ethereal 12.00 pext : 3317 6 6 9000 43.1 % 3371 50.8 %
30 Igel 2.8.0 popavx2 : 3315 6 6 8000 37.1 % 3419 48.6 %
31 Ethereal 11.75 pext : 3309 6 6 9000 39.3 % 3392 53.2 %
32 Xiphos 0.6 bmi2 : 3304 3 3 32000 36.7 % 3418 49.0 %
33 Fire 7.1 popc : 3301 2 2 42000 42.0 % 3371 50.7 %
34 Xiphos 0.5.6 bmi2 : 3289 7 7 7000 41.2 % 3356 54.6 %
35 Minic 2.51 nasc_nutr : 3284 6 6 7000 31.2 % 3437 45.0 %
36 Ethereal 11.53 pext : 3281 6 6 7000 42.2 % 3343 53.4 %
37 Komodo 12.3 MCTS : 3276 7 7 7000 42.7 % 3334 46.3 %
38 Ethereal 11.25 pext : 3272 7 7 6000 38.4 % 3362 51.0 %
39 rofChade 2.3 bmi2 : 3258 5 5 11000 33.8 % 3388 47.5 %
40 Booot 6.4 popc : 3245 7 7 6000 31.1 % 3394 46.5 %
41 Schooner 2.2 popc : 3242 7 7 6000 31.3 % 3391 50.3 %
42 Laser 1.7 bmi2 : 3219 8 8 6000 30.8 % 3371 45.8 %
43 Fizbo 2 bmi2 : 3214 8 8 5000 36.0 % 3325 39.0 %
44 Fritz 17 : 3213 8 8 6000 29.4 % 3377 44.2 %
45 Shredder 13 x64 : 3210 8 8 6000 31.9 % 3359 42.6 %
46 RubiChess 1.8 popc : 3208 6 6 7000 32.0 % 3345 46.1 %
47 Defenchess 2.2 popc : 3206 8 8 5000 26.6 % 3395 41.8 %
48 Booot 6.3.1 popc : 3200 8 8 5000 34.0 % 3328 44.1 %
49 Andscacs 0.95 popc : 3168 9 9 5000 23.1 % 3391 35.4 %
The version-numbers (180622 for example) of the engines are the date of the latest patch, which was included in the Stockfish sourcecode,
not the release-date of the engine-file. Especially the asmFish-engines are often released much later!!
Below you find a diagram of the progress of Stockfish in my tests since August 2020. And below that diagram, the older diagrams. You can save the diagrams (as a JPG-picture (in originial size)) on your PC with mouseclick (right button) and then choose "save image"...The Elo-ratings of older Stockfish dev-versions in the Ordo-calculation can be a little different to the Elo-"dots" in the diagram, because the results/games of new Stockfish dev-versions - when getting part of the Ordo-calculation - can change the Elo-ratings of the opponent engines and that can change the Elo-ratings of older Stockfish dev-versions (in the Ordo-calculation / ratinglist, but not in the diagram, where all Elo-"dots" are the rating of one Stockfish dev-version at the moment, when the testrun of that Stockfish dev-version was finished). 
