LondonFrau Admin
Posts : 1291 Reputation : 3534 Join date : 2010-02-27 Location : ???
| Subject: BrainFish 170826 Test by Stefan Pohl +50 Elo to latest Stockfish 170810. +40 Elo to asmFish 170819 Fri Sep 01, 2017 3:58 pm | |
| Testrun of Brainfish 170826 (playing with Cerebellum-Library (2017/08/26 (Release 147))) finished. +50 Elo to latest Stockfish 170810. +40 Elo to asmFish 170819.Next testrun: Stockfish 170831. Result not before next Friday. Important news: The domain-name of my website has changed to "http://www.sp-cc.de". But the old name "http://spcc.beepworld.de/" should still work. I decided to restart my long thinkg-time tournament with only "Big 3" engines (asmFish, Komodo, Houdini), because my SALC-book was updated to V3.0 and the games of asmFish and Stockfish of my long thinking-time tournament are analyzed in the Stockfish-forum (and that led to a working functional patch in the past!). And with only 3 engines, a lot more asmFish-games are played in the same time. Stay tuned. My new SALC_V3_10moves opening book is ready. Download it [You must be registered and logged in to see this link.]The new 10moves book is bigger (nearly 13000 endpositions (+30%)) and better (all endpositions analyzed with Komodo 11.01 (100 seconds (!!) per move, 3 threads (=5x more thinking time, than older SALC-books), with Komodo-evaluation inside [-0.6,+0.6]). I use it for my long thinking-time tournament from now.Playing conditions: Hardware: i7-6700HQ 2.6GHz Notebook (Skylake CPU), Windows 10 64bit, 8GB RAM Fritzmark: singlecore: 5.3 / 2521 (all engines running on one core, only), average meganodes/s displayed by LittleBlitzerGUI: Houdini: 2.6 mn/s, Stockfish: 2.2 mn/s, Komodo: 2.0 mn/s Hash: 512MB per engine GUI: LittleBlitzerGUI (draw at 130 moves, resign at 400cp (for 4 moves)) Tablebases: None Openings: HERT testset (by Thomas Zipproth) (download the file at the "Download & Links"-section or [You must be registered and logged in to see this link.]) Ponder, Large Memory Pages & learning: Off Thinking time: 180''+1000ms ( = 3'+1'') per game/engine (average game-duration: around 7.5 minutes). One 5000 games-testrun takes about 7 days.The version-numbers of the Stockfish-development engines are the release-date, written backwards (year,month,day))(example: 170526 = May, 26, 2017), downloaded at [You must be registered and logged in to see this link.] I a lways use the latest version of one day, if more than one version per day is released. And I use the version "Haswell+" (=bmi2). Each Stockfish-version plays 1000 games against Komodo 11.2.2, Houdini 5, Shredder 13, Fizbo 1.9, Andscacs 0.91b. All engines are running with default-settings.To avoid distortions in the Ordo Elo-calculation, from now, only 2x Stockfish (latest official release (because of my new testsettings, at the moment Stockfish 170526 is the latest "official release", because it was the last version, which was tested with the old testsettings and the first one with the new testsettings)+ the latest version) and 1x asmFish and 1x Brainfish are stored in the gamebase (all older engine-versions games will be deleted, every time, when a new version was tested). Stockfish, asmFish and BrainFish older Elo-results can still be seen in the Elo-diagrams below). BrainFish plays always with the latest Cerebellum-Library of course, because otherwise BrainFish = Stockfish, so testing would make no sense at all... Latest update: 2017/09/01: BrainFish 170826 (Ordo-calculation fixed to Stockfish 170526 = 3420 Elo, which was the final result of Stockfish 170526 in the old gamebase. So the Elo-development of Stockfish has no "break" and can continue from the last point of the old gamebase) See the individual statistics of engine-results [You must be registered and logged in to see this link.]Download the current gamebase [You must be registered and logged in to see this link.]Download the gamebase-archive (all played games with the HERT-set) [You must be registered and logged in to see this link.] Program Elo + - Games Score Av.Op. Draws 1 BrainFish 170826 bmi2 : 3467 8 8 5000 76.1 % 3246 41.8 % (new) 2 asmFish 170819 bmi2 : 3427 7 7 5000 72.1 % 3246 46.4 % 3 Stockfish 170526 bmi2 : 3420 7 7 5000 71.3 % 3246 45.6 % 4 Stockfish 170810 bmi2 : 3417 7 7 5000 71.0 % 3246 46.4 % 5 Komodo 11.2.2 x64 : 3383 5 5 8000 57.3 % 3322 52.7 % 6 Houdini 5 pext : 3369 5 5 8000 55.3 % 3324 54.3 % 7 Shredder 13 x64 : 3197 6 6 8000 31.5 % 3345 41.7 % 8 Fizbo 1.9 bmi2 : 3175 6 6 8000 28.8 % 3348 36.1 % 9 Andscacs 0.91b bmi2 : 3103 6 6 8000 20.6 % 3357 31.2 % Below you find a diagram of the progress of Stockfish in my tests since the end of 2016. And below that diagram, the older diagrams. You can save the diagrams (as a JPG-picture (in originial size)) on your PC with mouseclick (right button) and then choose "save image"...The Elo-ratings of older Stockfish dev-versions in the Ordo-calculation can be a little different to the Elo-"dots" in the diagram, because the results/games of new Stockfish dev-versions - when getting part of the Ordo-calculation - can change the Elo-ratings of the opponent engines and that can change the Elo-ratings of older Stockfish dev-versions (in the Ordo-calculation / ratinglist, but not in the diagram, where all Elo-"dots" are the rating of one Stockfish dev-version at the moment, when the testrun of that Stockfish dev-version was finished). _________________ Bettina...............The greatest happiness of life is the conviction that we are loved.
Victor Hugo
| |
|
VitruviusH Advanced Member
Posts : 177 Reputation : 367 Join date : 2013-02-21 Location : San Antonio, Texas
| Subject: Re: BrainFish 170826 Test by Stefan Pohl +50 Elo to latest Stockfish 170810. +40 Elo to asmFish 170819 Sat Sep 09, 2017 8:50 am | |
| Good work again! Thanks for the opening book. | |
|