Programmers Open Letter to ICGA on Rybka/Fruit

Programmers Open Letter to ICGA on Rybka/Fruit

Postby BB+ » Tue Mar 01, 2011 3:13 am
This has appeared elsewhere, and I copy it here:

Open letter to the ICGA about the Rybka-Fruit issue

Dear David Levy, Jaap van den Herik and the ICGA Board,

Recently the author of Fruit, Fabien Letouzey, wrote an open letter to the computer chess community where he raised the concern that Rybka 1.0 beta may be a derivative of Fruit 2.1 in this public post: [You must be registered and logged in to see this link.]

Since then it has emerged from highly respected sources like Zach Wegner, Bob Hyatt and others that there is a lot of evidence that has been accumulated over the last few years that Rybka 1.0 beta is a derivative of Fruit 2.1.

Zach Wegner has presented evidence of alleged significant copied/derived Fruit evaluations in Rybka 1.0 beta here: [You must be registered and logged in to see this link.]

A collection of evidence of the many cases of alleged copied/derived Fruit structure, code & data appearing in Rybka 1.0 beta has been put together in this PDF by Mark Watkins: download/file.php?id=304

It is also worth considering that prior to Rybka 1.0 beta, previous Rybka versions were many hundreds of Elo points weaker than the Rybka 1.0 beta version that suddenly emerged in public in December 2005, just a few months after the open source public release of Fruit 2.1 under the GPL license. That same month Rybka beta entered and won the International Paderborn Computer Chess tournament.

The evidence alleges that by using and deriving code, data and structure from Fruit 2.1, Vasik Rajlich was able to make dramatic and huge progress with "his" program Rybka to the detriment of his fellow competitors. In our view this has made competitions involving Rybka grossly unfair.

As chess programmers we find this overwhelming evidence compelling. We believe Rybka is a Fruit derivative albeit an advanced one.

It is very likely that later Rybka versions have derived and benefited from Rybka 1.0 beta and hence in the circumstances our view is they should also be considered derivatives of Fruit 2.1 until proven otherwise.

We wish to make an official complaint to the ICGA that Rybka is a Fruit 2.1 derivative. Furthermore we believe it is a breach of the GPL license under which Fruit 2.1 was released.

We believe as an unauthorized Fruit derivative Rybka's entry into ICGA events has been contrary to the ICGA rules and the rules of fair play.

We ask the ICGA to carefully review the evidence, assess its validity, and act accordingly.

We note that the ICGA is intending on setting up a tribunal to assess such allegations and we believe this evidence should be strongly considered in that process.

In addition, we think the ICGA should in future insist that all authors of entries to ICGA events must submit to the ICGA the same executable(s), that is taking part in the ICGA event, where they can be stored for future analysis of potential derivative claims should they arise. Each author should also make a full and clear statement as to the originality of the entry, its contributors and any acknowledgements. Should justified suspicions exist authors must be willing to submit source code on a private and confidential basis to a select group of impartial programmers to privately determine source code origin.

Co-signed by the following chess programmers,
Fabien Letouzey
Zach Wegner
Mark Uniacke
Stefan Meyer-Kahlen
Ed Schröder
Don Dailey
Christophe Theron
Richard Pijl
Amir Ban
Anthony Cozzie
Tord Romstad
Ralf Schäfer
Gerd Isenberg
Johannes Zwanzger

BB+

Angels_77 Monday, 6:54 am
A Little Perspetive in laymans terms
( common sence )

by Jeremy Bernstien

Then let's break the silence. Have you ever heard of a software license? Software licenses grant particular usage rights to users of the software and/or source code. Fruit 2.1 was released under the GPL, which means that anyone who used the source code for development of derivative works was legally bound to release that work under the GPL, as well, with open source code. Let me repeat that in terms you can understand: Fruit 2.1 was released under the GPL, which means that anyone who used the source code for development of derivative works was bound to release that work under the GPL, as well, with open source code. Did you get it that time?

If, and I'll emphasize the word if so that you know that I'm serious. If Rajlich used Fruit sources for the development of Rybka, he was bound, contractually, legally bound to abide by the licensing terms. And if he didn't, and if this can be proven, he deserves every bit of scorn and disgrace that comes his way. This isn't a witch-hunt, it's an evidence-based investigation into a potentially serious violation of a software license, a violation which potentially led to significant amounts of money flowing into the pocket of one programmer (Vasik Rajlich) that by rights should have either not flown at all (in the case of an open-source GPL Rybka), or into the pocket of Fabien Letouzey (in the case of some licensed version of his code).

J.B.

[You must be registered and logged in to see this link.]

Angels_77 re: ICGA Fruit clone
Angels_77 3 minutes ago
Re: A lawyer on the GPL and Fruit/Rybka

Postby BB+ » Thu Mar 10, 2011 4:57 am
For the question of copyright infringement, here is a relevant opinion (link from Jeremy):
[You must be registered and logged in to see this link.] ... ntext=bclr

J. Dianne Brinson wrote:For a court to find infringement, the defendants must have copied not just the plaintiff's ideas, but the plaintiff's expression of the ideas. Accordingly, the substantial similarity that constitutes copyright infringement must be expression-level similarity, not merely idea-level similarity. [...]
Professor Chafee, writing in 1945, offered further guidance about line-drawing, stating that "the protection covers the 'pattern' of the work.' Professor Nimmer combined Hand's observation that any work can be broken into patterns at different levels of abstraction with Chafee's suggestion that substantial similarity should be determined by comparing common elements at a level that is somewhat abstract but still concrete enough to constitute an expression. Levels of abstraction provide answers to the question "What is the work about?" The most abstract answer might be, to use Romeo and Juliet as an example, "boy meets girl." A slightly less abstract answer is "boy meets girl, and the two come from hostile groups." An even less abstract answer is "boy meets girl at a dance, the two come from hostile groups, and they eventually take the marriage vows." The least abstract answer is the exact language chosen by the author Shakespeare to express that theme of "boy meets girl." Nimmer's treatise uses a comparison of Romeo and Juliet and West Side Story to illustrate this approach to the idea/expression distinction, identifying thirteen common elements and concluding that the common elements form a pattern sufficiently concrete to provide a basis for a finding that the two works are substantially similar. The pattern of the works are similar, involving a meeting and romance between a boy and girl from hostile groups, with resulting killings and misunderstandings.

In the Fruit-Rybka situation, the evaluation features would seem to meet this "substantially similar" criterion.


[You must be registered and logged in to see this link.]

This has become a very interesting discussion:

Quote :

By bob Date 2011-03-09 21:23
I don't know why you have so much trouble grasping this simple statement, but here goes, once again.

When Strelka came along, Vas claimed that Strelka had huge parts of Rybka code in it. So much that he initially posted "I am claiming strelka as my own code and will release it in a few days..." That was step 1. Someone looking at strelka, and already being familiar with fruit, noted substantial similarities. And posted this. That was step 2. Several of us looked at this and posted explicit details. This was step 3. Several pro-rybka people then attacked the idea of comparing strelka saying that it was not exactly Rybka. So several of us, led by Zach, started to look at disassembled Rybka 1 beta binary stuff to compare that directly to fruit. And we found lots of places where they were identical. And more places where they were very close, but with a value changed here and there. That's was Step 6. Then we see the claim that Rybka was original from the very early version that did poorly in CCT6. Zach looked and discovered that this looks like Crafty. Several of us have looked at this for a week or two now and have created a document showing lots of copied code. Code that comes from crafty. Code that either (a) has an obvious bug that is present in both, or code that was superfluous and completely worthless today, yet which was in both programs (Rybka 1.6.1 and Crafty 19.10...) That was step 7, and that's where we are today.

We didn't just "wait". We simply had it waved in front of us and we looked.

Now is that _really_ so hard to grasp/understand. We didn't discover all of this 5 years ago and then get together and say "let's wait for 5 years to reveal this, just to be jackasses." It _just_ came to light with the release of Strelka, and it has been bubbling/stewing along ever sense. It has not reached the point where it is going to be carefully investigated and documented so that all can reach a rational conclusion about what happened.

That's all there is to it. No conspiracy theory, no nothing...

And this:

Quote :

By bob Date 2011-03-10 03:36
The proof is clear, concise, and anybody with a CS degree will be able to follow it point by point. It is not some wild concoction of a group of people that are drunk or on drugs. It really is becoming a very clearly written document. We have two parallel tasks going on. Finding additional examples of copied Crafty code, and then presenting the evidence in a form that is clear to anyone qualified to understand it. And providing a parallel narrative that explains why each of the "unusual pieces of code" was included in Crafty and how it came to be left there when it was no longer needed. The idea is to discover pieces of copied code that everyone will agree had to be copied, because no one would write such code today. One example. Edwards egtb's did not include en passant, which meant that if you reached a 4 piece ending where each side had kp, you could not probe if the pawns were on adjacent files, one had not moved from it's original square, and the other pawn had not advanced so far that ep would never be possible. Way before Rybka was developed, we threw those tables out and switched to Eugene's EGTBs, for several reasons. Steven never did the 5 piece files, they were too big and the generation was too slow. Steven's tables didn't have on the fly decompression so they would have been _very_ large for the 5's and 6's. And of course, steven omitted EP captures, while Eugene did not. Short summary, the code to avoid probing egtb if an ep is possible was present in early versions of Crafty, and this was not caught until someone reported a bug about Crafty not using EGTBs for a particular KPKP ending. I looked, and found that old code still in iterate.c and removed it before releasing version 23.1. That code is in Rybka 1.6.1. Why? It doesn't even work for an ending where one side has two pawns, it will break and avoid lots of egtb probes that could be helpful. Who would add such code when EP was _never_ an issue with Nalimov tables? Edwards tables were not available starting with Crafty version 16.0, which was in the 1997 time frame, _way_ before there was a Rybka of any kind. By 1997 we were using Nalimov tables, and nobody would write the code that was in Crafty. So how did it get into Rybka? Only one answer.

That is what we are providing. line by asm line from Rybka 1.6.1 (for this example) with a line-by-line listing of the code in Crafty, showing _perfect_ equivalence. Then a detailed explanation (easier to read than the above) explaining why the code was in Crafty, why it shouldn't have been in the 19.x version(s), particularly the version copied to build the first Rybka. And why no program developer starting after 2000 would even think about adding such a piece of code to their program.

And we are doing this example by example, and yes, there are several, not just one or two. And each is very similar in nature. Some oddball bug in Crafty that I left in by accident. Something nobody would add on their own because it simply makes no sense. Yet it is there in both.

That's our purpose, to put that together. And then give Vas a chance to address each point, one by one, in whatever way he sees appropriate. I do not know if he will do so or not. That's beyond my ability to control or predict.

This should never have happened. CCT rules have never allowed clones/derivatives. I suppose that those of us that organized the many CCT events will have to decide what the proper action is once the ICGA makes their ruling.

And then we move on until the next such case springs to life...

As far as the link provided in your post goes, I can only suppose that this person is not a computer scientist. And that is OK. But he is not going to be able to pass judgement on the merits of what Mark, Zach, and the rest of us will provide to the ICGA without a proper understanding of how compilers work, a working knowledge of assembler language, and so forth. If you can go from C to asm, you can go back. Guaranteed. And two different programmers, writing an "idea or algorithm" independently will not produce a C program that translates to the _same_ assembly language. That implies semantic equivalence. It does not happen by serendipity when you look at the typical number of lines in the critical parts of a chess engine. It happens by copying source code. I don't know what else can be said. There are always going to be those that don't understand the process of "decompiling" a program, and therefore are going to cast aspersions on the conclusions drawn from this process. Not anything we can do for those that simply refuse to believe. But our goal is going to be to provide examples for the compiler literate, and a concise narrative for those that are not. Then, hopefully judgement will lead most to the only reasonable conclusion.

More of this debate u can read here [You must be registered and logged in to see this link.]

Angels_77 re: a shade more than circumstantial

[You must be registered and logged in to see this link.]

By bob (Hyyatt ) [us] Date 2011-03-13 00:37

I was asked for this information. Thought I would put it here, rather than buried in the big thread where the discussion was.

8c730ba94879c733ec291538e1a9c320 *Rybka-1.6.1.exe

That was created by doing this in Linux:

md5sum -b Rybka-1.6.1.exe

Perhaps someone is willing to verify that so that everyone is convinced this is the program we are looking at for the Crafty comparison...

The string "1.6.1" occurs in that executable.

Some others of interest:

From Rybka 1.6.1 exe file:

generated/made/unmade %d moves per second
generated/made/unmade %d moves, time=%.2f seconds
generated %d moves per second
generated %d moves, time=%.2f seconds

From Crafty-19.10:

option.c: printf("generated/made/unmade %d moves, time=%.2f seconds\n",
option.c: printf("generated/made/unmade %d moves per second\n",(int) (((float) (PERF_CYCLES*
option.c: printf("generated %d moves, time=%.2f seconds\n",
option.c: printf("generated %d moves per second\n",(int) (((float) (PERF_CYCLES*

Crafty 19.10 and Rybka have the same 6 positions for the "bench" command also:

Rybka 1.6.1:

r1bqk2r/pp2bppp/2p5/3pP3/P2Q1P2/2N1B3/1PP3PP/R4RK1
2r2rk1/1bqnbpp1/1p1ppn1p/pP6/N1P1P3/P2B1N1P/1B2QPP1/R2R2K1
r3r1k1/ppqb1ppp/8/4p1NQ/8/2P5/PP3PPP/R3R1K1
4b3/p3kp2/6p1/3pP2p/2pP1P2/4K1P1/P3N2P/8
rnbqkb1r/p3pppp/1p6/2ppP3/3N4/2P5/PPP1QPPP/R1B1KB1R
3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5

Crafty-19.10:

strcpy(args[0],"r1bqk2r/pp2bppp/2p5/3pP3/P2Q1P2/2N1B3/1PP3PP/R4RK1");
strcpy(args[0],"2r2rk1/1bqnbpp1/1p1ppn1p/pP6/N1P1P3/P2B1N1P/1B2QPP1/R2R2K1");
strcpy(args[0],"r3r1k1/ppqb1ppp/8/4p1NQ/8/2P5/PP3PPP/R3R1K1");
strcpy(args[0],"4b3/p3kp2/6p1/3pP2p/2pP1P2/4K1P1/P3N2P/8");
strcpy(args[0],"rnbqkb1r/p3pppp/1p6/2ppP3/3N4/2P5/PPP1QPPP/R1B1KB1R");
strcpy(args[0],"3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5");

Obviously not critical code, but you get the idea...
Parent - - By Labyrinth () [us] Date 2011-03-13 01:01
I had never heard of Rybka 1.6.1 before I saw you mention it on this forum a couple days ago :-/

What was the first Rybka version that had any dealings with the ICGA?
Parent - - By Uly (Gold) [mx] Date 2011-03-13 01:20 Edited 2011-03-13 02:45

> I had never heard of Rybka 1.6.1 before I saw you mention it on this forum a couple days ago :-/


It was [EDIT]a[/EDIT] private Rybka that Vas was passing around before Rybka 1.0 Beta existed.
Parent - By bob () [us] Date 2011-03-13 02:09
There were several. Rybka 1.4, 1.5, 1.6.1, and then the renumbered 1.0 beta when Vas started on the commercial release.
Parent - - By Lukas Cimiotti (*) [de] Date 2011-03-13 09:17

>What was the first Rybka version that had any dealings with the ICGA?


If I'm not totally mistaken, Rybka's first start in an ICGA tournament was in May 2006 in Torino, Italy. That WCCC took place a few days after Rybka 1.2 was released - so it must have been Rybka 1.2.
Parent - By turbojuice1122 (Gold) [us] Date 2011-03-13 15:16
I'm pretty sure that this was Rybka 2.0 beta because it had MP capabilities, which was one of the reasons that it performed poorly due to bugs in the new implementation.
Parent - - By kunah (**) [cz] Date 2011-03-13 13:20
I'm very disappointed. Just checked the R1b-Fruit evidence and quickly compared myself to the disassembly of R1B I just downloaded and to Fruit sources.
I tried just a few things but seems that everything in the PDF is correct. I even noticed movestogo being clamped to 30 just like in Fruit. (Fabien calls this a HACK)

If I would take "ideas" I certainly wouldn't
1) copy code
2) copy uci parser and similar things worth nothing if I really started from scratch (this makes no sense; UCI parser can be written in a few hours)
3) tell that I took a lot but it's only worth 20 elo (especially if it's worth 2700)

and I certainly would convert seconds (double) to milliseconds properly (time management)

Puting it simply: embarassing.

I've nothing else to say on this topic because now I'm convinced. The only thing I regret is being defending him here. Nevermind I can live with that.
Can Vas? Sure if he has no pride at all. At least a statement or apology would be appropriate (read to those involved, not to me).
I was sort of proud that a Czech wrote the strongest chess engine in the world. So the best original Czech engine remains Pseudo 0.7c :sad:.

Regards



~~~~~~~~~~~~~~~


- Angels_77 Angels_77 just now

guess the mods missed deleating this :)


[You must be registered and logged in to see this link.]

Angels_77 re: ICGA Fruit clone
[You must be registered and logged in to see this link.]
The Rybka Cloning Issue

[img][You must be registered and logged in to see this link.] [/img]

Anthony Cozzie

I had originally intended to stay out of this mess, since I am after all retired, but after some discussions with Mark Uniake and Zach Wegner I was persuaded put my name on the Rybka cloning letter. Since my position is not precisely identical with that letter (I'm sure none of the other authors' are either) I thought I'd do it right and write down my exact thoughts here. It seems computer chess has found yet another way to waste my time.
Clone Evidence

I have always been slightly suspicious of Rybka. Strong new programs are usually either clones or the second attempt of a previous author. It just takes time to experiment and tweak things until you get something that works. Most people forget this, but Vasik's first attempt at an engine finished 53rd out of 54 engines in CCT6. His second attempt was about 1000 elo higher; so clearly something changed in the interim. It's easy to compare this with Zappa's 17th, 3rd, and 1st place finishes as I gradually improved it over two years.

In addition Rybka was and continues to be the only program that obfuscates its output of nodes per second and depth. This is usually only done by clones who have something to hide, since users don't really care about nodes per second or reported depth except owners of big hardware who like to brag about it (you can find a hilarious thread of me trolling George Worthington in the CCC archives).

Finally I became convinced of the dubious nature of Rybka with the release of Strelka (which is so obviously similar to Fruit that it is amazing Vasik was able to dodge those accusations) and with the clear evidence from Dr. Watkins and Zach that one piece of user interface code was copied verbatim from Fruit. Most people do not understand just how large the exponential space of programs is - it is totally impossible for two people to write even a small piece of code in an exactly identical way. It is now becoming clear that 90% of Rybka 1.0's evaluation is a direct translation of Fruit, and that even Vasik's original engine was a direct copy of Crafty (how he managed to make it 500 elo weaker, we may never know). I suspect the evidence will only increase from here on out.
The (Accidental?) Genius of Vasik Rajlich

Most people don't understand how, if Vasik Rajlich is an evil cloner stealing the work of others, he could dominate the computer chess world so thoroughly for so long. In fact, I remember having dinner in Torino with Stefan, Amir, Shay, and a few other guys and we were all stumped as to why Rybka was so effective. We all thought he must have invented some new super technique, the equivalent of null move or history pruning.

This puzzle was resolved with the release of the Strelka code. I was very curious to read it, of course, but it rapidly became apparent that there were no special techniques in it at all. It was simply a highly optimized and well tuned program. After thinking about this for a few weeks I realized the secret sauce: hyperbullet testing.

If you want to see how an engine performs, you have it play games against other engines. In 2005 while I was working on what would become Zappa Reykjavik, my methodology was to play standard time control games, look at games where Zappa lost, analyze, and make changes until it would play better moves in the critical positions. This is a very reasonable way of doing things, but it isn't very scientific. It's easy to fix one position while breaking 10 others.

This all changed in 2005 when Fabien Letouzey appeared with Fruit. He used blitz testing: make changes, play a few thousand games, and compute the error bars for your change. This is nicely analytical but requires a huge number of games; even after several thousand games played there is an 95% error bar of +/- 20 elo, and many changes have a much smaller effect. Nonetheless, blitz testing was effective enough for Fruit to become a top engine in only a year or two (it's also worth noting that unlike Rybka, its easy to see a solid progression in the strength of the Fruit versions). The surprise for a lot of us was how accurately strength at slow time controls matches strength at long time controls, which I thought at the time was more dominated by evaluation and less by search.

I had a discussion with Vasik in Mexico about his tuning. His procedure was to play games at 3-4 ply which would finish in 10 seconds or so. At that speed the context switching and parsing of UCI would be much to slow, so he had written a special tester to link two versions of Rybka. To accurately measure 2-3 elo changes requires about 100,000 games; at 10 seconds per game that equates to about 10 days of computer time. With a few quad core machines this is eminently practical and the logical extension of Fabien's approach. I think he and Fabien can be compared to Kepler and Copernicus: Fabien broke the ground, and Vasik perfected it. The massive improvement in engine strength from 2005 (when GMs could still play computers and not embarrass themselves too badly) to 2010 is mostly due to them.

So I think Vasik Rajlich is simply a good programmer with the chess knowledge of an International Master and no moral issues with plagiarizing the work of others who was using a better tuning method than the rest of us. I'm guessing here, but I don't think he really understood that and instead considered other engine authors incompetent morons (understandable when you go from zero to domination in 6 months, even when you are starting with something decent), which also explains why he was willing to tell me his procedure in Mexico.

Of course the other possibility is that Vasik is a genius and the rest of us really are incompetent morons, but I think the simpler explanation is a lot more likely.
Sour Grapes

One of the things that I found very surprising about the whole Strelka controversy was how people were willing to give Vasik every benefit of the doubt, and claim that all these cloning accusations were just attempts to win by legal means what we could not do at the chessboard. To me it was quite obvious at that point that Rybka was at least very shady if not a full-blown clone at its inception, tainting any further success, and it was amazing to see people making every possible excuse for him. Some of this is celebrity bias, like how pro athletes can get away with rape and murder (chess engine authors can only get away with minor crimes), but a lot of it is that for some reason people tried to associate themselves with Rybka to gain status and are now having to backpedal.

Many people have pointed out that the source code from Fruit and Crafty is available and everyone reads it. This is certainly true; I have read both and taken some ideas. For example, Zappa's SEE routine is quite similar to Crafty's (but better, because it uses CMOV. Interestingly Vasik also figured out this optimization since it shows up in Strelka). But what most of these people do not understand is that its very difficult to take one piece of a chess engine and add it to another, because all the pieces depend on each other. An evaluation pattern that fixes a gaping hole in one program may be partially covered by three other patterns in another, resulting in no gain, and the same is true for search terms. To give a personal example: after the release of Strelka, I decided to give futility pruning a shot. The biggest difference between Zappa Mexico and Zappa Mexico II is that futility pruning is on by default. The new version crushed the old one in self play, but against other engines it was a miniscule improvement. According to CCRL there is only an 80% chance that its an improvement at all, and there were a few bugfixes in there as well. So copying a full engine and modifying it is a completely different thing from looking at the source code and taking a few pieces. As a final note, I really got almost nothing from Fruit. In fact I was always kind of irritated because I would figure things out which would then later appear in Fruit for all the world to see. The free version of Zappa is within 80 elo of Fruit 2.1 on CCRL and was released 6 months earlier.

There is no doubt in my mind that we would never have seen a strong Rybka in 2005, and probably not in 2006 either (and remember, without sales he gets to do what the rest of us did, and that is work in the evenings for two years) if Vasik had tried to develop an engine from scratch on his own, so he obviously obtained a huge advantage from his intellectual theft. For comparison, it took me two years of steady work (2003-2005), and I had access to the Crafty source code, I just didn't copy it wholesale. Again, it's not impossible that Vasik is 10 times smarter than I am, but I doubt it.

I consider stealing code immoral, but there is one big point where I diverge from the open letter. In my opinion, his advantage was not insurmountable to the rest of us. To make a sports analogy, Vasik was not using steroids, making him much stronger and faster than any normal athlete at the cost of testicle size and roid rage, but rather having his rich parents buy him training sessions with excellent coaches. Or it's like playing against a basketball team that gets a few bad calls in their favor: if your team is good, you should win anyway. So I feel that even though he clearly had an unfair advantage, and without his intellectual theft it's quite possible no one in computer chess would have ever heard the name of Rajlich, it is not one that we as commercial engine authors could not have overcome.
Conclusion

At this point I am out of computer chess with no intention of returning. I think as a field it is almost solved at this point, and my collaboration with Zach on Rondo was about 10% me and 90% him (you guys may not believe this, but in 2005 after winning the title in Reykjavik I considering forming a similar partnership with an aspiring author since I needed to devote a lot of time to graduate school. The first candidate I had in mind: Vasik Rajlich, mainly because he seemed reasonably smart from my conversations with him during the CCTs. As it happened, he found another, cheaper, codebase to use).

So I while I would like the truth to come out, I don't really have a dog in this fight. Technically Zappa would pick up a title in 2007 if Rybka is forfeited, but under the circumstances I couldn't really be proud of that.

Finally, I find it incredibly amusing and hypocritical that the Rybka team is constantly attacking the various engines based on disassembling Rybka 1.1 (Strelka) and Rybka 3 (Ippolit). I can't really condone it, but AFAIK disassembly is legal, while direct code theft is not.

[You must be registered and logged in to see this link.]

Angels77 wrote:

Angels_77 re: ICGA Fruit clone
[You must be registered and logged in to see this link.]
The Rybka Cloning Issue

[img][You must be registered and logged in to see this link.] [/img]

Anthony Cozzie

I had originally intended to stay out of this mess, since I am after all retired, but after some discussions with Mark Uniake and Zach Wegner I was persuaded put my name on the Rybka cloning letter. Since my position is not precisely identical with that letter (I'm sure none of the other authors' are either) I thought I'd do it right and write down my exact thoughts here. It seems computer chess has found yet another way to waste my time.
Clone Evidence

I have always been slightly suspicious of Rybka. Strong new programs are usually either clones or the second attempt of a previous author. It just takes time to experiment and tweak things until you get something that works. Most people forget this, but Vasik's first attempt at an engine finished 53rd out of 54 engines in CCT6. His second attempt was about 1000 elo higher; so clearly something changed in the interim. It's easy to compare this with Zappa's 17th, 3rd, and 1st place finishes as I gradually improved it over two years.

In addition Rybka was and continues to be the only program that obfuscates its output of nodes per second and depth. This is usually only done by clones who have something to hide, since users don't really care about nodes per second or reported depth except owners of big hardware who like to brag about it (you can find a hilarious thread of me trolling George Worthington in the CCC archives).

Finally I became convinced of the dubious nature of Rybka with the release of Strelka (which is so obviously similar to Fruit that it is amazing Vasik was able to dodge those accusations) and with the clear evidence from Dr. Watkins and Zach that one piece of user interface code was copied verbatim from Fruit. Most people do not understand just how large the exponential space of programs is - it is totally impossible for two people to write even a small piece of code in an exactly identical way. It is now becoming clear that 90% of Rybka 1.0's evaluation is a direct translation of Fruit, and that even Vasik's original engine was a direct copy of Crafty (how he managed to make it 500 elo weaker, we may never know). I suspect the evidence will only increase from here on out.
The (Accidental?) Genius of Vasik Rajlich

Most people don't understand how, if Vasik Rajlich is an evil cloner stealing the work of others, he could dominate the computer chess world so thoroughly for so long. In fact, I remember having dinner in Torino with Stefan, Amir, Shay, and a few other guys and we were all stumped as to why Rybka was so effective. We all thought he must have invented some new super technique, the equivalent of null move or history pruning.

This puzzle was resolved with the release of the Strelka code. I was very curious to read it, of course, but it rapidly became apparent that there were no special techniques in it at all. It was simply a highly optimized and well tuned program. After thinking about this for a few weeks I realized the secret sauce: hyperbullet testing.

If you want to see how an engine performs, you have it play games against other engines. In 2005 while I was working on what would become Zappa Reykjavik, my methodology was to play standard time control games, look at games where Zappa lost, analyze, and make changes until it would play better moves in the critical positions. This is a very reasonable way of doing things, but it isn't very scientific. It's easy to fix one position while breaking 10 others.

This all changed in 2005 when Fabien Letouzey appeared with Fruit. He used blitz testing: make changes, play a few thousand games, and compute the error bars for your change. This is nicely analytical but requires a huge number of games; even after several thousand games played there is an 95% error bar of +/- 20 elo, and many changes have a much smaller effect. Nonetheless, blitz testing was effective enough for Fruit to become a top engine in only a year or two (it's also worth noting that unlike Rybka, its easy to see a solid progression in the strength of the Fruit versions). The surprise for a lot of us was how accurately strength at slow time controls matches strength at long time controls, which I thought at the time was more dominated by evaluation and less by search.

I had a discussion with Vasik in Mexico about his tuning. His procedure was to play games at 3-4 ply which would finish in 10 seconds or so. At that speed the context switching and parsing of UCI would be much to slow, so he had written a special tester to link two versions of Rybka. To accurately measure 2-3 elo changes requires about 100,000 games; at 10 seconds per game that equates to about 10 days of computer time. With a few quad core machines this is eminently practical and the logical extension of Fabien's approach. I think he and Fabien can be compared to Kepler and Copernicus: Fabien broke the ground, and Vasik perfected it. The massive improvement in engine strength from 2005 (when GMs could still play computers and not embarrass themselves too badly) to 2010 is mostly due to them.

So I think Vasik Rajlich is simply a good programmer with the chess knowledge of an International Master and no moral issues with plagiarizing the work of others who was using a better tuning method than the rest of us. I'm guessing here, but I don't think he really understood that and instead considered other engine authors incompetent morons (understandable when you go from zero to domination in 6 months, even when you are starting with something decent), which also explains why he was willing to tell me his procedure in Mexico.

Of course the other possibility is that Vasik is a genius and the rest of us really are incompetent morons, but I think the simpler explanation is a lot more likely.
Sour Grapes

One of the things that I found very surprising about the whole Strelka controversy was how people were willing to give Vasik every benefit of the doubt, and claim that all these cloning accusations were just attempts to win by legal means what we could not do at the chessboard. To me it was quite obvious at that point that Rybka was at least very shady if not a full-blown clone at its inception, tainting any further success, and it was amazing to see people making every possible excuse for him. Some of this is celebrity bias, like how pro athletes can get away with rape and murder (chess engine authors can only get away with minor crimes), but a lot of it is that for some reason people tried to associate themselves with Rybka to gain status and are now having to backpedal.

Many people have pointed out that the source code from Fruit and Crafty is available and everyone reads it. This is certainly true; I have read both and taken some ideas. For example, Zappa's SEE routine is quite similar to Crafty's (but better, because it uses CMOV. Interestingly Vasik also figured out this optimization since it shows up in Strelka). But what most of these people do not understand is that its very difficult to take one piece of a chess engine and add it to another, because all the pieces depend on each other. An evaluation pattern that fixes a gaping hole in one program may be partially covered by three other patterns in another, resulting in no gain, and the same is true for search terms. To give a personal example: after the release of Strelka, I decided to give futility pruning a shot. The biggest difference between Zappa Mexico and Zappa Mexico II is that futility pruning is on by default. The new version crushed the old one in self play, but against other engines it was a miniscule improvement. According to CCRL there is only an 80% chance that its an improvement at all, and there were a few bugfixes in there as well. So copying a full engine and modifying it is a completely different thing from looking at the source code and taking a few pieces. As a final note, I really got almost nothing from Fruit. In fact I was always kind of irritated because I would figure things out which would then later appear in Fruit for all the world to see. The free version of Zappa is within 80 elo of Fruit 2.1 on CCRL and was released 6 months earlier.

There is no doubt in my mind that we would never have seen a strong Rybka in 2005, and probably not in 2006 either (and remember, without sales he gets to do what the rest of us did, and that is work in the evenings for two years) if Vasik had tried to develop an engine from scratch on his own, so he obviously obtained a huge advantage from his intellectual theft. For comparison, it took me two years of steady work (2003-2005), and I had access to the Crafty source code, I just didn't copy it wholesale. Again, it's not impossible that Vasik is 10 times smarter than I am, but I doubt it.

I consider stealing code immoral, but there is one big point where I diverge from the open letter. In my opinion, his advantage was not insurmountable to the rest of us. To make a sports analogy, Vasik was not using steroids, making him much stronger and faster than any normal athlete at the cost of testicle size and roid rage, but rather having his rich parents buy him training sessions with excellent coaches. Or it's like playing against a basketball team that gets a few bad calls in their favor: if your team is good, you should win anyway. So I feel that even though he clearly had an unfair advantage, and without his intellectual theft it's quite possible no one in computer chess would have ever heard the name of Rajlich, it is not one that we as commercial engine authors could not have overcome.
Conclusion

At this point I am out of computer chess with no intention of returning. I think as a field it is almost solved at this point, and my collaboration with Zach on Rondo was about 10% me and 90% him (you guys may not believe this, but in 2005 after winning the title in Reykjavik I considering forming a similar partnership with an aspiring author since I needed to devote a lot of time to graduate school. The first candidate I had in mind: Vasik Rajlich, mainly because he seemed reasonably smart from my conversations with him during the CCTs. As it happened, he found another, cheaper, codebase to use).

So I while I would like the truth to come out, I don't really have a dog in this fight. Technically Zappa would pick up a title in 2007 if Rybka is forfeited, but under the circumstances I couldn't really be proud of that.

Finally, I find it incredibly amusing and hypocritical that the Rybka team is constantly attacking the various engines based on disassembling Rybka 1.1 (Strelka) and Rybka 3 (Ippolit). I can't really condone it, but AFAIK disassembly is legal, while direct code theft is not.

[You must be registered and logged in to see this link.]

[img][You must be registered and logged in to see this link.] [/img]