In case anyone hasn’t heard, the chess engine Rybka was disqualified and forever banned from competing in ICGA sponsored chess events.  Here is a link to the story:

http://www.chessvibes.com/reports/rybka-disqualified-and-banned-from-world-computer-chess-championships/

The chief bone of contention appears to be this passage from the ICGA:

Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their submission details. Programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director.

The first thing I would like to do is to examine the charges against Vasik Rajlich.

The claim is put forward that Rybka borrows much of its evaluation from Fruit, which is a chess engine written by Fabien Letouzey.  This claim is certainly bolstered by the following quotations:

Vasik Rajlich (from an interview in 2005 conducted by Alexander Schmidt:):

“Yes, the publication of Fruit 2.1 was huge. Look at how many engines took a massive jump in its wake: Rybka, Hiarcs, Fritz, Zappa, Spike, List, and so on. I went through the Fruit 2.1 source code forwards and backwards and took many things.”

 

This is from “Rybka 1.0 Beta readme.rtf”, with bolding provided by me:

Special Thanks

I hesitate to include this section because I know I'll forget people who have been helpful in this project, but (with advance apologies to the omitted) here goes:

Robert Hyatt - For Crafty. There is nothing like an open source program for passing knowledge to the next generation.

Fabien Letouzey - For Fruit, which shattered a number of computer chess myths, demonstrated several interesting ideas, and made even the densest of us aware of fail-low pruning.

Tord Romstad - For making Fabien aware of fail-low pruning :-), and more seriously for sharing in every way possible his considerable knowledge.

Eugene Nalimov - For his cryptic but somehow fully functional endgame tablebase access code.

Uri Blass, Gerd Isenberg, Dieter Burssner, Vincent Diepeveen, Raschid Chan, Anthony Cozzie, Mridul M* :), Thomas Gaksch, Peter Berger, Sandro Necchi, Ed Shroeder, Amir Ban, Christophe Theron and every one else, past and present, on the computer chess club: For sharing their computer chess knowledge despite the fact that in principle computer chess is a competitive field.

Heinz van Kempen, Guenther Simon, Olivier Deville, Sergio Martinez, Claude Dubois: for testing early versions of Rybka despite countless bugs and annoying problems.

Alex Dumov, Gabriel Luca: for helping a Windows newbie get up to around half-speed without excessive derision (or at least open derision :))

and Iweta: for being great! :) and a pretty good Rybka tester and web master to boot

Happy testing, and best chess regards,

Vasik Rajlich

Budapest Hungary

December 4, 2005

 

This is from “Rybka 1.0 Preview readme.rtf”, with bolding provided by me:

Special Thanks

I hesitate to include this section because I know I'll forget people who have been helpful in this project, but (with advance apologies to the omitted) here goes:

Robert Hyatt - For Crafty. There is nothing like an open source program for passing knowledge to the next generation.

Fabien Letouzey - For Fruit, which shattered a number of computer chess myths, demonstrated several interesting ideas, and made even the densest of us aware of fail-low pruning.

Tord Romstad - For making Fabien aware of fail-low pruning :-), and more seriously for sharing in every way possible his considerable knowledge.

Eugene Nalimov - For his cryptic but somehow fully functional endgame tablebase access code.

Uri Blass, Gerd Isenberg, Dieter Burssner, Vincent Diepeveen, Raschid Chan, Anthony Cozzie, Mridul M* :), Thomas Gaksch, Peter Berger, Sandro Necchi, Ed Shroeder, Amir Ban, Christophe Theron and every one else, past and present, on the computer chess club: For sharing their computer chess knowledge despite the fact that in principle computer chess is a competitive field.

Heinz van Kempen, Guenther Simon, Ed Murak, Leo Dijksman, Djordje Vidanovic, Chris Taylor, Wael Deeb, Graham Banks, Jeroen Noomen, Sandro Necchi, Olivier Deville, Sergio Martinez, Claude Dubois, and all members of the fantastic CEGT test team: for testing Rybka and all of the other computer chess engines; without testers, there is no computer chess as we know it

Alex Dumov, Gabriel Luca: for helping a Windows newbie get up to around half-speed without excessive derision (or at least open derision :))

and Iweta: for being great! :) and a pretty good Rybka tester and web master to boot

Please enjoy the product, and best chess regards,

Vasik Rajlich

Budapest Hungary

December 11, 2005

So, considering these “confessions” it seems pretty clear that Vas admits to using ideas from Fabien Letouzey’s Fruit, Robert Hyatt’s Crafty, and Eugene Nalimov’s endgame tablebase files from the very start.  Since these remarks were made with the very first release of Rybka betas, it seems pretty clear that the lineage has been spelled out from the beginning.

Now, in the case made against Vasik Rajlich, there are various claims based upon 1.x.y beta versions of Rybka.  However, the first version of Rybka to engage in an ICGA contest appears to me to be Rybka 2.0 Beta since the first ICGA tournament that Rybka entered was the 2006 WMCCC.  The first day of the 2006 WMCCC tournament was 2006.05.25 and the public release of the Rybka 2.0 Beta was June 10, 2006:

http://www.rybkachess.com/docs/Rybka20BetaFAQ.htm

Hence, banter about the 1.x.y beta versions has very little bearing on the ICGA contests, because none of these versions ever entered the contests.  And yet there are reams and reams of analysis of these early beta programs entered into evidence.  Also, any versions that have entered beyond Rybka 2.3.2 appear to have been given a clean bill of health by various experts.  So Rybka 3.x and 4.x contests appear to be untainted, at least at first glance.  Now, lets’ consider this interesting post by Uly on the Rybka forum:

Topic Rybka Support & Discussion / Rybka Discussion / Vasik Rajlich: Innocent of code copying (1642 hits)

1 2  

By Uly Date 2011-07-16 15:00 Edited 2011-07-16 15:05

I'm now fully convinced that Vas is innocent from the accusation that he copied code illegally from Fruit to build Rybka. Every day things were becoming more and more clear, but I have to thank this post from Zach Wegner for the epiphany:

[...]

Coming back to R1/Fruit, yes, if you look at each example in detail, you can't say there is much hard evidence of direct code copying. As I said to Vas, I'm only completely certain that three characters were copied ("0.0"). But given the entire picture (how similarities to Fruit are all over the place, and the previous Rybka versions shared no code) it's just so obvious to me that Vas took Fruit as a base and rewrote things on top of it, presumably until he felt it was "clean". For all the debate, I'm really puzzled why people (who are sufficiently knowledgeable) would dispute this. A lot of talk has been about the evaluation, and while that is interesting from an originality standpoint, it's perfectly reasonable to say that it's OK as long as no code copied. But once the pre-beta Rybkas were released, it was all over. The pre-beta Rybkas were IMO the lynchpin of the entire case--no wonder Vas was pissed when they were leaked (once again, huge credit to Olivier). After that, it became very hard to claim that Rybka didn't start life as Fruit. [...]


(Emphasis mine)

It turns out Vas based Rybka on Fruit, but if there was no code copied* then he didn't do anything wrong about it.

And it seems that without the tangential Rybka 1.6.1 (clone of Crafty, makes Vas a liar but is irrelevant to the case) the ICGA wouldn't have anything to base their decision on, despite the fact that supposedly, Vas broke some rule because he copied code in the engines that participated in the WCCC. But if R1 is clean of copied code*, then Rybka 2.3.2a is also clean of copied code, and there's no basis to believe that R3 or R4 would have copied code in them, so Vas shouldn't have been banned and he is owed an apology.

(*) besides the UCI parser and the "0.0" characters.

Anyone wanting to contest this would like to reply to
this post by Banned for Life, which summarizes the report of the ICGA that supposedly proved that Vas was guilty, but he found this:

EVAL_COMP.pdf                          - has no code whatsoever.
ZW_Rybka_Fruit.pdf                    - shows a comparison of Fruit C with Rybka C. Where the Rybka C came from is not clear. There is a claim that the two functions are nearly identical, but correlation between static eval values is not nearly identical, so this can't be right.
[Ed. note: this document is by Zach Wegner that claims there's no hard evidence of code copying* from Fruit anywhere]
crafty_rybka_evidence.rtf             - irrelevant comparison with a non-released engine.
Rybka_Investigation_report.pdf    - no code whatsoever.
Rybka_Investigation.pdf               - no code whatsoever.
RYBKA_FRUIT_Mar11.pdf             - States that "The crux of the conclusion is that Rybka 1.0 Beta and Fruit 2.1 have exactly the same evaluation features". This is not consistent with testing that shows that the static eval of Rybka 1.0 Beta and Fruit 2.1 are only slightly better correlated than any two supposedly non-related engines.
[Ed. note: this document commended by turbojuice below]
R1x64.eval.txt                            -  Commented R1x64 asm eval but no attempt to correlate with Fruit 2.1.
Ryb232eval.txt                           -  Commented R2.3.2a asm eval but no attempt to correlate with Fruit 2.1.


turbojuice1122 also read the report,
he said about it:

I've now downloaded the version on the ICGA page so that we can compare like versions to make sure that no document names and such have been changed.  To me, the most relevant part for most people should be the document RYBKA_FRUIT_Mar11.pdf.  I cannot find any instances of copied code here.  Plagiarized?  Perhaps...but in a way that is quite legal in this situation, and when rewritten in a different way (bitboards), I don't see any problem in terms of copyright or GPL legality.  I'm particularly looking at a comparison of pages 17 vs. 20, which has been a major contention concerning the search.  The most problematic aspect that I see is actually on pages 14 vs. 15, but as this is in the UCI parser, this may be immune.

(Emphasis mine)

Found Vas guilty yet?

No.

 

We also have this post by Mark Watkins (BB+ is a pseudonym):

Re: The Evidence against Rybka

by BB+ » Mon Jul 18, 2011 1:24 pm

It appears that phrase "copying" is now being subjected to semantic gymnastics. So my claims are:

*) The Rybka 1.0 Beta executable contains no literally copied evaluation code from Fruit 2.1.
*) Rybka 1.0 Beta contains sufficiently much creative expression from the Fruit 2.1 evaluation code so as to transgress the ICGA Rules.
*) The question of whether and to what extent Rybka 1.0 Beta and later versions infringe the copyright of Fruit 2.1 will be the subject of future civil action. My own opinion is that this is closer in spirit to the second point.

Personally, if I heard the word "copy", I would not parse it as being limited to literal copying.

Regarding the first point, there is more than one instance of literally copied code from Fruit 2.1 in Rybka 1.0 Beta, any one of which should suffice for probative similarity in a copyright action.

I've been trying to find a list of court decisions that cite the
Abstraction-Filtration-Comparison Test, but have been unable to do so as of yet. The best I found (with highs/lows of its applicability) was http://www.ladas.com/Patents/Computer/S ... twa06.html

Another useful reference place starts at
http://digital-law-online.info/lpdi1.0/treatise21.html

[...] it is of course essential to any protection of literary property ... that the right cannot be limited literally to the text, else a plagiarist would escape by immaterial variations.

 

Hence, the two main disassemblers of the Rybka code (Zach Wegner and Mark Watkins) both say that the code is not directly copied (both of whom did excellent work and in my opinion did their best to be open and fair).  That leaves two possibilities.  Either the code was written from the idea in memory (which is totally fine unless there is a patent), or it was used without permission.  If it was used without permission, was it fair use?  We would have to examine the code carefully to know.  Comments citing the original sources would be removed by the compile process so it is impossible to know if the original authors have been given proper credit for the ideas.  Whether or not a passage was a large enough percentage of the original to require the written permission by the original authors is another matter which I am not qualified to comment on.

So the issue appears (to me) to boil down to the ICGA rule itself (reiterated here so you won’t have to hunt for it):

Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their submission details. Programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director.

Did Vas use things from Fruit which made his program a lot stronger?  I think clearly the answer is “YES” since he even said so himself.  But wait just a minute here, so does everyone else.  Is there any extremely strong chess program that does not use Late Move Reductions (often called LMR which come from Fruit)?  If so, and they entered the ICGA contest, did they list Fabien as the co-author?  More to the point, did they list Newell and Simon as co-authors for the use of Alpha-Beta pruning (or other appropriate reference such as Donald Knuth’s TAOCP)?  Did they list Albert Zobrist as co-author for the use of his Zobrist hashing?  Did they name Donskoy, Adelson-Velsky, and Arlazarov for the use of their null move heuristic invention?  Literally 100% of successful strong chess programs used these ideas (along with many, many others), copied from other programmers.

Did Vas clearly violate the ICGA rule about using game playing code written by others?  Definitely!  But so does everyone else.  Allow me a moment to expand on this.  If, for instance, I take a web post or a book or an ICGA article or someone else’s program and transcribe some significant idea like “null move pruning” or “pvs search” into my program without including a citation, then that is not only violation of the ICGA rule, it is also copyright violation.  Now, most things of this nature are a tiny fraction of a chess program, and therefore use with citation would probably be considered fair use from a legal standpoint.  However, most chess programmers are extremely lax about citations for the work of others.  One notable exception would be Dr. Hyatt, who is fairly diligent with his use of citations (see attached file literate.pdf for two occasions where I have made a plea for a literate style of programming which offers citations and this concept is discussed briefly).  Of course, unless there is a patent, it would not be illegal to do the same thing from memory, because the fundamental algorithms are only protected by a patent.  But it would be incredibly difficult to tell if I have transcribed something from memory, verses having a book or other source propped open in front of me to copy from.  Considering also that chess International Masters are known to have prodigious memory also compounds that difficulty (and Vasik Rajlich is a chess International Master).   It is not clear to me from the ICGA rule whether any sort of legal violation is necessary to violate the contest rules but I suspect that 100% of programs are in violation by some measure, depending upon how you interpret the ICGA rule.

Now, I would like to address the topic of fairness (which is my main bone of contention).

·         As far as ICGA rules go, the ICGA violates its own rules (see attached file icga-violates-its-own-rules.pdf for an example).  

·         There is a rule about delivery of source code in the ICGA contests, and Vas did not supply the source code when asked.  Unfortunately, he was asked long after the contest was over.  There are some who say that good programmers do not lose their source code, but I have evidence to the contrary that it can happen even to an excellent programmer (see attached file missing-crafty.pdf).  If Vas had been asked for the code while at the tournament and if he then refused to provide it, then I think he would have clearly violated a rule.  Since the request was for code that was used several years ago, I think it is reasonable to say, “I have lost it.”  Now, the rules of the ICGA do not spell out time limits, so I can see a logical debate over this issue on both sides.

·         During the debate over how to punish Vas, there was one person who wanted to speak up in behalf of Vas, which was Chris Whittington.  This lone voice of support was not allowed to participate.  (See attachment retrospect.pdf for a posting authored by Ed Schroder on that topic).

·         The penalties applied against Vas were decided in advance.  See (for instance): http://chessprogramming.wikispaces.com/ICGA+Investigations which spells out all of the exact penalties that were applied against Vas before the fact-finding even began.  I suppose that it could be coincidence that all of the maximum penalties imposed could be the things spelled out in this document, but it seems unlikely to me.  I find the penalties to be unbelievably harsh, but that is nothing more than my opinion and it is no better than anyone else’s.

·         Who voted against Vas? It was his direct competitors and also those who performed the disassembly of his source code.  Both of these groups have an inherent motive for finding him guilty, even though I believe that not one single member of the board acted in a manner which they considered vindictive.  But those sorts of conditions are obviously undesirable for achieving justice because as humans our hearts are treacherous.  We can tell ourselves that we are really seeking justice but is there possibly some little underlying hidden feeling that steers us ever so slightly away from a truly just path?  There is a reason that we don’t have Airbus decide if Boeing did something wrong or have Ford, Chevrolet and GM decide if Honda has done something wrong.

·         The evidence for ICGA violations used against Vas was almost entirely consisting of analysis of Beta programs which never participated in ICGA contests.  In my view, this evidence is not relevant.  Analysis of Rybka 2.0 (which is the first version to participate in ICGA contests) or Rybka 2.3.2a would have been far more relevant.  There is some discussion of Rybka 2.3.2a but it is only relating Rybka 1.0 to Rybka 2.3.2a rather than Rybka 2.3.2a to Fruit.  We can make some assumptions based upon induction but it was not clearly spelled out in the supplied documentation.

There is one flaw in the analysis that I would like to point out.  The comparisons for similarity involve very strong programs and very weak programs.  This is like comparing formula one race cars with automobiles found at a used car sales lot.  A logical group of open source programs to use would have been:

Bison, Booot, Crafty, Delfi, Fruit, Glaurung, Gull, Protector, Scorpio, Slow Chess, Stockfish, Toga, and Umko.

The use of weak programs like Resp, Phalanx, Faile, Pepito and Exchess for comparisons is literally absurd.

It is like saying, “This Fairlane 500 does NOT have sodium cooled valves, but BOTH the Ferrari and the Porsche open wheeled roadsters DO have sodium cooled valves!”

Now, I would like to summarize my opinion.  Clearly, Rybka has ideas borrowed from Fruit and Crafty.  Vas admits this in his earliest beta release notes for Rybka, so it would be silly to deny it.  Whether Vas has violated the law is not clear to me and I do not think has been proven.  Whether Vas has done something unusual or different than the other chess programmers is also unclear to me and I do not think has been proven.  I think that the process to convict Vas was flawed.  I think that the body that met to debate his guilt or innocence was sincere in their efforts and they also sincerely feel that they have arrived at a fair conclusion but I think that the very construction of the group itself made a fair decision debatable at best.

I do not want to vilify the programmers who did their best to find facts or reach decisions about whether or not Vas broke rules.  I am not sure one way or the other (despite having read the hundreds of pages of PDF documentation provided by the fact finding committee).

Neither do I claim that Vas is free from blame.  Only that his method of program construction is quite likely extremely similar to the vast majority of chess program constructions.  Personally, I think that all chess programs (as well as all other sorts of programs) should be written in a literate manner, with proper citations.  With the example of chess programming, we largely have a collection of well-known tricks bound together with the author’s own original thinking for every successful chess program.  I think that to deny this is false thinking.

There are clearly things in Rybka that are interesting and original.  The symmetrical multi-processing aspect of Rybka uses processes instead of threads.  Since there are lots of open source chess programs that do SMP via threads and at the time of Rybka’s publication, there were zero open source chess programs doing SMP via processes, it is clear to me that Vas wanted to do things his own way instead of a simple Frankenstein of available source.  His titanic tables of material imbalance is an idea that is now widely copied.  And his greatest contribution, the idea that we should test our ideas with thousands of test games to quantify the result, is now copied by literally each and every successful author of a strong chess program.  Assuming that the razoring idea in Strelka is (as is often claimed) the same idea as that used in Rybka {supposedly found via reverse engineering}, then it is also true that most modern and strong chess programs are using an idea very similar to his razoring concept (see: http://chessprogramming.wikispaces.com/Razoring).

My main complaint is with the process of fault-finding itself. I think that the design of the process was flawed and with a flawed process flawed decisions are far more probable than via correct process.  I do not expect the majority of chess programmers to agree with me or with my analysis.  My opinion is mine alone but I wanted to say something since it appears to me that a bad process has been used to reach a final decision that affects a man’s career in a significant way.