I have a dream....
that one day many chess programmers will participate in this new type of competition.
The goal is to improve the evaluation in a new way, that is, without the obstacle of search. Imagine a reasonable strong (open source) engine with a reasonable good search, readable source code and we replace the evaluation funnction with our own. What are the advantages c.q disadvantages?
1. It's much more easy to discover the weaknesses of your evaluation since search hardly (to none) plays its dominant role. You don't lose (or win) a game because you are outsearched. You lose (or win) a game because of your evaluation.
2. Playing X versus Y -- since the 2 searches are indentical -- you are measuring the evaluation strength.
3. If we can determine strength we can create a competition based on fixed depth games in order to avoid the last issue that may influence the result as engine X and Y have different time cycles, engine X might have a slow evaluation while engine Y has a fast one. As such we eliminate the last obstacle for a reasonable fair estimation who has the strongest eval based within the scope of this project.
4. The learning effect. Will depend on the number of participants considering the status is open source and GPL.
I can name a few but I leave the issue open for a public discussion first on Talkchess (aka CCC).
The technical part
Step-1 - I found a good candidate in TOGA II 3.0 that will serve as a base. It's FRUIT based, mailbox, readable source code and probably well known to many programmers. It's CCRL rated 2852 ranked at place 61.
Step-2 - In a nutshell, I isolated the evaluation of my first engine for the PC (Mephisto Gideon 1993) and included it in EVAL.CPP and replaced the TOGA evaluation with GIDEON. Compiled it and played 2 matches of each 4000 games at D=8 and D=10. See results below.
Step-3 - Made a start updating the GIDEON (1993) evaluation to the ProDeo (2017) evaluation. I am half-way and called it REBEL for the moment. Played the same 2 matches against TOGA II 3.0 Results so far: