MEA Rating List

(M)ea (R)ating (L)ist

Download

MRT.EXE creates from a MEA Excel (*.csv) file a rating list as used here so you can create (and publish) your own findings. Syntax: mrt [source] [destination]

 

MRE.EXE creates from a MEA log-file 2 EPD files, one that contains the positions an engine scored no point at all and a second one that contains the positions that the best (10 points) move is not found. See the examples [ one ] and [ two ] of the Stockfish 9 log-file at 60 seconds per move. Syntax: mre [source]

This new type of rating list is inspired by MEA a tool developed by Ferdinand Mosca processing the 1500 theme based STS positions created by Dann Corbit and Swaminathan Natarajan. As an experiment shows the results are reasonable in sync with the CCRL 4/40 rating list and insipired me to give the STS testsuite more attention in the form of 6 rating lists.

 

  • Bullet 0.2 second per move
  • Blitz 0.5 second per move
  • 1 second per move
  • 5 seconds per move
  • 10 seconds per move
  • about 60 seconds per move established by 10 seconds per move running on 8 real cores (no hyperthreading)

 

An ELO rating (between 0 and max. 4000) is assigned based on the score (points) an engine produces. If an engine scores the maximum of 1500*10 points the ELO rating is 4000, else lower.

 

It's amazing the see the old (2010-2012) Rybka 3 -> Ippolit -> Robolito based clique (Houdini 1.5, Bouquet, Critter) to dominate the various lists while programs rated 250-300 elo higher, like Komodo and Stockfish, are unable to surpass them.

____________________________________________________________________________________________________

 

TOOLS

___________________________________________________________________________________________________

 

MEA experiment

with ProDeo

and the reliability of the 1500 positions STS test-suite

 

For this purpose we create a special ProDeo version that stores all the 1500 10 point solutions into the program and assign a bonus to it in the root move. We begin with a bonus of 5 pawns and lower the bonus gradually to see at which points it starts to break and no longer scores the maximum of 15,000 points and the maximum of 4000 ELO.

Click to enlarge

 

As one can see the break starts at [1.50] missing only 3 positions, with the [1.00] bonus 8 are missing, thereafter it rapidly goes down. Looking at the ELO difference of ProDeo 2.2 and the current top programs the test-suite gives a reliable impression considering the purpose why it was created, a tool to tune the evaluation of a chess program.

 

For fun the bonus gradually was further decreased all the way to [0.05] and it is amazing to see how much even a small push of [0.05] can do, 852 points more (±85 more moves found). A push of only [0.25] would put ProDeo on top of the 1 second rating list.