INTRODUCTION

The goal of this project is to create an extended opening book (or multiple books) for all chess engines that roughly will ensure:

1. Book hits for the first 10 plies with a 90-95% security
2. Book hits for the first 15 plies with a 75-80% security
3. Book hits for the first 20 plies with a 50-55% security
4. Book hits for the first 25 plies with a 25-30% security
5. Book hits for the first 30 plies with a 10-15% security.

If these numbers are feasible has to be seen, we are going to try it anyway.

All moves in an *.EBF book are analysed by at least one top engine at 30 seconds a move. Meaning that:

• The quality of the moves for the vast majority of chess engines is good enough to play it as a book move, gain some strength and win time on the clock. If Stockfish 7 plays a move analysed for 30 seconds it likely will be better than 98% of other engines.

• For variety reasons a position is (can be) analysed by more engines. Actually the system allows you to up to 127 engines you can freely define yourself.

Once your engine is out of your (own) opening book consult the (an) *.EBF opening book (just one call with as input an EPD string of the current position) and it for instance will return:

______________________________________________________________________________________________________

CREATING BOOKS

The base of the book origins from a 883.000 PGN database that contains a mixture games of:

1. CRRL 40/40 games, ELO at least 2600
2. CEGT 40/20 games, ELO at least 2500
3. MegaBase (ChessBase), ELO at least 2500
4. My own MillionBase 2.5, ELO at least 2500

From these 883.000 games with ProTools 1.6 (to be released soon) 30 (depth based) EPD files will be extracted and analysed using the Arena 2.01 Automatic Analysis feature. Note that Arena 3 and 3.5 are bugged, hence version 2.01. This has resulted in the following schedule to analyse.

Once an EPD is analysed by Arena the results (move + score) will be extracted from the Arena *.LOG file and the positions are added to the book (with ProTools) and made available for download on this page.

'When Protools 1.6 will be available for download (need to clean things up first) you can let Arena 2.01 analyse the games of your own engine with your prefered (stronger) engine and add the results to the book.

______________________________________________________________________________________________________

TIME

With the above schedule in mind it's clear from the onset this will be a very time consuming project. On an 8-core PC I can analyse 23.040 (30 seconds) positions a day. So this project will run for months for only 1 engine, Stockfish 7 being the first candidate. The D=29 with 672.211 positions alone will run for 29 days.

An alternative is to decrease the time control from 30 seconds to 20 | 10 | 5 even 2 seconds, it's still good enough for many engines, SF7 is so strong. The result, time based *.EBF books.

Another solution to tackle the time problem is your participation, not limited to chess programmers only, experienced users are very welcome to help as well. Participation is possible in various ways.

1. I can email you the EPD set(s) to analyse with Arena 2.01, you return the created *.LOG file. Based on a time control of 30 seconds EPD sets are usually not bigger than 3,000 positions (runs 25 hours) so you can divide multiple EPD sets over your availables cores. With Protools 1.6 you can split large EPD files to any number of positions yourself.

2. Once Protools 1.6 is available you can do it all by yourself. Create EPD's from your own prefered PGN database (human and/or comp games), analyse with Arena 2.01 and create your own *.EBF books.

3. I would love to include the analysis of the latest Komodo and Houdini 4 too. Since I don't have these engines I am looking for volunteers.

______________________________________________________________________________________________________

FUTURE

*.EBF books can be extended endlessly, size isn't an issue, speed neither, positions are found quickly using the binary search technique. As such one (for instance) can take all the 2.5 million games of MillionBase 2.5 and analyse all the positions from depth 0 to 20 it won't hurt the performance.

Whether it is useful to do such a job elowise is an open question. Depth=10 from our project already is about 33.971 positions, extracting Depth=10 from MillionBase 2.5 the number almost goes up 5-fold to 156.394

On the other hand if you like to play with your engine on chess servers like ICC, FICS or PLAYCHESS then it could be effective. For that purpose consider the massive number of PLAYCHESS games available at the Gladiators website.

______________________________________________________________________________________________________

BETA PERIOD

Before starting with the giant job I would like to have a short beta period if everything functions well. There is no fun in analysing positions for months and then to conclude one has to do it all over. Provided is a small *.EBF book made of all (2005-2015) CCRL 40/40 games analysed at 2 seconds (depths 0-8) with Stockfish 7, Komodo 7 and Gull 3 in total 110.562 positions. Source code included.

Questions either via the contact form or at Talkchess, my handle is REBEL.

______________________________________________________________________________________________________

SUPPORTED ENGINES

This is a typical project (like with PGN & EPD or Winboard & UCI) that stands or falls with the number of chess programmers that are going to support the EBF Extended Book Format. The more engines that support it, the more activity of users that will build stronger and stronger EBF books.

To get things to work takes no longer than 1 hour, you might even want to rewrite the source code yourself for your own purposes, the data structure of the EBF format (as described in the Source Code is quite simple. Therefore I like to maintain a list of engines that support the format. Drop me a note if you want to be named.

______________________________________________________________________________________________________

DOWNLOADS

Source code - Run demo.exe for a first impression.

ProTools 1.6 - Documentation on a special page.

EPD - the 30 EPD sets (see above) (65 Mb) to analyse with Arena 2.01

______________________________________________________________________________________________________

PARTICIPATION

If you want to participate in the project analysing parts of the 30 EPD sets (or your own) then mail me the results via the contact form in case you don't have my email address and I will make them available for download as an *.EBF book.

______________________________________________________________________________________________________