USB

The (U)ltimate (S)hort (B)oook

Project

 

For the first 10 moves

INTRODUCTION

 

The goal of this project is to create a Polyglot opening book [called usb.bin] for the first 10 moves only that roughly will ensure:

 

  1. Book moves for the first 5 moves with a 90+% security;
  2. Book moves for the first 10 moves with a 50+% security.

 

If these numbers are feasible has to be seen, we are going to try anyway.

The USB Polyglot book can be used in two ways:

 

  1. To merge into an existing Polyglot book, instructions below, no program change needed;
  2. To consult after the regular book search has failed to produce a move, a program changed is needed then.

 

______________________________________________________________________________________________

Phase ONE

 

For this reason we created a PGN database of 10+ million games combining several databases, 5 million human and 5 million computer games for maximum opening variety. From these 10 million games we extract the first 10 moves (20 plies) and store each unique position into an EPD record to be analyzed with SOMU Analyze EPD ++ using Stockfish 10 running 40 threads. In total about 13 million EPD positions to analyze, first run at 100ms resulting in usb-100.bin (taking 9 hours to complete) and then on 1000ms resulting in usb-1000.bin (taking almost 4 days), then play ProDeo matches at 40/20 and see what's happening.

Table 1, ProDeo self-play, USB-100ms

Testset

Score

ELO

Book Depth

2moves.pgn

54.5%

+31

7.16

3moves.pgn

55.9%

+41

7.46

4moves.pgn

54.8%

+32

7.41

5moves.pgn

54.9%

+33

7.37

6moves.pgn

54.0%

+28

7.91

7moves.pgn

52.9%

+19

8.39

Table-1, - Score is the match percentage, ELO the corresponding ELO gain, Book Depth the average book depth. For 2moves.pgn this means it stayed 7.16 - 2.00 = 5.16 moves longer iin book, for 3moves.pgn 7.46 - 3.00 = 4.46 and so on, measuring the effectiveness of the system.

Table 2, ProDeo self-play, USB-1000ms

Testset

Score

ELO

Book Depth

2moves.pgn

59.1%

+64

7.31

3moves.pgn

58.6%

+60

7.61

4moves.pgn

56.9%

+47

7.51

5moves.pgn

56.1%

+42

7.40

6moves.pgn

54.5%

+30

7.89

7moves.pgn

50.6%

+3

8.40

Table-2, A massive ELO increase due to a longer Stockfish analysis time. Also a slightly higher book depth and that's what we are going to work on in phase two, adding more games with short opening llines.

_______________________________________________________________________________________________

Table 3, Fruit self-play, USB-100ms

Opening Book

Score

ELO

Book Depth

Fruit big

55.0%

+35

8.24

Fruit default

45.0%

-35

5.80

Table-3 - Looking at the first initial good results we are going to try if it make sense to simply merge the created Polyglot usb-100.bin into an existing Polyglot opening book.

 

We took the good old Fruit 2.1 book and merged usb-100.bin into it, then started a match between Fruit 2.1 (defaultl) vs Fruit 2.1 (book-big) with excellent results.

Table 4, Brainfish self-play 1000ms

Testset

Score

ELO

Book Depth

2moves.pgn

55.5%

+38

8.38

3moves.pgn

54.8%

+33

8.43

4moves.pgn

54.4%

+30

8.63

5moves.pgn

54.6%

+32

8.19

6moves.pgn

50.5%

+3

8.38

7moves.pgn

51.5%

+10

8.75

Table-4 - Can a top program also benefit from an ultra short book? For this reason we took Brainfish (Stockfish plus Polyglot book support) and the answer indicates a yes.

_______________________________________________________________________________________________

 

Phase TWO

Testset

Score

ELO

Book Depth

2moves.pgn

64.0%

+101

8.43

3moves.pgn

60.5%

+74

8.67

4moves.pgn

57.3%

+51

8.64

5moves.pgn

57.5%

+52

8.80

6moves.pgn

56.1%

+42

9.19

7moves.pgn

56.8%

+47

9.54

8moves.pgn

54.3%

+39

9.41

9moves.pgn

 

 

 

We have extended the base database of 10 million games by adding 990,000 games with short opening llines from various sources in order to increase the Book Depth because after all that is the main goal of this project:

 

  1. Book moves for the first 5 moves with a 90+% security;
  2. Book moves for the first 10 moves with a 50+% security.

 

We play the ProDeo matches again and also produce a statistic if the goal of the project (see above) is in reach.

Book Depth split on move number

Testset

Games

Move 3

Move 4

Move 5

Move 6

Move 7

Move 8

Move 9

Move 10

Book Moves

2moves.pgn

400

95.7%

94.5%

93.2%

88.5%

83.2%

67.2%

52.2%

22.8%

2.570

3moves.pgn

1000

---

94.9%

94.3%

89.9%

83.2%

72.6%

60.3%

24.2%

5.665

4moves.pgn

1000

---

---

93.5%

91.4%

82.6%

69.1%

55.1%

24.3%

4.640

5moves.pgn

1000

---

---

---

96.1%

86.6%

70.3%

53.4%

23.3%

2.552

6moves.pgn

1000

---

---

---

---

95.5%

83.1%

66.3%

26.3%

3.194

7moves.pgn

1000

---

---

---

---

---

95.2%

78.0%

33.5%

2.543

8moves.pgn

1000

---

---

---

---

---

---

72.0%

22.6%

1.413

9moves.pgn

1000

---

---

---

---

---

---

---

 

 

Aside of the excellent results it seems we managed our goal.

 

GREEN - if the percentage matches the green goal of 90+%

BLUE - if the percentage matches the blue goal of 50+%

RED - if the percentage fails the green or blue goal.

USB.BIN [1000 ms]

Actually we think the results are too good to be realistic. 2moves.pgn and 3moves.pgn are from Jeroen Noomen and 4moves.pgn ... 9moves.pgn were created a download from Stefan Pohl's site and apparently a lot of games from these testsets ended up in our database.

_____________________________________________________________________________________________

 

Phase THREE

 

Arrived here it would be advisable to increase the analysis time of the 14 million positions, the last step. Good advice is invaluable. 14 million positions at 1 second already took 4 days. An alternative would to analyze them with a fixed depth. We choose for the latter, since contrary to fixed time, fixed depth finishes the iteration. We analyze the 14 million positions with depth=20 plies.

 

Alternatively we offer the 14 million EPD positions ifor download so you can analyze them yourself for an even better result and produce an even stronger Polyglot USB (Ultra Short Book). Drop me a note via the contact form how to do that.

USB.BIN [depth=20]

soon

ALL.EPD

the 14 million EPD positions

___________________________________________________________________________________________

Technical and credits

Source code of the Polyglot format, here and here.

Download Polyglot by Fabien Letouzy (and others).

Documentation Polyglot and how to merge Polyglot books.

Stockfish by T. Romstad, M. Costalba, J. Kiiski.

2moves.pgn and 3moves.pgn by Jeroen Noomen.

4moves.pgn ... 9moves.pgn created from Stefan Pohl's site.