Enriched EPD

EPD

version 1.0a

60 Mb

Enriched EPD

a tool for

connoisseurs

EPD.EXE is a tool to create a large and fast searchable enriched EPD database. Enriched means that each position is extended with useful information for the purpose to extract theme based EPD collections later. While the emphasis of the tool is on EPD's with deep computer analysis (score and depth) the tool is also usable for positions without.


In a nutshell, there are 4 functions:

1. PGN to Rich EPD - From a PGN with annotations (score and depth) create an 'Enriched' EPD database file.

2. EPD to Rich EPD - Enrich an existing EPD (preferable with score (ce) and depth (acd) to a new EPD database.

3. Search rich EPD - Search an EPD database, make selections and create an EPD with the positions you are looking for.

4. EPD Statistic - Create statistic from an enriched EPD, example.

________________________________________________________________________________________________


1. PGN to Rich EPD - From a PGN with annotations (score and depth) create an 'Enriched' EPD database file.


Select the games.pgn and strart with the default settings by pressing the 'S' key, from the 50,000 Stockfish 12 and Lc0 games (long time control) it will create games.epd, 317.191 positions precisely. What's "enriched" in games.epd ?


8/6k1/1p1p1p2/1P1Pb3/4P1pP/2qB2P1/3R3K/3R4 b - - bm Qc5; ce 80; acd 35; id sf12; fmvn 72; c0 rrb5-qb4 draw;

8/3B3q/8/kp4p1/1p2N3/1P1R2P1/6K1/8 w - - bm Nf2; ce 394; acd 32; id sf12; fmvn 69; c0 rbn2-q3 1-0; mob 103 1-0;

7Q/8/3q1pk1/6pp/7P/6P1/6NK/8 w - - bm Qe8+; ce 392; acd 39; id sf12; fmvn 56; c0 qn2-q3 1-0; ksw 112 1-0;

r7/P2P1k2/3Rn1p1/1B5r/2P5/8/1P6/1K6 w - - bm Bc6; ce 491; acd 32; id sf12; fmvn 45; c0 rb4-rrn1 1-0; ppw 134 1-0;


  1. Score, the "ce" tag;
  2. Depth, the "acd" tag;
  3. The "id" (identification) tag which desrcibes the content, in this case we have chosen for "sf12" since we extracted EPD's from Stockfish 12 games only;
  4. The "fmvn" tag contains the move number;
  5. The material on the board + result of the game in the "c0" tag;
  6. The "c2" tag contains the white name.
  7. The "c3" tag contains the black name.
  8. There are 3 evaluation tags from the initial position, Mobility (mob), King Safety (ksw and ksb) and Passed Pawn evaluation (ppw and ppb), the 3 main pillars of chess a game mostly is decided.

Making Selections, some hints


1. Engine - Ending with a '*' serves as suffix else type exact name. To create from Lc0 games type Lc0* To select all games type a * only.


2. Maximum moves - can be handy to extract opening moves only by enterering "10" or "20".


3. Minimum Search Depth - decides the quality of the created EPD database. Using for instance a search depth of 20 will create a much larger EPD, 2.1 million positions.

4. EPD id tag - describe (classify) your collection.

6-8 : For instance, setting the 3 values on 0 will show every position with the Mobility, King Satefy and Passed Pawn evaluation. The default setting of 100 centipawns only includes evaluation >= 100 centipawns. Scores are from the white point of view.


9. Options are Yes and No. Engine - Engine matches usually contain a lot of uninteresting moves when both engines agree on a draw (score = 0.00) and neverthless play 20-30 moves further.


0. Win / Loss Margin - Skip moves when the engine score indicates a value less than -5.00 or greather than +500.


___________________________________________________________________________________________________


3. Search rich EPD - Search an EPD database, make selections and create an EPD with the positions you are looking for.

With this function we can extract the positions we are looking for as the selection options are similar to the 0-9 build options as listed above. As such we will concentrate on 8. Search on Material. Each position (in the c0 tag) is marked for white and black in the following way:


qrrbbn5-qrrbbn6


white has a queen, 2 rooks, 2 bishops, a knight and 5 pawns.

black has a queen, 2 rooks, 2 bishops, a knight and 6 pawns.


If we want to extract this pattern, we enter that value, or more flexible use the "?" for the number of pawns.

qrrbbn?-qrrbbn? and the pattern will be extracted ignoring the number of pawns. Other examples:


1. r?-r? - will extract rook endings with pawns.

2. r0-?  - will extract a lone rook versus x pawns endings.

3. nn?-bb? - will extract 2 knights with pawns versus 2 bishops with pawns endings.


And so on.


_________________________________________________________________________________________________


2. EPD to Rich EPD - Enrich an existing EPD (preferable with score (ce) and depth (acd) to a new EPD database.

Select the 100.000.epd file and press the "S" key.


After the conversion compare the input and output.

__________________________________________________________________________________________________


4. Convert to SF12 - Enrich an EPD with the Stockfish evaluation scores on Mobility, King Satefy and Passed Pawn for making interesting selections.


So far (in option 1-2) the Mobility, King Safety and Passed Pawn evalutions were provided by an old ProDeo version. With this function we can move to the Stockfish evalution instead by using a tool written by Ferdinand Mosca. Nothing really changes (option 2 = option 4) except that the conversion takes a while longer and it's best to use option 5. Search SF12 EPD instead of option 3 when searching on evaluation.

__________________________________________________________________________________________________


Additional Downloads

with Stockfish 12

evaluations

EPD

Positions

Download Size

Statistic

Contents

4.778.474

69 Mb

Made from 50,000 Stockfish and Lc0 games at decent time control. id=sf12 and id=Lc0

1.558.841

21 Mb

TCEC - Season 1-19 id=TCEC

1.497.734

20 Mb

CCC - CCC 1-9 id=CCC-Season 1-9

344.880

3.5 Mb

CCRL openings used in 2020 analysed at depth=24. id=CCRL-2020-d24

268.339

5.5 Mb

id=Last Noomen Book analysed at 60 seconds per move by Stockfish 10.

4.041.664

58 Mb

CCRL 40/15 games between 3200+ elo rated engines. Minimum Search depth=25. id=CCRL-3200.

1.268.378

17.8 Mb

CCRL 40/15 games between 3300+ elo rated engines. Minimum Search depth=25. id=CCRL-3300.

171.852

2.3 Mb

CCRL 40/15 games between 3400+ elo rated engines. Minimum Search depth=25. id=CCRL-3400.

ProDeo-d24

667.454

in progress

ProDeo Book analysed with SF12, depth-24. id=ProDeo Book

Used PGN's

PGN

Games

Download Size

50.663

41 Mb

19.555

131 Mb

15.916

113 Mb

70.664

48 Mb

20.192

13.8 Mb

2.496

1.8 Mb

Credits

 Ferdinand Mosca for the EVAL utility.

 Dann Corbit for the 100.000.epd

__________________________________________________________________________________________________

12 popular test suites analyzed with Stockfish 12 at extreme long time control up to one hour per move depending on the difficulty of the test suite.

EPD

Positions

Found

Time per move

Contents

css-wm

100

87

10 minutes

From World Champions games

Hard Talkchess

213

142

one hour

STS

1500

1210

1 minute

Strategic Test Set

Nolot

11

10

one hour

Author Pierre Nolot

Tim Krabbe

110

59

10 minutes

Brilliant-Moves

60

43

10 minutes

Source unknown

Der-Brillante-Schachzug

100

71

30 minutes

Author Walter Eigenmann

Arasan suite version 21

200

185

30 minutes

Fischer

100

60

1 minute

Mainly positional, source unknown

Endspiel-Test

100

72

30 minutes

Author Walter Eigenmann

Alpha Zero - Stockfish 8

10

5

one hour

Author Walter Eigenmann

Kai Laskos

1000

716

5 minutes

1000 middle game positions