ProDeo 2.9

The Programmer

version

Maximum

information

As an engine programmer (without an Interface) it's an annoying fact you can't present

your chess engine as you want it to be because the Interface decides

what you are going to see on your screen.


This new version is going to change that, via a freeware text program 15 extra

information pages are available.

1

Will show the PGN, EPD and the main lines of the current position.

2

Will show the book moves of the current position.

3

Will show the chess tree of the current position.

4

Will show an overview of the main evaluation parts of the current position.

5

Will show the root moves of each iteration.

6

Programmer info of the finished search.

7

Search overview since installation.

8

Debug a move sequence.

9

include.txt

Search interesting move(s) only, not all.

10

exclude.txt

Exclude move(s) from searching.

11

ProDeo.eng

Modify ProDeo settings.

12

Modify Pawn Evaluation Tables.

13

Modify several other Evaluation Tables.

14

Modify Piece Square Tables.

15

View how others (human and computers) think about the current position.

To refresh a page press F4


Additional information

book.txt

Switch from the old Spracklen (2 bytes) move format which was in use by many others (Genius, ChessMaster, TASC, Frans Morsch) to the (16 bytes) Polyglot format. Advantage, faster and easier to maintain, for instance with SCID for PC. The standand version comes with an opening book of 260.000 positions, the extended vesion with an opening book of 22 million positions.


eoc.txt

For a full explanation what the goals are of an EOC database check the old REBEL website. The standard version comes with a small database of 1.8 million positions created from games of the world champions of the last 150 years. In the extended version the database was created from Millionbase and contains 108 million positions.


eval.txt

Something like this every engine programmer must have. Numbers are in binary where 256 is 1.00 being one pawn. The final score, the sum of the evaluation parts is displayed in decimal.


search.txt

Gives an insight of the diminishing effect at which depth the engine no longer changes its mind about the best move. The statistic is split into 3 parts, statistics for the middle game, end game and late end game. The example given is made from self play matches at fast time controls.


trace.txt

Sometimes as a programmer you want to know what's going on in that giant chess tree of millions of positions, tracing the search can (emphasis added) help. The example text is the result of typing "e2e4 e7e5 g1f3 b8c6 d2d4" from the start position, press the save button and start analyzing, thereafter keep pressing F4 after each iteration for an update. Some explanatons:


Score - is the return value of the search after d2d4 is fully searched which will be stored into the TT.

No - is the place in the (sorted) move list.

ND - is the new modified depth by the various extensions and reductions.


include.txt

Another debug tool to check the reliability of the search. The negative side effects of modern search techniques are often amazing. Just 2 random examples:


r3qrk1/4bppp/4p3/p2pP2Q/1p1B4/1PpPP3/P1P2RPP/5RK1 w - - bm Rf6;

With a normal search it will take almost an half hour to find the winning Rf6 move, when we type f2f6 + save + analyze and thus only search Rf6 as move a winning score shows up quickly. And just when you might think that's ProDeo specific you may want to try this position with Stockfish 9, it needs a 37 ply search.


r1b1k2r/p2n1ppp/1p2p3/3p2B1/P2P4/1Nn1P3/5PPP/R3KB1R w KQkq - bm f3;

We type f2f3 (winning a pawn) + save + analyze and a positive score shows up after 3 seconds at depth 14. A normal search needs 18 plies and 1 minute.


exclude.txt

r1b1k2r/p2n1ppp/1p2p3/3p2B1/P2P4/1Nn1P3/5PPP/R3KB1R w KQkq - bm Bxh7+;

This is Kasparov - Deep Blue and (chicken) Kaspy did not dare Bxh7+ instead he played the solid a3. ProDeo does the same, after 16 plies a3 is still the best move but surprisingly has Bxh7+ as continuation! So what if we exclude a2a3 from the search? And so we type a2a3 + save + analyze and indeed Bxh7+ at iteration 16 shows up as best move.


So basically this function can be used as a NEXT BEST alternative.

ProDeo.eng

Engine parameters to change various aspects of ProDeo. For instance the evaluation, the search and file management. For an overview of its functions consider the ProDeo 2.6 Personality Creator which explains most of the parameters.


To make changes effective you need to restart the program.

pst.txt

pawns.txt

tables.txt

Modify specific evaluation tables. Also here counts you will need to restart the program before the changes are effective.

consult.txt

Contains the history of the current position from the human and computer point of view. Further explained in the update-2 part.

Installation


To run ProDeo you will need to announce the engine first, under Chessbase this means:

Engine -> New UCI engine -> Browse -> RebelUCI.exe


Under ChessPartner do:

Extra -> Engine Import Wizard -> Winboard engines -> Install -> ProDeo.exe


For Arena:

Engines -> Install new engine -> RebelUCI.exe

___________________________________________________________________________


Final remarks


1. Some Interfaces (like Arena) when launched won't load the current engine until it is

commanded something to do, no worries, Editpad will show up as soon the engine is

approached.


2. Like ProDeo 2.3, 2.4, 2.5, 2.6, 2.7 this is a new feature version only, no changes to

the engine itself.


3. Because of all the maintenance of the text files the engine might even lose some elo points in strength.

Standard version Approx 21 Mb

Extended version

Approx 837 Mb

Added a new extensive feature in the spirit of this version, maximum information and presenting your engine as you wanted it to be, not limited any longer by the GUI it is running.


This update  is about recording chess history and use it as an analysis (study) tool. It will consult a PGN database of 2.3 million (human-human) games and communicate the search results for the current position in your browser. Looking for a specific game? For instance the immortal Zurich (1953) Averbach-Kotov game? Copy the position into your favorite GUI and ProDeo will find the game instantly. Or Rubenstein's immortal Rxc3!! etc. etc. Games matching the position standard are stored in consult.pgn.


Like with EditPad your browser will only start up once at program start and it will warn you with one or two clicks if the board position is found in the database. Then refresh the browser page, in most browsers this quickly can be done by pressing F5.


Parameters are found at the beginning of the ProDeo.eng file in the Personal folder.

Parameter

Meaning

[Consult File = consult\human.pgi]

PGI databases are stored in the consult folder

[Consult Max = 10]

Number of games visible [maximum = 500]

[Consult Sound = on]

Turn on or off the warning clicks.

[Player One = ]

Explained below

[Player Two = ]

Explained below

The [Player One] and  [Player Two] options are meant for opening study (preparation). Suppose you are Vishy Anand and the next tournament game is against Gary Kasparov, and you have a database anand.pgi (included in the download) where all your games are stored, you run:

[Consult File = consult\anand.pgi]

[Player One = Anand]

[Player Two = Kasparov]

And you will notice [ link ] that the results against Kaspy (only 36.7%) are not very florescent. Clear the [Player Two = ] and you will get the overall score of all the games of Anand [ link ] and with a 62.2% score that's the Anand we know.

In case you have a collection of your games in PGN you can make a PGI database yourself and use it in ProDeo 2.9. Steps:


1. Download, install and run SOMU 1.3 and make sure your PGN is in the sub-folder PGN.

2. Press F6 and select your PGN, the PGI file will be created.

3. Copy the PGN and PGI into the ProDeo Consult folder.


. With SOMU 1.3 you can create a PGI database from any PGN this way, even if that PGN only contains FEN strings.

. Use the EPD to PGN function to make an EPD set suitable for a PGI conversion.

Remarks


1.This function performs best in analysis mode since GUI's (tested are ChessPartner, Fritz and Arena) spur on engines somewhat different.


2. When you make changes to the PGN you will have to rebuild the

PGI index file again with SOMU.


3. On your right two downloads are offered, ProDeo 2.9b with a PGN of 2.3 million games and a light version ProDeo 2.9c with a smalll PGN database.

ProDeo 2.9b

Approx 876 Mb

ProDeo 2.9c

Approx 20 Mb

___________________________________________________________________________________________

This section has been greatly worked on.


The Polyglot ProDeo.bin book has been extended with 2 new features.

Move

Weight

Score

Depth

Learn

d2d4

51.60%

0.06

20

55

e2e4

41.96%

0.19

20

60

c2c4

4.13%

0.05

20

33

g1f3

2.31%

-0.01

20

0

All positions in the book are analyzed with Stockfish 10 with depth=20 which gave an improvement of 24 elo points.


Added a learning algorithm. Learning against itself gave an 49 elo improvement.


The 2 new features are valid for any Polglot book, it is explained in detail at the book page.

Parameters in the ProDeo.eng tabulator

Parameter

Meaning

[Book File = books\ProDeo.bin]

Any valid Polyglot book

[Book-File = ]

Second Polyglot book

[Book Depth = 80]

Stop searching the book after move 40

[Book Learner = off]

on | off

[Book Score = off]

on | off

[Book Use = random]

best | random

The above are the default settings, play random for diversification.


. [Book Use = best] plays the move the highest percentage.

. [Book Score = on] plays the move with the highest Stockfish score.

. [Book Learner = on] plays the move with the highest learn score.

____________________________________________________________________________________________


The CONSULT tabulator


consult.txt contains the history of the current position from the human and computer point of view. A snippet:

Reference : consult\3000.dat

Positions : 10.963.142

Move

Depth

Score

Engine

e2e4

25

0.74

Stochfish 10

d2d4

28

0.50

Stockfish 9

e2e4

22

0.19

Komodo 11

d2d4

23

0.43

Komodo 11

The computer list with depths and scores contains only the results of single threated engines rated above 3000 CCRL. With almost 11 million positions it start making sense to maintain such a database.


Sources

. CCRL 40/40

. CEGT 40/20

. CEGT 40/120

. CEGT 5+3 PBON

. others, minimum time control of 15 seconds average.


Not supported yet

. TCEC

. CCC


Statistic per engine and how many available moves.


Parameters in the ProDeo.eng tabulator

[Comp File = consult\3000.dat]

Database location

[Display Max = 10]

Maximum = 195

Reference : consult\gm.dat

Positions : 2.999.013

Moves

Player

Game

e2e4 e7e5 g1f3

Caruana

Giri 1-0

d2d4 d7d5 c1f4

Vachier-Lagrave

Caruana 1-0

e2e4 c7c5 g1f3

Carlsen

Nepomniachtchi

e2e4 c7c5 g1f3

Anand,V

Kasparov,G ½-½

Database containing the games of the historic players throughout history plus the players current FIDE top-20.


Player statistic and how many available moves.


Parameters in the ProDeo.eng tabulator

[GM File = consult\gm.dat]

Database location

[Display Max = 10]

Maximum = 195

ProDeo 2.91

161 Mb

_____________________________________________________________________________________________

ProDeo 2.92 is the next edition of the programmer version making visible as much as possible information as a chess engine may have to offer. With the new Poly 1.5 version we introduced a new of tuning Polyglot opening books via WDL (won, loss, draw) move statistics. ProDeo 2.92 makes the statistics visible and also allows you to tune the book with a parameter. Consider the position afrer 1.e4 c5 2.Nf3, click the book tab and press [F4]

Opening Book : books\elo2500.bin

Positions : 5.279.294

WDL Factor : 2.50

Book

Weight

Total

Won

Draw

Loss

Perc

WDL

d6

47.27%

14.374

3.110

6.806

4.458

45.31%

70.54%

e6

25.91%

8.261

1.602

3.936

2.723

43.21%

15.69%

Nc6

24.59%

7.634

1.479

3.818

2.337

44.38%

13.77%

g6

1.20%

373

77

178

118

44.50%

0.00%

a6

0.56%

163

41

72

50

47.23%

0.00%

Nf6

0.38%

128

29

48

51

41.40%

0.00%

b6

0.06%

23

5

8

10

39.13%

0.00%

Qc7

0.01%

3

1

1

1

50.00%

0.00%

f5

0.00%

1

0

1

0

50.00%

0.00%

Totals

30.960

6.344

14.868

9.748

Explanations

• Weight - is the original Polyglot weight percentage.

• WDL - is the new recalculated weight using the WDL Factor parameter 2.50, see further the Poly 1.5 page and this book is 30-40 elo stronger than the old ProDeo book.


• The WDL can be changed with the following parameter in the ProDeo.eng script file.


[WDL Factor = 250]

ProDeo 2.92

60 Mb