ProDeo 2.9

The Programmer

version

Maximum

information

As an engine programmer (without an Interface) it's an annoying fact you can't present your chess engine as you want it to be because the Interface decides what you are going to see on your screen. This new version is going to change that, via a freeware text program 15 extra information pages are available.

 

Text in green relates to the new features in the UPDATE-2 section and download being the last

episode how I would like to see a Graphical Chess Interfaces (GUI)

regale its users with optimal information.

1

game.txt

Will show the PGN, EPD and the main lines of the current position.

2

book.txt

Will show the book moves of the current position. Added analysis and learning.

3

eoc.txt

Will show the chess tree of the current position.

4

eval.txt

Will show an overview of the main evaluation parts of the current position.

5

root.txt

Will show the root moves of each iteration.

6

misc.txt

Programmer info of the finished search.

7

search.txt

Search overview since installation.

8

trace.txt

Debug a move sequence.

9

include.txt

Search interesting move(s) only, not all.

10

exclude.txt

Exclude move(s) from searching.

11

ProDeo.eng

Modify ProDeo settings.

12

pawns.txt

Modify Pawn Evaluation Tables.

13

table.txt

Modify several other Evaluation Tables.

14

pst.txt

Modify Piece Square Tables.

15

consult.txt

View how others (human and computers) think about the current position.

To refresh a page press F4

 

Additional information

book.txt

Switch from the old Spracklen (2 bytes) move format which was in use by many others (Genius, ChessMaster, TASC, Frans Morsch) to the (16 bytes) Polyglot format. Advantage, faster and easier to maintain, for instance with SCID for PC. The standand version comes with an opening book of 260.000 positions, the extended vesion with an opening book of 22 million positions.

 

eoc.txt

For a full explanation what the goals are of an EOC database check the old REBEL website. The standard version comes with a small database of 1.8 million positions created from games of the world champions of the last 150 years. In the extended version the database was created from Millionbase and contains 108 million positions.

 

eval.txt

Something like this every engine programmer must have. Numbers are in binary where 256 is 1.00 being one pawn. The final score, the sum of the evaluation parts is displayed in decimal.

 

search.txt

Gives an insight of the diminishing effect at which depth the engine no longer changes its mind about the best move. The statistic is split into 3 parts, statistics for the middle game, end game and late end game. The example given is made from self play matches at fast time controls.

 

trace.txt

Sometimes as a programmer you want to know what's going on in that giant chess tree of millions of positions, tracing the search can (emphasis added) help. The example text is the result of typing "e2e4 e7e5 g1f3 b8c6 d2d4" from the start position, press the save button and start analyzing, thereafter keep pressing F4 after each iteration for an update. Some explanatons:

 

Score - is the return value of the search after d2d4 is fully searched which will be stored into the TT.

No - is the place in the (sorted) move list.

ND - is the new modified depth by the various extensions and reductions.

 

include.txt

Another debug tool to check the reliability of the search. The negative side effects of modern search techniques are often amazing. Just 2 random examples:

 

r3qrk1/4bppp/4p3/p2pP2Q/1p1B4/1PpPP3/P1P2RPP/5RK1 w - - bm Rf6;

With a normal search it will take almost an half hour to find the winning Rf6 move, when we type f2f6 + save + analyze and thus only search Rf6 as move a winning score shows up quickly. And just when you might think that's ProDeo specific you may want to try this position with Stockfish 9, it needs a 37 ply search.

 

r1b1k2r/p2n1ppp/1p2p3/3p2B1/P2P4/1Nn1P3/5PPP/R3KB1R w KQkq - bm f3;

We type f2f3 (winning a pawn) + save + analyze and a positive score shows up after 3 seconds at depth 14. A normal search needs 18 plies and 1 minute.

 

exclude.txt

r1b1k2r/p2n1ppp/1p2p3/3p2B1/P2P4/1Nn1P3/5PPP/R3KB1R w KQkq - bm Bxh7+;

This is Kasparov - Deep Blue and (chicken) Kaspy did not dare Bxh7+ instead he played the solid a3. ProDeo does the same, after 16 plies a3 is still the best move but surprisingly has Bxh7+ as continuation! So what if we exclude a2a3 from the search? And so we type a2a3 + save + analyze and indeed Bxh7+ at iteration 16 shows up as best move.

 

So basically this function can be used as a NEXT BEST alternative.

ProDeo.eng

Engine parameters to change various aspects of ProDeo. For instance the evaluation, the search and file management. For an overview of its functions consider the ProDeo 2.6 Personality Creator which explains most of the parameters.

 

To make changes effective you need to restart the program.

pst.txt

pawns.txt

tables.txt

Modify specific evaluation tables. Also here counts you will need to restart the program before the changes are effective.

consult.txt

Contains the history of the current position from the human and computer point of view. Further explained in the update-2 part.

Standard version Approx 21 Mb

Extended version

Approx 837 Mb

Installation

 

To run ProDeo you will need to announce the engine first, under Chessbase this means:

Engine -> New UCI engine -> Browse -> RebelUCI.exe

 

Under ChessPartner do:

Extra -> Engine Import Wizard -> Winboard engines -> Install -> ProDeo.exe

 

For Arena:

Engines -> Install new engine -> RebelUCI.exe

_____________________________________________________________________________________________

 

Final remarks

 

1. Some Interfaces (like Arena) when launched won't load the current engine until it is

commanded something to do, no worries, Editpad will show up as soon the engine is

approached.

 

2. Like ProDeo 2.3, 2.4, 2.5, 2.6, 2.7 this is a new feature version only, no changes to

the engine itself.

 

3. Because of all the maintenance of the text files the engine might even lose some elo points in strength.

_____________________________________________________________________________________________

Added a new extensive feature in the spirit of this version, maximum information and presenting your engine as you wanted it to be, not limited any longer by the GUI it is running.

 

This update is about recording chess history and use it as an analysis (study) tool. It will consult a PGN database of 2.3 million (human-human) games and communicate the search results for the current position in your browser. Looking for a specific game? For instance the immortal Zurich (1953) Averbach-Kotov game? Copy the position into your favorite GUI and ProDeo will find the game instantly. Or Rubenstein's immortal Rxc3!! etc. etc. Games matching the position standard are stored in consult.pgn.

 

Like with EditPad your browser will only start up once at program start and it will warn you with one or two clicks if the board position is found in the database. Then refresh the browser page, in most browsers this quickly can be done by pressing F5.

 

Parameters are found at the beginning of the ProDeo.eng file in the Personal folder.

Parameter

Meaning

[Consult File = consult\human.pgi]

PGI databases are stored in the consult folder

[Consult Max = 10]

Number of games visible [maximum = 500]

[Consult Sound = on]

Turn on or off the warning clicks.

[Player One = ]

Explained below

[Player Two = ]

Explained below

The [Player One] and [Player Two] options are meant for opening study (preparation). Suppose you are Vishy Anand and the next tournament game is against Gary Kasparov, and you have a database anand.pgi (included in the download) where all your games are stored, you run:

[Consult File = consult\anand.pgi]

[Player One = Anand]

[Player Two = Kasparov]

And you will notice [ link ] that the results against Kaspy (only 36.7%) are not very florescent. Clear the [Player Two = ] and you will get the overall score of all the games of Anand [ link ] and with a 62.2% score that's the Anand we know.

In case you have a collection of your games in PGN you can make a PGI database yourself and use it in ProDeo 2.9. Steps:

 

1. Download, install and run SOMU 1.3 and make sure your PGN is in the sub-folder PGN.

2. Press F6 and select your PGN, the PGI file will be created.

3. Copy the PGN and PGI into the ProDeo Consult folder.

 

. With SOMU 1.3 you can create a PGI database from any PGN this way, even if that PGN only contains FEN strings.

. Use the EPD to PGN function to make an EPD set suitable for a PGI conversion.

Remarks

 

1.This function performs best in analysis mode since GUI's (tested are ChessPartner, Fritz and Arena) spur on engines somewhat different.

 

2. When you make changes to the PGN you will have to rebuild the

PGI index file again with SOMU.

 

3. On your right two downloads are offered, ProDeo 2.9b with a PGN of 2.3 million games and a light version ProDeo 2.9c with a smalll PGN database.

ProDeo 2.9b

Approx 876 Mb

ProDeo 2.9c

Approx 20 Mb

___________________________________________________________________________________________

All positions in the book are analyzed with Stockfish 10 with depth=20 which gave an improvement of 24 elo points.

 

Added a learning algorithm. Learning against itself gave an 49 elo improvement.

 

The 2 new features are valid for any Polglot book, it is explained in detail at the book page.

Move

Weight

Score

Depth

Learn

d2d4

51.60%

0.06

20

55

e2e4

41.96%

0.19

20

60

c2c4

4.13%

0.05

20

33

g1f3

2.31%

-0.01

20

0

The last episode (of this drama version) how I would like to see a Graphical Chess Interfaces (GUI) regale its users

with optimal information and create more facilities for engine programmers.

 

The Polyglot ProDeo.bin book has been extended with 2 new features.

 

Parameters in the ProDeo.eng tabulator

Parameter

Meaning

[Book File = books\ProDeo.bin]

Any valid Polyglot book

[Book-File = ]

Second Polyglot book

[Book Depth = 80]

Stop searching the book after move 40

[Book Learner = off]

on | off

[Book Score = off]

on | off

[Book Use = random]

best | random

The above are the default settings, play random for diversification.

 

. [Book Use = best] plays the move the highest percentage.

. [Book Score = on] plays the move with the highest Stockfish score.

. [Book Learner = on] plays the move with the highest learn score.

____________________________________________________________________________________________

 

The CONSULT tabulator

 

consult.txt contains the history of the current position from the human and computer point of view. A snippet:

Reference : consult\3000.dat

Positions : 10.963.142

Move

Depth

Score

Engine

e2e4

25

0.74

Stochfish 10

d2d4

28

0.50

Stockfish 9

e2e4

22

0.19

Komodo 11

d2d4

23

0.43

Komodo 11

The computer list with depths and scores contains only the results of single threated engines rated above 3000 CCRL. With almost 11 million positions it start making sense to maintain such a database.

 

Sources

. CCRL 40/40

. CEGT 40/20

. CEGT 40/120

. CEGT 5+3 PBON

. others, minimum time control of 15 seconds average.

 

Not supported yet

. TCEC

. CCC

 

Statistic per engine and how many available moves.

 

Parameters in the ProDeo.eng tabulator

[Comp File = consult\3000.dat]

Database location

[Display Max = 10]

Maximum = 195

Reference : consult\gm.dat

Positions : 2.999.013

Moves

Player

Game

e2e4 e7e5 g1f3

Caruana

Giri 1-0

d2d4 d7d5 c1f4

Vachier-Lagrave

Caruana 1-0

e2e4 c7c5 g1f3

Carlsen

Nepomniachtchi

e2e4 c7c5 g1f3

Anand,V

Kasparov,G ½-½

Database containing the games of the historic players throughout history plus the players current FIDE top-20.

 

Player statistic and how many available moves.

 

Parameters in the ProDeo.eng tabulator

[GM File = consult\gm.dat]

Database location

[Display Max = 10]

Maximum = 195

ProDeo 2.91

161 Mb