Practical Engine Tuning

Practical Engine Tuning

for starters

The PET package is a ready to use platform (everything pre-installed) for engine testing (tuning) and consist 4 parts.

Gold digging (hunting) for candidate improvements in a fast way;

The actual testing of those candidate improvements playing eng-eng matches;

Managing PET eng-eng matches.

Match Control.

2 well known engines (Fruit 2.1 and Stockfish 5) will serve as an explanation what PET can do. Fruit 2.1 and Stockfish 5 both offer some tunenable UCI settings, these are:

Fruit 2.1

Stockfish 5

History Threshold = 60

Mobility (Midgame) = 100

Futility Pruning = false

Mobility (Endgame) = 100

Futility Margin = 100

Pawn Structure (Midgame) = 100

Delta Pruning = false

Pawn Structure (Endgame) = 100

Delta Margin = 50

Passed Pawns (Midgame) = 100

Quiescence Check Plies = 1

Passed Pawns (Endgame) = 100

Material = 100

Space = 100

Piece Activity = 100

Aggressiveness = 100

King Safety = 100

Cowardice = 100

Pawn Structure = 100

Passed Pawns = 100

For both engines we are going to tune its mobility


Part-one : Gold Digging

We use the STS test suite that was developed during the years 2009-2010 by Dann Corbit and Swaminathan Natarajan containing 1500 theme based STS positions for the purpose of tuning engines and in particular the evaluation function. And with Fruit 2.1 we are going to explore its mobility. With MEA (pre-installed) we run STS at 5 different fast time controls and produce and overview.

Click picture to enlarge

The first table shows the sorted individual results of each setting for each time control used. The second table shows a sorted list of the overall rating by adding up the 5 gained ratings divided by 5.

We are now ready for step-2, the eng-eng testing part. And we are especially interested [mob=105] candidate improvement version and if it scores better against the official Fruit 2.1. Also if [mob=125] will do worse against the official Fruit 2.1 but let's test them all anyway.


Part-two : eng-eng matches

We use cutechess-cli (pre-installed) and pitch the official Fruit 2.1 against the various mobility versions above at 40/10, 1000 games and run the new overview.

As one can see from the results not only [mob=105] on the STS part tops but also after playing 1000 bullet games.


Stockfish 5 mobility testing

As a wild guess we changed the Mobility (Midgame) = 100 parameter to 125 and ran 2 matches, 4000 games at 40/10 and 1000 games at 40/60. Note the surprising results.

Click picture to enlarge


Part-three : Managing eng-eng matches

Keep track of your eng-eng match results. As one can see from the above picture PET offers you 2 time controls. Just store your match resullts in the tc-one.txt and tc-two.txt files, see pre-installed example. A more advanced (stand alone) system for match control based on eng-eng results we like to introduce now.


Part four : Match Control

LOS.EXE is a simple command line tool that will read the PGN of your matches and stores the overall information in a text-based database called LOS.TXT whereafter your browser will start and show you the results. When using the 6 Fruit and 2 Stockfish matches of this page we get:

Click picture to enlarge or view in browser.

Usage from the command line


Will produce the above web-page.

LOS test.pgn

Will add test.pgn to the LOS.TXT database and show you the updated web-page.

LOS test.pgn 100

A third paramter will override the default setting of 1000 and produce the development of the match in steps of 100 games instead of 1000. It can very insightful to study the obstinacy how matches (which engines are so close in strength) may sometimes develop and why programmers play so many games. Consider an old ProDeo match as an example. It's only after 4500 games ProDeo X is starting to see the light.

Format of the LOS.TXT database

The most easy way the maintain the database is after adding a match to open LOS.TXT with a text editor and modify the first [Match] and second [Parameter] items to your wishes.


Fruit [mob=95] - Fruit 2.1,[Piece Activity = 95],1000,49.9%,46.8%,40/10,2018.06.20,mob-95.pgn

Just stick to the comma driven format.

While the LOS program is included in the PET download below you can download it seperately. Source code included for further development.


General remarks

1. Install the archieve in the root of your C-drive (obliged).

2. Operation goes via pre-installed batch files, see the introduction on the next page.

3. Chess engines that offer a lot of parameter tuning are Deuterium, Rodent III and of course ProDeo itself.



Ferdinand Mosca

for MEA

Tord Romstad, Marco Costalba and Joona Kiiski

for Fruit 2.1

Ilari Pihlajisto, Arto Jonsson

Dann Corbit, Swaminathan Natarajan

for STS

Fabien Letouzey and others

fot Polyglot

Aprox 8½ Mb