Practical Engine Tuning
The PET package is a ready to use platform (everything pre-installed) for engine testing (tuning) and consist 4 parts.
2 well known engines (Fruit 2.1 and Stockfish 5) will serve as an explanation what PET can do. Fruit 2.1 and Stockfish 5 both offer some tunenable UCI settings, these are:
History Threshold = 60
Mobility (Midgame) = 100
Futility Pruning = false
Mobility (Endgame) = 100
Futility Margin = 100
Pawn Structure (Midgame) = 100
Delta Pruning = false
Pawn Structure (Endgame) = 100
Delta Margin = 50
Passed Pawns (Midgame) = 100
Quiescence Check Plies = 1
Passed Pawns (Endgame) = 100
Material = 100
Space = 100
Piece Activity = 100
Aggressiveness = 100
King Safety = 100
Cowardice = 100
Pawn Structure = 100
Passed Pawns = 100
For both engines we are going to tune its mobility
Part-one : Gold Digging
We use the STS test suite that was developed during the years 2009-2010 by Dann Corbit and Swaminathan Natarajan containing 1500 theme based STS positions for the purpose of tuning engines and in particular the evaluation function. And with Fruit 2.1 we are going to explore its mobility. With MEA (pre-installed) we run STS at 5 different fast time controls and produce and overview.
Click picture to enlarge
The first table shows the sorted individual results of each setting for each time control used. The second table shows a sorted list of the overall rating by adding up the 5 gained ratings divided by 5.
We are now ready for step-2, the eng-eng testing part. And we are especially interested [mob=105] candidate improvement version and if it scores better against the official Fruit 2.1. Also if [mob=125] will do worse against the official Fruit 2.1 but let's test them all anyway.
Part-two : eng-eng matches
We use cutechess-cli (pre-installed) and pitch the official Fruit 2.1 against the various mobility versions above at 40/10, 1000 games and run the new overview.
As one can see from the results not only [mob=105] on the STS part tops but also after playing 1000 bullet games.
Stockfish 5 mobility testing
As a wild guess we changed the Mobility (Midgame) = 100 parameter to 125 and ran 2 matches, 4000 games at 40/10 and 1000 games at 40/60. Note the surprising results.
Click picture to enlarge
Part-three : Managing eng-eng matches
Keep track of your eng-eng match results. As one can see from the above picture PET offers you 2 time controls. Just store your match resullts in the tc-one.txt and tc-two.txt files, see pre-installed example. A more advanced (stand alone) system for match control based on eng-eng results we like to introduce now.
Part four : Match Control
LOS.EXE is a simple command line tool that will read the PGN of your matches and stores the overall information in a text-based database called LOS.TXT whereafter your browser will start and show you the results. When using the 6 Fruit and 2 Stockfish matches of this page we get:
Will produce the above web-page.
Will add test.pgn to the LOS.TXT database and show you the updated web-page.
LOS test.pgn 100
A third paramter will override the default setting of 1000 and produce the development of the match in steps of 100 games instead of 1000. It can very insightful to study the obstinacy how matches (which engines are so close in strength) may sometimes develop and why programmers play so many games. Consider an old ProDeo match as an example. It's only after 4500 games ProDeo X is starting to see the light.
Format of the LOS.TXT database
The most easy way the maintain the database is after adding a match to open LOS.TXT with a text editor and modify the first [Match] and second [Parameter] items to your wishes.
Fruit [mob=95] - Fruit 2.1,[Piece Activity = 95],1000,49.9%,46.8%,40/10,2018.06.20,mob-95.pgn
Just stick to the comma driven format.
While the LOS program is included in the PET download below you can download it seperately. Source code included for further development.
1. Install the archieve in the root of your C-drive (obliged).
2. Operation goes via pre-installed batch files, see the introduction on the next page.
3. Chess engines that offer a lot of parameter tuning are Deuterium, Rodent III and of course ProDeo itself.