Utilities

Some of my utils

About 5.5 Mb

Source code included

EPDSETS is a command line tool to create EPD Opening Sets for eng-eng Matches. It uses a database of 174.000+ opening positions created from the openings as used by CCRL and CEGT during the 2006-2017 period. The database of 174.000 positions is analyzed by Stockfish 9 in order to create selective opening EPD sets based on score.


Functions

File

Select EPD file. Default: database.epd

Overview

Score overview database, see example.

Split

Split the database in parts of (any number)

Evaluate

Create EPD opening sets based on score.

Remove

doubles

Is an extra, removes doubles in EPD sets.

Score overview, number of EPD 174707
 0.00 | 0.10 | 0.20 | 0.25 | 0.30 | 0.40 | 0.50 | 0.75 | 1.00 | >2.00
29048 |24196 |11862 |11652 |21844 |19159 |33242 |15197 | 8257 |  250

_____________________________________________________________________________________________

SOMU 1.3

SOMU is a command line utility operated by the arrow keys. It contains 6 PGN and 8 EPD tools. PGN files are found (stored) in the PGN folder, likewise EPD files in the EPD folder. It's functions:


The PGN part

Match Stats

From a PGN create and (automatically) view a HTML file, example-1 is created from gambit.pgn (provided in the download) taken from the YAT gambit tournament. Example-2 is created from a CCRL pgn testing Komodo 12.


Match Compare

Check the PGN of an engine-engine PLY-Match, games should be indentical else perhaps something is wrong. Different moves are marked and stored in OUTPUT.PGN Background: it's extremely important (especially for starters) that a PLY-MATCH (say PLY=10) produces the exact same reverse game, so game-1 and game-2 in the PGN should be 100% equal, game-3 = game-4 and so on.


Cross-table

This function will create a cross table from a (large) PGN file. Maximum number of players (human and/or engine) is 100. Example taken from the gambit.pgn mentioned earlier. Another example, the final FCP rating list by Frank Quisinsky.


Strip PGN

Strip played moves by engines from a PGN. What is left and stored in OUTPUT.PGN are the book moves. Handy for opening book tuning with Polyglot.

Split PGN

Split large PGN files into smaller parts.

Remove Double Openings (games)

From a PGN collection display the match score, then remove the double openings and calculate the new match score without the double openings. This util is handy in 3 ways:


. checking an eng-eng match for opening doubles, see screenshot (file bf.pgn)

. pre-checking your predefined opening sets for eng-eng matches.

. In case a PGN doesn't contain comments it will check for duplicated games.




The EPD part

Remove Doubles

Remove doubles from an EPD collection.

EPD to PGN

Converts EPD to PGN, bm and ce tags included when present. Try this with the included 150 positions example.epd file.

Split EPD

Split large EPD files into smaller parts.

EPD Stats

Create a statistic from an EPD collection. It also will create 10 separate EPD files:

. Queen | Rook | Bishop and Knight promotions

. Possitive Mate in X moves | Negative Mate in -X moves

. Pawn Endings | Bishop Knight endings | Rook endings | Queen Endings

Mirror EPD

Reverse (mirror) an EPD collection. Output stored as OUTPUT.EPD

Analyze EPD

Analyze an EPD collection with Stockfish 8, 9 and higher. Example before and after.

Analyze EPD ++

Analyze an EPD collection with Stockfish 8, 9 and higher splitting the analyzing over a user defined number of threads for speed reasons. Example before and after.


The advantage of Analyze EPD and Analyze EPD ++ is that it will work with every new version provided the Stockfish team doesn't change the bench function. Just copy a newer Stockfish version in the SOMU folder.


Current default version is SF10x a beta version considerable stronger than SF9, see results.

EPD legal check

Remove invalid positions from an EPD collection. It's highly recommanded before running large EPD sets with Analyze EPD or Anylyze EPD ++ to check an EPD file first to avoid SF to crash because not seldom EPD sets created from a PGN database may contain garbage positions.



Hidden features


There are a number of hidden features under the function keys


F1

Delete bad or too good EPD positions controlled by a flexible score margin.

F3

Quickly count the number of games in a PGN.

F6

Create PGN index for the use in ProDeo 2.9a

Remark - The PGN part is compatible with Arena and Cutechess-cli, the gibberish ChessBase PGN's work for the most part except for depth and time statistics.

Credits

Tord Romstad, Marco Costalba and Joona Kiisk for Stockfish.

4.6 Mb

_____________________________________________________________________________________________


SONU 1.0

13.2 Mb

Included is an example PGN taken from the gambit tournament which may serve to get familiar with the 9 utilities.

SONU is equal in layout and operation as SOMU above and consists a selection of 9 of the 46 PGN utilities written by Norman Pollack now made more easy to operate.


PGN's are expected in the PGN folder, all new created PGN's are stored in the OUT folder.

Trim

Produces an output pgn file that puts each 'full move' on the same line while also removing comments.

ResultTable

Creates a sorted list of players based on points with the S-B (Sonneborn-Berger) value as tiebreaker.

PairSplit

Collects and stores games for each pair of players in a separate PGN file called box00.pgn, box01.pgn etc. with a maximum of 100 pairs.

GameSplit

Split (large) PGN files into x smaller parts.

NameExtract

Extracts games in which a player name begins with a user-specified name_search-string.

NameChange

Performs user-specified player name changes based on a user-created text file named 'changes', see example given or the README for details.

MinOccur

Extracts games of players who meet or exceed a user-specified minimum number of games.

Pctlist

Lists player names, the number of games, the number of points and the the score percentages. The list is sorted in descending order by percentage.

Summary

Produces statistics involving the number of games, the number of players, dates, results, Elo ratings, plycounts and ECO's and more.

README

Detailed description of the above utils.

Homepage

Homepage Norm Pollack, get all the 46 PGN utils and more.

Release info

This page.