STS revisited

Analyzed with SF15

revisited

Analyzed with LC0

STS stands for Strategic Test Suite. From the CPW we read - Strategic Test Suite, (STS) a series of themed test suites by Dann Corbit and Swaminathan Natarajan designed to evaluate chess engine's long term understanding of
strategical and positional concepts. More recently, the positions were revised and converted by Ferdinand Mosca to be used by an analysis tool, MEA.


Created in 2008 and containing 1500 positions in total. Each position was analyzed with the then strongest engines giving points for the four best moves varying from 1 to 10 points. Examples:


1r6/1R2bk2/P3p1pp/4Pp2/1r1P3P/1N2B1P1/5PK1/8 w - - bm Rxb4; c0 "Rxb4=10, Rxb8=9, Na5=7, Nc5=2";
1r6/3n4/p2b3p/2nN1kpP/1BP1p3/3rP3/3NKP2/1R1R4 w - - bm Bxc5; c0 "Bc3=10, Bxc5=8, Rdc1=0, Re1=0";


As such people were able to more or less create their own rating list based on these 1500 positions, not as good the real rating lists, but not bad either. Anno 2022 the set is completely outdated, current top engines play ~500 elo stronger than those in 2008.


We made an effort to give STS a major update by re-analyzing the 1500 positions with nowadays strongest engines, Stockfish 15 and Lc0. And with MEA you can produce your own alternative rating list. See -


[ SF15 ] [ LC0 ] or as text file [ SF15 ] [ LC0 ]

___________________________________________________________________________________________________


INSTALLATION

The STS package comes in 2 folders, STS-SF15 and STS-LC0, the 1500 positions in each folder are analyzed at 15 seconds per move, SF15 using 4 threads and Lc0 using version 28.0 with the advised 130Mb net for that time control.


OPERATION

Operation goes via the provided sts.bat file. For a first smell double click on sts.bat and it will run the 6 pre-installed engines and produce a html (automatically updated in your browser) and a text file mea-result.txt for posting results.


Adding new engines - open the file sts.bat with a text editor, you will see -


set MT=100
set HASH=64
set THREADS=1
set PROTOCOL=uci
set EPD=epd\sts-sf15.epd
set MRL=mea_results


set EXE=engines\Koivisto_8.0.exe
set NAME=Koivisto_8.0


set MT stands for "move-time", here you can change the average time in milliseconds, 1000 is one second.

set EXE defines the place of the engine, engines typically are stored in the engines folder.

set NAME is the name you want the engine named in the output.


Add as many engines as you wish in one run, see the 6 engines example.


Hints

1. Don't change the blue listed paramter unless you know what you are doing.

2. Several engines don't handle low time controls (such as 100ms) well, resulting in bad results. Advice use 1000ms.


Delete engines - results are kept in the file mea-results.csv and here you can make changes or delete engine results.


make-html.bat - double click for new html and text output, for instance after you have made changes to mea-results.csv.

Credits

Ferdinand Mosca

The programmers for the fair use of their engines