Evaluating the Evaluations

In the most recent post on my chess960 blog, CCC C960 Blitz Championship (October 2021), which was itself a crossover idea from this current blog, I ended the post saying,

Given that engines' evaluations for every move are available in the event's PGN game scores, perhaps there is something to be learned about the 960 different start positions. That investigation would make a good follow-up post.

My first problem was to learn how to find my way around the PGN files available from the TCEC and the CCC. I went back to TCEC Season 21 -- reported on this blog a few months ago in TCEC Stockfish Wins S21; CCC Romance Continues (August 2021) -- then downloaded the PGN file containing the 100 games played in the match and loaded it into a normal database, i.e. not a chess database. After an hour or two sorting out a few technical problems, I had a working tool for further exploration.

Of the 26 decisive games played in the S21 Stockfish - LCZero final match, only one was won by Black : game 85. The same opening with colors reversed, game 86, was drawn. All of the games in the match started with openings dictated by the organizers.

Games 85 & 86 used an opening called the English ultra-symmetrical variation, where the initial moves were 1.c4 c5 2.g3 g6 3.Bg2 Bg7 4.Nc3 Nc6 5.a3 e6 6.b4 Nxb4 7.axb4 cxb4 8.Nb5. The TCEC pages for the two games display the following charts showing both engines' evaluations during the course of the games.

In game 85, both engines calculated an advantage for Black that increased steadily. In game 86, the initial advantage for Black slowly evaporated.

I once based an entire post on detecting problems with the evaluations: TCEC Season 8 - Evaluation Anomalies (December 2015). It took me as long to identify those anomalies as it did to create the database for the current post.

Now that I have a tool for rapidly evaluating the engine evaluations, what can I do with it? The first task will be to put it to work on the 960 start positions used in chess960.