23 November 2015

TCEC Season 8 Superfinal Week 3

My first post about the TCEC Season 8 Superfinal in Progress closed with the comment, 'It is, after all, showing us the future of chess.' My second post, Superfinal Week 2, noted that the then-current score, with Komodo leading Stockfish, was '+4-1=41, with 54 more games to be played'.

During the intervening week, another 29 games have been played, with a score of +1-0=28, the sole win again for Komodo. That gives an overall score of +5-1=69. At this point it's safe to conclude that the future of chess is a snoozefest.

What's going on? The following composite chart shows some basic TCEC statistics from game 74, although it could have been taken from almost any of the drawn games.

Top: Evaluation, Time usage, Depth
Bottom: Speed, Tablebase hits

The first graph ('Evaluation') shows White starting with an advantage of ~0.60 Pawns in the opening, eventually dropping to 0.00 in the endgame. The third graph ('Depth') shows the principal variation ('PV' in chess engine jargon) consistently at ~40 ply, i.e. 20 moves for each player. The fifth graph ('TB hits') shows the number of times an engine reached a six-piece position that can be looked up in a tablebase. TB hits were registered before move 10 was reached in this game.

In other words, the pattern of a typical game is start with a position that favors White, trade off most of the pieces, reach a known draw. How much of this is caused by the choice of opening? The 'Opening Book' tab on the TCEC Archive Mode page informs,

Stage 1 will be bookless: every engine will compete from the starting position. We’re not worried about repetitive openings; with a good mix of engines we don’t expect that to be much of a problem.

Stage 2 [...] will have a double round-robin format, but this time we will use a two-move book

Stage 3, the qualifying round for the Superfinal, will revert to the format we’ve used in recent seasons: an eight-move book

Superfinal [...] will use 50 positions selected by our guest IM Erik Kislik, of which 33 are his own and the remaining 17 are made by [Nelson Hernandez, aka Cato].

This season, as you can see, we’ve decided to cover more numbers on the roulette table. Bookless, short-book, medium-book advocates should all be satisfied that their cause is at least being represented and that the responsibility for selecting these positions is distributed among multiple individuals, thus mitigating possible biases.

For the superfinal, possible opening biases have been 'mitigated', except perhaps the bias of the experts doing the selection. Chess engine competitions, whether man-machine or engine-to-engine, have always been skewed by the unseen human hands creating the engine's opening book. Isn't there another way? How about offering the engine an incentive to take a risk that a human wouldn't take.

No comments: