30 August 2019

Playing Without a Horizon

After a few digressions for the ongoing Deepmind/AlphaZero saga (last seen a week ago in A Chess Epiphany), let's return to AlphaZero's Zeros (July 2019). After a few excerpts from Sadler & Regan's book 'Game Changer', we left GM Sadler lamenting the 0.00 evaluations offered too often by the traditional A/B (alpha/beta) engines like Stockfish. He wrote,

It’s perhaps the most irritating and obstructive thing that these otherwise fantastic engines do during analysis.

I would add that their habit of trading off many active pieces in mass exchanges is just as irritating. Could it be that the 0.00 evaluations and the mass exchanges are somehow related? Later in the same chapter, GM Sadler continues his discussion:-

The meaning of these 0.00 evaluations is something puzzling. At the point when the possibilities should be boundless, a draw by repetition is apparently the best option. However, if you attempt to take on the engine and prove it to be wrong, then you often end up losing... or discovering more repetitions.

That's a good statement of the issue, which is then illustrated by positions taken from published games between AlphaZero and Stockfish:-

We’ll examine a series of positions in which AlphaZero’s evaluation and that of Stockfish and other top engines varied significantly. In each position, we will try and understand the reason for the evaluation of both sides, attempt to assess the position (both objectively and from the human perspective) and also attempt to assess what effect the evaluation might have on subsequent play.

At this point, the book launches into a brief discussion of a key difference between the two engines : how they calculate evaluations as a single number. I covered this a few weeks ago in Winning Percentage to Centipawns.

Sadler & Regan give three examples of positions where AlphaZero has a more nuanced evaluation than the A/B engines, which all return 0.00 evaluations for their main lines. The first position resembles a typical position from the family of openings known as King's Indian Attack, where AlphaZero uses an atypical idea.

The second position is shown below. AlphaZero is playing White.

After 55.Bf2-e3

The five A/B engines used in the example all consider three lines...

55...bxa3 56.bxa3
55...Re8 56.Bf4
55...Qd7 56.Bf4

...where all of these lines result in a 0.00 evaluation. In the published game, Stockfish played 55...bxa3, which AlphaZero answered with 56.Ka2 (Sadler: 'a total shock for me'), sacrificing a Pawn. AlphaZero went on to win the game. Sadler summarizes the third (and last) example of a 0.00 position as 'another epic struggle':-

Unfortunately, there is a period in the middle in which AlphaZero seems determined to enjoy its position without taking immediate action, which makes it rather long! [...] My feeling when playing through the game was that Black was making steady progress. I felt that AlphaZero was playing well, but I felt that the main progress was due to needless concessions from Stockfish.

All three example positions are analyzed elsewhere in 'Game Changer' within the context of the full game. For me, it's obvious that AlphaZero is often playing beyond Stockfish's horizon. I regularly see the symptoms of this when a strong A/B engine plays a weaker A/B engine. The act of reducing a complicated chess position to a single number is fraught with danger.

No comments: