Throwing the Book at AlphaZero

In last week's post, AlphaZero Match Conditions, I quoted from the Talkchess.com thread that dissected last month's update on AlphaZero, appropriately titled Alphazero news (p.20):-

by Laskos >> Tue Dec 11, 2018 12:23 pm • 'I don't remember what Cerebellum book lines are chosen by what UI, but using for SF8 a regular polyglot opening book like the small, but good BookX.bin and with Lc0 [Leela] without any book in Cutechess-Cli, I did get very varied openings. And a decrease of Lc0 strength of at least 50 Elo points compared to just playing from Initial Board position, but at short time controls. I think in Cutechess-Cli one has a random seed for a .bin book, but I don't remember well now.'

My immediate problem was to understand exactly what the knowledgeable Laskos was saying:-

Since that entire dialog is riddled with chess engine jargon, I'll come back to it in another post.

I picked eight terms from the paragraph that fall into my definition of 'jargon':-

Cerebellum, UI, SF8, polyglot opening book, BookX.bin, Lc0 [Leela], Cutechess-Cli, Elo points

Four of those terms are obvious enough that even I can understand them -- Elo points, Lc0 [Leela], SF8, UI -- but I needed help with the other four. My first stop was Opening Book (chessprogramming.org), which starts, 'Chess programs often look up the positions at the beginning of the game in an Opening Book', and gives three formats for the books:-

ABK - Arena's book format; BIN - PolyGlot book fomat; CTG - ChessBase book format

That explained two more terms -- 'polyglot opening book' and 'BookX.bin' -- about which more info is available from the same Chessprogramming wiki on its PolyGlot page (same source):-

An adapter that allows UCI engines to use interfaces and GUIs supporting the "Chess Engine Communication Protocol".

That left two terms to be explained -- Cerebellum and Cutechess-Cli. The first is related to Brainfish, the opening book that DeepMind let Stockfish use in the matches with AlphaZero. The introduction to Brainfish (same source as the two previous links) explains,

A Stockfish based chess engine introduced by Thomas Zipproth in July 2016, which contains a spin-off of the engine independent Cerebellum library, which keeps a self generated, persistent tree of minimaxed, deeply analyzed chess positions, used as playing or opening book with one or two best moves per node available but without score information. The initial Brainfish release of July 2016 consists of about 4.4 million positions, containing the most frequently played positions from engine games, human games, and some positions of rating list.

As for the last term, the software resides at Cutechess (github.com), which explains,

Cute Chess is a graphical user interface, command-line interface [CLI] and a library for playing chess.

Why bother learning about all of this? It goes to the heart of a point I covered in a post last year, Stockfish in a Straitjacket? (March 2018; 'four factors that could have hurt Stockfish's performance'), i.e. 'No opening book'. Even when Stockfish was allowed access to Brainfish, it got clobbered by AlphaZero, although by a less convincing margin.

A new question arises from this discussion. Given that AlphaZero has been trained only on games arising from the traditional start position (RNBQKBNR), how much of its strength is due to its familiarity with that position? How well does it play given a random chess960 start position? Even more importantly, how well does it play given a random position from a game between two chess masters?

The answer to these questions might determine how valuable a tool AlphaZero is for general chess analysis. I assume that achieving its great strength doesn't require it to be trained over millions of instances of an unfamiliar position.