Leela Evolution

In the previous post, TCEC S17 L1 & CCC12 Bonus Series Ongoing, I mentioned a 'smaller net T59 10x128'. In my posts I sometimes include quotes that reference T40 etc., without explaining what the code means.

A similar question is What are the T40 and T60 in reference to Leela Chess Zero (Lc0) chess engine? (quora.com; October 2019; see also 'Related Questions'). Here's their explanation:-

T40 and T60 stand for iterations of the neural network [NN] that were trained from scratch. The Lc0 chess engine can work with either T40 or T60. T40 is the older NN architecture and T60 will supersede it.

The Lc0 NN is a Convolutional NN [CNN] with multiple layers called blocks. T40 was 20x256 (20 blocks, 256 filters); T60 is 320x24 (24 blocks, 320 filters).

Having more blocks and filters make the network deeper and wider and increase the possibility of it storing or leaning more chess knowledge and hopefully will let it play stronger chess. Training a much bigger network may fail because not enough computation may be available to fully take advantage of the bigger network. Many tests were done to make the best guess and assure success.

A chronology from the keepers of the candle is on What is Lc0? : For non-programmers (github.com/LeelaChessZero). Along with 'J Nets' (jhorthos), and 'Other Nets' (Leelenstein++), the page explains 'T Nets':-

• The previous generation, that is no longer being trained, is called T40.
• T50 is a set of small, lightweight nets that are used to test different training methods before applying those training methods to T60.
• The current (Dec 6, 2019) generation that is being trained is called T60.

The same resource, in Project History, also mentions jhorthos ('edited this page on 12 May 2019') and gives an evolution of the 'T Nets'. The page currently stops at T53. The switch T40 to T60 was announced on the Lczero blog in End of era (blog.lczero.org; July 2019):-

The era of test40 [T40] has finished, that training of test40 has been stopped. Instead we've just started to train test60 [T60]. What's new there (relative to test40): [...]

As the second-to-last post, it was also nearly the end of that blog. Too bad! Tnn, T-Nets, testnn; it's really all the same thing.