Chess Piece Recognition

After that digression for kudos to the Chess Programming Wiki (CPW), let's return to the series last seen in The Limits of Image Recognition (June 2018). A problem particular to computer chess is to recognize the relevant portions of a chess board. CPW has a page on Piece Recognition, which it defines as:-

Piece Recognition, (Chess Board or Chess Position Recognition) • The ability of dedicated chess computers or chess playing robots to automatically recognize all the pieces on a chessboard, or in computer vision to convert an image of a real chessboard with pieces, or a chess diagram into a machine readable format specifying a chess position, such as Forsyth-Edwards Notation (FEN) or Extended Position Description (EPD).

Computer Vision • Piece recognition is an interesting topic in computer vision, machine learning and pattern recognition using one or more cameras along with digital image processing and object recognition, more recently supported by deep learning techniques as demonstrated by Daylen Yang with his Chess ID project.

We've already seen Daylen Yang on this blog in How Stockfish Works (September 2015; 'Daylen Yang, University of California at Berkeley'). That video introduced a long series which meandered through different topics until Chess Engine Summary (January 2016).

The CPW page on piece recognition points to a Daylen Yang article, Building Chess ID (medium.com; January 2016). Here is a summary of its main points, all taken from that article.

The first step to identifying chess pieces from a picture of a board is to detect the board and segment it into 64 little squares. [...] The next step is to identify the chess piece on each of the 64 squares.

Deep convolutional neural networks like GoogLeNet are pretty good at image recognition. What we can do is take a neural network that’s been trained on the ImageNet dataset and fine-tune it with our own data and labels. This process is called transfer learning. For this to work, though, I would need a ton of data: images of chess pieces along with their labels.

The pieces and their labels are '10,000+ images' structured something like the following chart.

The 'Chess ID' article continues,

With data in hand, it was time to train. Using the Caffe deep learning framework and starting with a pre-trained AlexNet (winner, ILSVRC 2012), the fine-tuned neural network was achieving 99% accuracy on the test set in no time. [...] I’ve made the data and the final model available on GitHub, so you can train your own models or deploy your own server.

The GitHub page is GitHub - daylen/chess-id: Board localization and piece recognition. Kudos to Daylen Yang for applying deep learning techniques to a small but important application involving chess.