Testing an Engine

After posting Fishing on the River Chess, it took me a few days before I found the time to install the chess software that I had downloaded. I started with Komodo, the engine that won the Thoresen Chess Engines Competition (TCEC; Wikipedia) a few months ago.

Starting with Komodo meant I had to plug it into the previous version of Arena. This suited me just fine because I don't like grappling with too many new unknowns at the same time. It took me some time to rediscover the parameters that require tailoring in Arena, but after a few minutes everything was up and running as expected.

For a history of the engine, see Komodo on chessprogramming.wikispaces.com. For some inside info from one of its developers, see Q&A with Larry Kaufman. Last year I ran a series largely based on GM Kaufman's work, Practical Evaluation, and know that he is one of the best in the field of chess programming.

I was a bit concerned when I noticed that the engine seemed to run considerably slower than Houdini, the engine I'm using currently. I found a plausible explanation on the same chess programming wiki under Nodes per second.

Whereas comparing different version of the same program is fine, comparisons between different engines are more difficult, since programmers use different schemes of counting nodes.

I also encountered a few tuning parameters I hadn't seen before. These are explained on the same wiki under Late Move Reductions (LMR) and Null Move Pruning. I decided to leave them alone, again so as not to introduce too many new unknowns. A discussion of the concepts by Don Dailey, another Komodo developer, can be found on a forum thread, Relationship between move ordering and pruning.

Over the next few days I'll run some comparison tests between Houdini and Komodo on a few of my current games where engines are allowed. Then I'll decide which engine will be my primary tool.