In the 15th Superfinal of the Top Chess Engine Championship (TCEC) – the unofficial computer chess World Championship – the self-learning Leela Chess Zero beat the previous best engine in the world, Stockfish 10. In a 100-game match (two hours per player, with a 10-second increment) Leela emerged victorious 53.5:46.5 (+14 =79 -7). In the qualification for the final it was still Stockfish 10 that had the edge, but in the final Leela was for the first time crowned the undisputed champion! Has this opened up a new age in computer chess?
What is the TCEC?
For years now the Top Chess Engine Championship has seen competition between the world’s best chess engines. It’s a league system with promotion and relegation, where engines that are improving gradually get to start in higher divisions. The higher the league, the longer the thinking time.
In the top league, the Premier Division, the eight best engines first play each other six times. After a total of 42 rounds the league is over – but not yet the season! The two chess engines that finished 1st and 2nd play each other in a clash of the titans - a duel fought out over no less than 100 games.
The career of Stockfish
This chess engine developed by Tord Romstad and others is probably known to almost all chess players. Its first version was developed in 2008 on the basis of the program Glaurung – an engine that Romstad had also developed in 2004. The dominance of Stockfish in recent years can clearly be seen from its placements and results in the TCEC:
One thing is therefore clear: in recent years there was basically no getting around the Norwegian fish! Just as in human chess, Norway was alone at the top. In Superfinals 11-13 Komodo and Houdini didn’t have the slightest chance and the matches were soon over as contests. It was only in the last season that Stockfish came up against an equal, an opponent it could only beat by the narrowest of margins: Leela Chess Zero.
The career of Leela Chess Zero
Leela Chess Zero would never have appeared in its current form without the much-hyped competition between AlphaZero and Stockfish 8. AlphaZero, the algorithm developed by Google’s DeepMind, came from nowhere with the announcement that it had beaten Stockfish 64:36, with 28 wins to its opponent’s 0. It was hard to avoid seeing it as a revolution. Without any chess knowledge except the rules of the game, it took AlphaZero four hours of playing against itself to be able to beat the best engine in the world. Even if at the beginning there was loud criticism (critics said Stockfish had been given unfair conditions), there was no denying the reality. A second, more rigorous 1000-game match ended with AlphaZero winning 574.5:425.5, and further tests showed clear domination.
Some of AlphaZero’s wins made a big impression, and particularly the way it treated the Queen’s Indian. There AlphaZero was very happy to give up a pawn as White for long-term compensation and the chance to hinder its opponent’s development and push Black's pieces to the edge of the board. Magnus Carlsen also recently pointed out in Lindores that we humans still have a lot to learn:
Looking at computer games it’s clear that we still have a very long way to go when thinking about long term compensation and such things, because simply we misjudge positions and we draw our conclusions too early. It’s not clear exactly how you can improve these things, but it’s very, very clear to see that we’ve only still scratched the surface of what is possible to do in chess, because we are human and we make mistakes.
Jan Gustafson analysed AlphaZero’s Queen’s Indian play from a theoretical point of view and Svidler even chose one of the games as one of his best games of 2017.
What does this have to do with Leela? It’s quite simple: its development began as an AlphaZero “clone” based on the information published by DeepMind. The programmers Gary Linscott (also involved with Stockfish) and Gian-Carlo Pascutto launched the open-source project Leela Chess Zero. Just as with AlphaZero, Leela originally knew only the rules of chess (zero stands for the prior knowledge implemented apart from the rules). For example, they didn’t teach it that in chess it’s important to control the centre, or ensure king safety, or that a queen is stronger than a pawn. Leela learnt which moves work and which don't simply by playing against itself. The basis for that was the MCTS (Monte-Carlo Tree Search), which roughly speaking means simply exploring the tree of variations at random at the start of the learning process. Later a neural network would begin to have an intuition for which moves were more promising.
Volunteers were able to “donate” the computing power of their computers during the learning process, enabling Leela to play and learn. The interesting thing is that after only a few months of training Leela was as strong as a human grandmaster. How fast Leela improved can be seen from its results in the TCEC:
What’s the difference between Stockfish and Leela?
Stockfish and Leela are, as we’ve already seen, totally different engines. That can most clearly been be seen in the evaluation function.
Stockfish and all the other “conventional” engines rate a position in term of pawn units. For an engine to do that there are various commands and rules in the program’s code, all programmed by humans themselves. When Stockfish evaluates a position, simply speaking, it doesn’t do anything other than note all the factors of a position, assign them the appropriate (human-determined) value in pawn units and then sum up all the values. The sum of all the positional factors is what we get when we see +0.73, 0.00 or -4.91. As humans we know at least that: White seems to have an advantage at +0.73, while at -4.91 Black is probably about to win (if there’s no fortress).
For Leela things are completely different: it doesn't work in pawn units. It selects legal moves from the starting position and “simulates” the likely outcomes of games. In addition, it also tries out moves that have not yet been tried by this random procedure but which have worked well in similar positions in the past. So you could say that Leela, like AlphaZero, chooses moves based on a semi-random system and, like humans, concentrates on moves that often work. We humans know that it’s typically not good to move the pawns in front of our own king or to sacrifice the queen for a single piece. That’s why we generally don’t do it – and that’s a good thing!
Of course this has a natural consequence for the way in which positions are evaluated: because Leela doesn’t know pawn units, but only possible game outcomes, Leela gives its evaluation of a position as the probability with which it believes it can win the current position. The technical notes to Leela Chess Zero also explain how Leela’s odds of winning can be approximated to classical “pawn units”. As an example, Leela’s expected win rate would increase from 0.5 (draw) to (0.75) if the pawn-unit rating rises from 0.00 to 1.11.
Season 15 Superfinal
But now for the final and the games played there! All 100 games in the match can be played through using the selector below:
As mentioned, Leela Chess Zero won the 100-game match against Stockfish 10 with a score of 53.5:46.5 (+14 =79 -7). That was astonishing since just a few months previously, in Season 14, it had narrowly lost against the favourite 49.5:50.5. Since then, however, Leela was able to boost its rating from 3404 to 3589 for Season 15. Leela even went into the Superfinal as a slight rating favourite, since Stockfish was rated “only” 3587. We could expect a close and exciting battle, since Stockfish (27.5/42) had clearly won the Premier League before that with a 2-point advantage over Leela (25.5). Komodo was a distant third on 21.5. You can find the games from that event here.
At the start the match developed very evenly. After 9 draws in a row it was Leela who drew the first blood. However, the lead after that win in Game 10 didn’t last long as Stockfish hit back in Game 12, when Leela’s attempt to go all out to win a game with an extra exchange went badly wrong! But with wins in Games 16, 18, 24 and 26 Leela built up a lead that it would never surrender. The final position of Game 16 in particular is a good example of why Leela can be regarded as AlphaZero’s little sister:
The position is reminiscent of Game 63 of the 14th Superfinal, where Leela also conjured up a crushingly dominant position against Stockfish’s French:
The king goes it alone
Even Stockfish’s fight-back with a win in Game 35 would be negated by Leela winning in the 36th and 38th games. Let’s take a quick look at those games, as they illustrate very well where Leela’s strength lies. In Game 36 after 47…Nb5+ the following position arose:
Stockfish has just given a check on b5 and was expecting the king to come to a4. Retreating with 48.Ka2 or 48.Kb2 looks more human, and you might prefer a2 so as not to risk becoming a target for the fianchettoed bishop on g7. But after that it also seems hard for White to make concrete progress, because the black knight blocks any entry squares on the queenside and it doesn’t seem to be going anywhere - the bishop on f2 is the wrong colour to be exchanged for Black’s blockading knight.
Leela therefore decided to go for the interesting positional exchange sacrifice 48.Rxb5!? That move not only removes the annoying knight from the board but also creates a dangerous passed pawn on the a-file, which will be difficult to stop since it’s supported by Nc6, Bf2 and potentially a heavy piece.
After 48…axb5 49.Rc2 Qb7 (diagram) we reach the next critical position. Black is perhaps threatening a counterattack on the white king with 50…Ra6 and 51…Rfa8, but rather than taking the threat out of the position Leela just ignores it with the cheeky 50.Kb4! That doesn’t only cover the pawn on a5 but also asks Black how he’s planning to defend his b5-pawn. After 50…Bf8 51.Qd3 Kg7 52.Qxb5 the pawn had already left the board, but of course Leela was thinking of more than a pawn - she went for a grandstand finish! After 52…Qd7 53.Ka4 Qh3 54.Qe2 Qh1 55.b4 Kf7 (diagram) the time had come to reap the rewards:
With 56.Kb5! the white king was “out of harm’s way”... The immediate 56.a6?, on the other hand, would not have been
good at all due to 56…Qa1+ 57.Ra2 Rxa6+! and White would have spoiled all the
hard work. But if you play as accurately as a computer everything drops into
place. After 56…Be7 57.a6 h6 58.a7 hxg5
59.hxg5 Bxg5 White now played 60.Ka6,
since the b-pawn must still join the race. After 60…Qh3 61.b5 Qc8+ 62.Ka5 Bd8+ 63.b6 it was sad enough for Stockfish
to give up a piece with 63…Bxb6+,
since the white passed pawns would otherwise have stormed forwards with even
more devastating consequences. The rest was relatively easy for an engine
of Leela’s class and White won after 89 moves.
In Game 38 we could admire a similar scenario. After a relatively quiet opening Leela accepted isolated doubled pawns on the d-file to leave Black's bishop locked out of the game. When the chance to sacrifice the exchange arose, Leela didn’t need to be asked twice!
White is doing well after 41…Rb8 (diagram) since he controls the a and c-files, but a real breakthrough is not visible at first. Black has counterplay against the white king with an annoying battery on the f-file and also the resource Nf3+, that White always has to have an eye on. Leela solved both problems with the surprising 42.Rxd4!? This not only takes the annoying knight off the table, but also fixes the problem of king safety with 42…exd4 43.f4!
Stockfish had expected only 42.Nxd6, but after 42…Qxf2+ 43.Qxf2 Rxf2 44.Kh3 it’s not so clear how White can bring home the advantage. After 44…h5!? the white king might even be in danger, but also 44…Rd2 asks a few questions. Leela’s approach is almost reminiscent of the World Champions Petrosian and Karpov at their best!
After Leela’s exchange sacrifice the counterplay against f2 has gone and White is threatening, with 44.Re6 followed by 45.Qe2 and 46.Qg4, to put the black king in the firing line. After all, the dark-squared bishop hardly helps in the defence, because White’s attacking bishop moves only on the other squares. The knight on e4 is untouchable, at least by the bishop on g7.
Stockfish tried to defend itself with 43…Ra8
44.Qb3 Qe8 (diagram). Black might have wanted to bother the white king with
moves like Ra5 and Qa8, but Leela anticipates that with 45.Kh3! The king also goes forward again, away from the danger
threatening him on the second rank. Stockfish seems to find nothing against the
attacking plan 56.Be6 57.Qe2 58.Qg4 and 59.Rc7, and decides to burn all its
bridges: 45...Ra5 46.Rb1 Qa8!? 47.Nxd6
Ra1 48.Be6 Bf6 49.Ne4 h5 50.d6 Bd8 51.Kg2 Rxb1 52.Qxb1 Qa3. But Leela had calculated
everything correctly and was in control of the situation: 53.Bd5 Kg7 54.Bxb7 Rf7 55.Bd5 Rd7 and now after 56.Qc2 Stockfish gave back the exchange
and was left with an impossible defence.
With Stockfish trailing badly in the match it was clutching at straws, and there was a glimmer of hope with the opening chosen in Games 43 and 44. Jeroen Noomen is responsible for the opening positions played in the TCEC, and for the 15th Superfinal one position he chose was the one after 1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Ng5 d5 5.exd5 Nxd5?. Black's last move is considered valid after both 6.d4! and 6.Nxf7!?, but in this position it's highly dubious, which is why almost all players with Black here go for 5...Na5. After the move in the game White gets a strong attack on the black king, even if it costs material. While Stockfish eventually won with White, Leela even seemed to have managed to lose!
In the end, though, the result was changed to a draw, since there had been a technical issue with the clocks and the final position is in any case a tablebase draw.
The moment the match was clearly going to go Leela Zero's way came in Games 61 and 62. The opening was the Trompowsky, and on move 18 the engines diverged completely in their evaluation of the structure Stockfish chose with White. Stockfish felt it was better, while Leela was already scenting victory on the distant horizon:
Seldom in computer chess events with top engines have we seen evaluations and suggested moves vary so much for so long, but it turns out it was Leela who was right!
Just to rub in how much better it understood the position Leela then went on to win the white side of the position in the very next game, increasing the lead to an unbridgable 5 points.
18 draws in a row followed after that, before the match finally concluded with the triumph of Leela Chess Zero, 53.5:46.5, with 14 wins to Stockfish's 7.
Will that mark the end of conventional engines being able to compete in the TCEC and the start of a battle between competing self-learning neural algorithms? Or can Stockfish mount a comeback? These are interesting times for computer chess!
We respect your privacy and data protection guidelines. Some components of our site require cookies or local storage that handles personal information.