Estimating the Puzzlingness of Chess Puzzles

Solving chess puzzles, which require finding a certain sequence of moves to achieve a winning position (such as checkmate or a significant material advantage), is a commonly used method for improving chess skills, particularly tactical awareness. Estimating the puzzlingness—i.e., the difficulty—of a chess puzzle is challenging and typically involves showing the puzzle to multiple players and measuring their success rate. In this work, we present a method for predicting the difficulty of chess puzzles using only the initial position and the sequence of correct moves. We generate multiple features, including both hand-crafted features and those extracted from chess engines such as Maia, Leela Chess Zero, and Stockfish. We also train a residual neural network to directly predict the Glicko-2 puzzle rating. The output of this neural network, along with the other generated features, serves as input to a gradient boosting decision tree model to predict the final Glicko-2 rating. Our model was applied to the IEEE BigData 2024 Cup competition on Predicting Chess Puzzle Difficulty, where it achieved third place.