Skip to Main content Skip to Navigation

Solving Jigsaw Puzzles with Deep Learning for Heritage

Abstract : This thesis aims to develop semantic methods of reassembly in the complicated framework of heritage collections, where some blocks are eroded or missing. The reassembly of archaeological remains is an essential task for heritage sciences: it improves the understanding and conservation of ancient vestiges and artifacts. However, some sets of fragments can not be reassembled with techniques using contour information or visual continuities. It is then necessary to extract semantic information from the fragments and to interpret them. These tasks can be performed automatically thanks to deep learning techniques coupled with a solver, i.e., a constrained decision-making algorithm. This thesis proposes two semantic reassembly methods for 2D fragments with erosion, as well as a new dataset and evaluation metrics. The first method, Deepzzle, proposes a neural network followed by a solver. The neural network is composed of two Siamese convolutional networks trained to predict the relative position of two fragments: it is a 9-class classification. The solver uses Dijkstra’s algorithm to maximize the joint probability. Deepzzlecan address the case of missing and supernumerary fragments. It can process about15 fragments per puzzle and outperforms the state of the art by 25%. The second method, Alphazzle, is based on AlphaZero and single-player Monte Carlo Tree Search(MCTS). It is an iterative method that uses deep reinforcement learning: at each step, a fragment is placed on the current reassembly. Two neural networks guide MCTS: an action predictor, which uses the fragment and the current reassembly to propose a strategy, and an evaluator trained to predict the quality of the future result from the current reassembly. Alphazzle considers the relationships between all fragments and adapts to puzzles larger than those solved by Deepzzle. Moreover, Alphazzle is compatible with constraints imposed by a heritage framework: at the end of reassembly, MCTS does not access the reward, unlike AlphaZero. Indeed, the reward, which indicates if a puzzle is well solved or not, can only be estimated by the algorithm because only a conservator can be sure of a reassembly quality.
Document type :
Complete list of metadata
Contributor : Marie-Morgane Paumard Connect in order to contact the contributor
Submitted on : Monday, January 4, 2021 - 5:01:43 PM
Last modification on : Monday, January 25, 2021 - 3:16:04 PM
Long-term archiving on: : Monday, April 5, 2021 - 9:16:46 PM


Files produced by the author(s)


  • HAL Id : tel-03095670, version 1


Marie-Morgane Paumard. Solving Jigsaw Puzzles with Deep Learning for Heritage. Machine Learning [cs.LG]. CY Cergy Paris Université, 2020. English. ⟨tel-03095670⟩



Record views


Files downloads