Skip to Main content Skip to Navigation

L'apprentissage profond pour l'évaluation et le retour d'information lors de l'apprentissage de gestes

Abstract : Learning a new sport or manual work is complex. Indeed, many gestures have to be assimilated in order to reach a good level of skill. However, learning these gestures cannot be done alone. Indeed, it is necessary to see the gesture execution with an expert eye in order to indicate corrections for improvement. However, experts, whether in sports or in manual works, are not always available to analyze and evaluate a novice’s gesture. In order to help experts in this task of analysis, it is possible to develop virtual coaches. Depending on the field, the virtual coach will have more or less skills, but an evaluation according to precise criteria is always mandatory. Providing feedback on mistakes is also essential for the learning of a novice. In this thesis, different solutions for developing the most effective virtual coaches are proposed. First of all, and as mentioned above, it is necessary to evaluate the gestures. From this point of view, a first part consisted in understanding the stakes of automatic gesture analysis, in order to develop an automatic evaluation algorithm that is as efficient as possible. Subsequently, two algorithms for automatic quality evaluation are proposed. These two algorithms, based on deep learning, were then tested on two different gestures databases in order to evaluate their genericity. Once the evaluation has been carried out, it is necessary to provide relevant feedback to the learner on his errors. In order to maintain continuity in the work carried out, this feedback is also based on neural networks and deep learning. A method has been developed based on neural network explanability methods. It allows to go back to the moments of the gestures when errors were made according to the evaluation model. Finally, coupled with semantic segmentation, this method makes it possible to indicate to learners which part of the gesture was badly performed, and to provide them with statistics and a learning curve.
Document type :
Complete list of metadata
Contributor : Abes Star :  Contact
Submitted on : Wednesday, April 7, 2021 - 9:11:30 AM
Last modification on : Friday, April 9, 2021 - 3:32:06 AM


Version validated by the jury (STAR)


  • HAL Id : tel-03191291, version 1


Mégane Millan. L'apprentissage profond pour l'évaluation et le retour d'information lors de l'apprentissage de gestes. Traitement du signal et de l'image [eess.SP]. Sorbonne Université, 2020. Français. ⟨NNT : 2020SORUS057⟩. ⟨tel-03191291⟩



Record views