Skip to Main content Skip to Navigation

Towards an understanding of neural networks : mean-field incursions

Abstract : Machine learning algorithms relying on deep new networks recently allowed a great leap forward in artificial intelligence. Despite the popularity of their applications, the efficiency of these algorithms remains largely unexplained from a theoretical point of view. The mathematical descriptions of learning problems involves very large collections of interacting random variables, difficult to handle analytically as well as numerically. This complexity is precisely the object of study of statistical physics. Its mission, originally pointed towards natural systems, is to understand how macroscopic behaviors arise from microscopic laws. In this thesis we propose to take advantage of the recent progress in mean-field methods from statistical physics to derive relevant approximations in this context. We exploit the equivalences and complementarities of message passing algorithms, high-temperature expansions and the replica method. Following this strategy we make practical contributions for the unsupervised learning of Boltzmann machines. We also make theoretical contributions considering the teacher-student paradigm to model supervised learning problems. We develop a framework to characterize the evolution of information during training in these model. Additionally, we propose a research direction to generalize the analysis of Bayesian learning in shallow neural networks to their deep counterparts.
Complete list of metadatas
Contributor : Abes Star :  Contact
Submitted on : Tuesday, August 25, 2020 - 12:08:08 PM
Last modification on : Wednesday, September 23, 2020 - 6:03:02 AM


Version validated by the jury (STAR)


  • HAL Id : tel-02921539, version 1


Marylou Gabrié. Towards an understanding of neural networks : mean-field incursions. Mathematical Physics [math-ph]. Université Paris sciences et lettres, 2019. English. ⟨NNT : 2019PSLEE035⟩. ⟨tel-02921539⟩



Record views


Files downloads