Skip to Main content Skip to Navigation
Theses

Analyzing and Introducing Structures in Deep Convolutional Neural Networks

Abstract : This thesis studies empirical properties of deep convolutional neural networks, and in particular the Scattering Transform. Indeed, the theoretical analysis of the latter is hard and until now remains a challenge: successive layers of neurons have the ability to produce complex computations, whose nature is still unknown, thanks to learning algorithms whose convergence guarantees are not well understood. However, those neural networks are outstanding tools to tackle a wide variety of difficult tasks, like image classification or more formally statistical prediction. The Scattering Transform is a non-linear mathematical operator whose properties are inspired by convolutional networks. In this work, we apply it to natural images, and obtain competitive accuracies with unsupervised architectures. Cascading a supervised neural networks after the Scattering permits to compete on ImageNet2012, which is the largest dataset of labeled images available. An efficient GPU implementation is provided. Then, this thesis focuses on the properties of layers of neurons at various depths. We show that a progressive dimensionality reduction occurs and we study the numerical properties of the supervised classification when we vary the hyper parameters of the network. Finally, we introduce a new class of convolutional networks, whose linear operators are structured by the symmetry groups of the classification task.
Complete list of metadata

Cited literature [125 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/tel-02353134
Contributor : Edouard Oyallon Connect in order to contact the contributor
Submitted on : Thursday, November 7, 2019 - 11:00:39 AM
Last modification on : Thursday, March 17, 2022 - 10:08:38 AM
Long-term archiving on: : Saturday, February 8, 2020 - 11:28:20 PM

File

thesis.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : tel-02353134, version 1

Collections

Citation

Edouard Oyallon. Analyzing and Introducing Structures in Deep Convolutional Neural Networks. Machine Learning [stat.ML]. Paris Sciences et Lettres, 2017. English. ⟨tel-02353134⟩

Share

Metrics

Record views

291

Files downloads

140