Active learning and input space analysis for deep networks

Abstract : Our work is presented in three separate parts which can be read independently. Firstly we propose three active learning heuristics that scale to deep neural networks: We scale query by committee, an ensemble active learning methods. We speed up the computation time by sampling a committee of deep networks by applying dropout on the trained model. Another direction was margin-based active learning. We propose to use an adversarial perturbation to measure the distance to the margin. We also establish theoretical bounds on the convergence of our Adversarial Active Learning strategy for linear classifiers. Some inherent properties of adversarial examples opens up promising opportunity to transfer active learning data from one network to another. We also derive an active learning heuristic that scales to both CNN and RNN by selecting the unlabeled data that minimize the variational free energy. Secondly, we focus our work on how to fasten the computation of Wasserstein distances. We propose to approximate Wasserstein distances using a Siamese architecture. From another point of view, we demonstrate the submodular properties of Wasserstein medoids and how to apply it in active learning. Eventually, we provide new visualization tools for explaining the predictions of CNN on a text. First, we hijack an active learning strategy to confront the relevance of the sentences selected with active learning to state-of-the-art phraseology techniques. These works help to understand the hierarchy of the linguistic knowledge acquired during the training of CNNs on NLP tasks. Secondly, we take advantage of deconvolution networks for image analysis to present a new perspective on text analysis to the linguistic community that we call Text Deconvolution Saliency.
Complete list of metadatas

Cited literature [211 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-02271840
Contributor : Abes Star <>
Submitted on : Tuesday, August 27, 2019 - 11:50:07 AM
Last modification on : Wednesday, August 28, 2019 - 1:17:29 AM

File

2018AZUR4115.pdf
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-02271840, version 1

Collections

Citation

Mélanie Ducoffe. Active learning and input space analysis for deep networks. Neural and Evolutionary Computing [cs.NE]. Université Côte d'Azur, 2018. English. ⟨NNT : 2018AZUR4115⟩. ⟨tel-02271840⟩

Share

Metrics

Record views

127

Files downloads

156