Skip to Main content Skip to Navigation

Approaches for incremental learning and image generation

Konstantin Shmelkov 1, 2
2 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
Abstract : This dissertation explores two related topics in the context of deep learning: incremental learning and image generation. Incremental learning studies training of models with the objective function evolving over time, eg, addition of new categories to a classification task. Image generation seeks to learn a distribution of natural images for generating new images resembling original ones.Incremental learning is a challenging problem due to the phenomenon called catastrophic forgetting: any significant change to the objective during training causes a severe degradation of previously learned knowledge. We present a learning framework to introduce new classes to an object detection network. It is based on the idea of knowledge distillation to counteract catastrophic forgetting effects: fixed copy of the network evaluates old samples and its output is reused in an auxiliary loss to stabilize learning of new classes. Our framework mines these samples of old classes on the fly from incoming images, in contrast to other solutions that keep a subset of samples in memory.On the second topic of image generation, we build on the Generative Adversarial Network (GAN) model. Recently, GANs significantly improved the quality of generated images. However, they suffer from poor coverage of the dataset: while individual samples have great quality, some modes of the original distribution may not be captured. In addition, existing GAN evaluation methods are focused on image quality, and thus do not evaluate how well the dataset is covered, in contrast to the likelihood measure commonly used for generative models. We present two approaches to address these problems.The first method evaluates class-conditional GANs using two complementary measures based on image classification - GAN-train and GAN-test, which approximate recall (diversity) and precision (quality of the image) of GANs respectively. We evaluate several recent GAN approaches based on these two measures, and demonstrate a clear difference in performance. Furthermore, we observe that the increasing difficulty of the dataset, from CIFAR10 over CIFAR100 to ImageNet, shows an inverse correlation with the quality of the GANs, as clearly evident from our measures.Inspired by our study of GAN models, we present a method to explicitly enforce dataset coverage during the GAN training phase. We develop a generative model that combines GAN image quality with VAE architecture in the feature space engendered by a flow-based model Real-NVP. This allows us to evaluate a valid likelihood and simultaneously relax the independence assumption in RGB space which is common for VAEs. We achieve Inception score and FID competitive with state-of-the-art GANs, while maintaining good likelihood for this class of models.
Document type :
Complete list of metadata

Cited literature [210 references]  Display  Hide  Download
Contributor : Abes Star :  Contact
Submitted on : Monday, July 15, 2019 - 11:11:08 AM
Last modification on : Friday, February 5, 2021 - 3:28:32 AM


Version validated by the jury (STAR)


  • HAL Id : tel-02183259, version 1



Konstantin Shmelkov. Approaches for incremental learning and image generation. Mathematical Physics [math-ph]. Université Grenoble Alpes, 2019. English. ⟨NNT : 2019GREAM010⟩. ⟨tel-02183259⟩



Record views


Files downloads