Skip to Main content Skip to Navigation
Theses

Non-Convex Optimization for Latent Data Models : Algorithms, Analysis and Applications

Abstract : Many problems in machine learning pertain to tackling the minimization of a possibly non-convex and non-smooth function defined on a Many problems in machine learning pertain to tackling the minimization of a possibly non-convex and non-smooth function defined on a Euclidean space.Examples include topic models, neural networks or sparse logistic regression.Optimization methods, used to solve those problems, have been widely studied in the literature for convex objective functions and are extensively used in practice.However, recent breakthroughs in statistical modeling, such as deep learning, coupled with an explosion of data samples, require improvements of non-convex optimization procedure for large datasets.This thesis is an attempt to address those two challenges by developing algorithms with cheaper updates, ideally independent of the number of samples, and improving the theoretical understanding of non-convex optimization that remains rather limited.In this manuscript, we are interested in the minimization of such objective functions for latent data models, ie, when the data is partially observed which includes the conventional sense of missing data but is much broader than that.In the first part, we consider the minimization of a (possibly) non-convex and non-smooth objective function using incremental and online updates.To that end, we propose several algorithms exploiting the latent structure to efficiently optimize the objective and illustrate our findings with numerous applications.In the second part, we focus on the maximization of non-convex likelihood using the EM algorithm and its stochastic variants.We analyze several faster and cheaper algorithms and propose two new variants aiming at speeding the convergence of the estimated parameters.
Complete list of metadatas

Cited literature [231 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-02319140
Contributor : Abes Star :  Contact
Submitted on : Thursday, October 17, 2019 - 4:42:07 PM
Last modification on : Wednesday, September 2, 2020 - 3:36:59 AM
Long-term archiving on: : Saturday, January 18, 2020 - 2:58:58 PM

File

82036_KARIMI_2019_archivage.pd...
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-02319140, version 1

Collections

Citation

Belhal Karimi. Non-Convex Optimization for Latent Data Models : Algorithms, Analysis and Applications. Machine Learning [stat.ML]. Université Paris Saclay (COmUE), 2019. English. ⟨NNT : 2019SACLX040⟩. ⟨tel-02319140⟩

Share

Metrics

Record views

303

Files downloads

327