Skip to Main content Skip to Navigation
Theses

Estimation and feature selection in high-dimensional mixtures-of-experts models

Abstract : This thesis deals with the problem of modeling and estimation of high-dimensional MoE models, towards effective density estimation, prediction and clustering of such heterogeneous and high-dimensional data. We propose new strategies based on regularized maximum-likelihood estimation (MLE) of MoE models to overcome the limitations of standard methods, including MLE estimation with Expectation-Maximization (EM) algorithms, and to simultaneously perform feature selection so that sparse models are encouraged in such a high-dimensional setting. We first introduce a mixture-of-experts’ parameter estimation and variable selection methodology, based on l1 (lasso) regularizations and the EM framework, for regression and clustering suited to high-dimensional contexts. Then, we extend the method to regularized mixture of experts models for discrete data, including classification. We develop efficient algorithms to maximize the proposed l1 -penalized observed-data log-likelihood function. Our proposed strategies enjoy the efficient monotone maximization of the optimized criterion, and unlike previous approaches, they do not rely on approximations on the penalty functions, avoid matrix inversion, and exploit the efficiency of the coordinate ascent algorithm, particularly within the proximal Newton-based approach.
Document type :
Theses
Complete list of metadatas

Cited literature [247 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-02469045
Contributor : Abes Star :  Contact
Submitted on : Thursday, February 6, 2020 - 12:34:09 PM
Last modification on : Monday, April 27, 2020 - 4:14:03 PM
Long-term archiving on: : Thursday, May 7, 2020 - 2:36:44 PM

File

sygal_fusion_28420-huynh-bao_t...
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-02469045, version 1

Citation

Bao Tuyen Huynh. Estimation and feature selection in high-dimensional mixtures-of-experts models. Numerical Analysis [math.NA]. Normandie Université, 2019. English. ⟨NNT : 2019NORMC237⟩. ⟨tel-02469045⟩

Share

Metrics

Record views

111

Files downloads

72