Skip to Main content Skip to Navigation

Estimation and feature selection in high-dimensional mixtures-of-experts models

Abstract : This thesis deals with the problem of modeling and estimation of high-dimensional MoE models, towards effective density estimation, prediction and clustering of such heterogeneous and high-dimensional data. We propose new strategies based on regularized maximum-likelihood estimation (MLE) of MoE models to overcome the limitations of standard methods, including MLE estimation with Expectation-Maximization (EM) algorithms, and to simultaneously perform feature selection so that sparse models are encouraged in such a high-dimensional setting. We first introduce a mixture-of-experts’ parameter estimation and variable selection methodology, based on l1 (lasso) regularizations and the EM framework, for regression and clustering suited to high-dimensional contexts. Then, we extend the method to regularized mixture of experts models for discrete data, including classification. We develop efficient algorithms to maximize the proposed l1 -penalized observed-data log-likelihood function. Our proposed strategies enjoy the efficient monotone maximization of the optimized criterion, and unlike previous approaches, they do not rely on approximations on the penalty functions, avoid matrix inversion, and exploit the efficiency of the coordinate ascent algorithm, particularly within the proximal Newton-based approach.
Document type :
Complete list of metadatas

Cited literature [247 references]  Display  Hide  Download
Contributor : Abes Star :  Contact
Submitted on : Thursday, February 6, 2020 - 12:34:09 PM
Last modification on : Monday, April 27, 2020 - 4:14:03 PM
Long-term archiving on: : Thursday, May 7, 2020 - 2:36:44 PM


Version validated by the jury (STAR)


  • HAL Id : tel-02469045, version 1


Bao Tuyen Huynh. Estimation and feature selection in high-dimensional mixtures-of-experts models. Numerical Analysis [math.NA]. Normandie Université, 2019. English. ⟨NNT : 2019NORMC237⟩. ⟨tel-02469045⟩



Record views


Files downloads