Skip to Main content Skip to Navigation

Model selection: a decision-theoretic approach

Abstract : This manuscript addresses the problem of model selection, studied in the linear regression framework. The objective is to determine the best predictive model based on observed data, that is, the model realizing the best tradeoff between goodness of fit and complexity. Our main contribution consists in deriving model evaluation criteria based on tools from Decision Theory, in particular loss estimation. Such criteria rely on a distributional assumption larger than the classical Gaussian hypothesis with independent observations: the family of spherically symmetric distributions. This family of laws allows us to relax the independence assumption and thus brings robustness, since our criteria do not depend on the specific form of the distribution. We also propose a method for comparing model evaluation criteria through a Mean-Squared Error type measure. Our second contribution tackles the problem of constructing the models we compare. The conditions of models considered are obtained from sparse regularization methods, namely the Lasso and related methods. In particular, we studied the Minimax Concave Penalty (MCP), which keeps Lasso's selection while correcting its estimation bias. However, this penalty corresponds to a non differentiable and non- convex optimization problem. The generalization of subdifferentials with Clarke differentials allowed us to derive the optimality conditions d'optimalité and to propose a regularization path algorithm for MCP. Finally, we compare our propositions to the literature through a numerical study, in which we verify the quality of the selection. The results especially show that our criteria yield performances similar to the literature, and that frequently used criteria such as Cross Validation do not always result in good performances.
Complete list of metadatas

Cited literature [144 references]  Display  Hide  Download
Contributor : Aurélie Boisbunon <>
Submitted on : Thursday, February 28, 2013 - 4:48:08 PM
Last modification on : Tuesday, February 5, 2019 - 11:44:23 AM
Long-term archiving on: : Sunday, April 2, 2017 - 4:42:18 AM


  • HAL Id : tel-00793898, version 1


Aurélie Boisbunon. Model selection: a decision-theoretic approach. Statistics Theory [stat.TH]. Université de Rouen, 2013. English. ⟨tel-00793898⟩



Record views


Files downloads