On the Links between Probabilistic Graphical Models and Submodular Optimisation

Abstract : The entropy of a probability distribution on a set of discrete random variables is always bounded by the entropy of its factorisable counterpart. This is due to the submodularity of entropy on the set of discrete random variables. Submodular functions are also generalisation of matroid rank function; therefore, linear functions may be optimised on the associated polytopes exactly using a greedy algorithm. In this manuscript, we exploit these links between the structures of graphical models and submodular functions: we use greedy algorithms to optimise linear functions on the polytopes related to graphic and hypergraphic matroids for learning the structures of graphical models, while we use inference algorithms on graphs to optimise submodular functions.The first main contribution of the thesis aims at approximating a probabilistic distribution with a factorisable tractable distribution under the maximum likelihood framework. Since the tractability of exact inference is exponential in the treewidth of the decomposable graph, our goal is to learn bounded treewidth decomposable graphs, which is known to be NP-hard. We pose this as a combinatorial optimisation problem and provide convex relaxations based on graphic and hypergraphic matroids. This leads to an approximate solution with good empirical performance. In the second main contribution, we use the fact that the entropy of a probability distribution is always bounded by the entropy of its factorisable counterpart mainly as a consequence of submodularity. This property of entropy is generalised to all submodular functions and bounds based on graphical models are proposed. We refer to them as graph-based bounds. An algorithm is developped to maximise submodular functions, which is NPhard, by maximising the graph-based bound using variational inference algorithms on graphs. As third contribution, we propose and analyse algorithms aiming at minimizing submodular functions that can be written as sum of simple functions. Our algorithms only make use of submodular function minimisation and total variation oracles of simple functions.
Document type :
Complete list of metadatas

Contributor : Abes Star <>
Submitted on : Thursday, March 29, 2018 - 5:03:11 PM
Last modification on : Tuesday, April 24, 2018 - 5:20:12 PM


Version validated by the jury (STAR)


  • HAL Id : tel-01753810, version 1



Senanayak Sesh Kumar Karri. On the Links between Probabilistic Graphical Models and Submodular Optimisation. Machine Learning [cs.LG]. PSL Research University, 2016. English. ⟨NNT : 2016PSLEE047⟩. ⟨tel-01753810⟩



Record views


Files downloads