HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation

Topology of statistical systems : a cohomological approach to information theory

Abstract : This thesis extends in several directions the cohomological study of information theory pioneered by Baudot and Bennequin. We introduce a topos-theoretical notion of statistical space and then study several cohomological invariants. Information functions and related objects appear as distinguished cohomology classes; the corresponding cocycle equations encode recursive properties of these functions. Information has thus topological meaning and topology serves as a unifying framework.Part I discusses the geometrical foundations of the theory. Information structures are introduced as categories that encode the relations of refinement between different statistical observables. We study products and coproducts of information structures, as well as their representation by measurable functions or hermitian operators. Every information structure gives rise to a ringed site; we discuss in detail the definition of information cohomology using the homological tools developed by Artin, Grothendieck, Verdier and their collaborators.Part II studies the cohomology of discrete random variables. Information functions—Shannon entropy, Tsallis alpha-entropy, Kullback-Leibler divergence—appear as 1-cocycles for appropriate modules of probabilistic coefficients (functions of probability laws). In the combinatorial case (functions of histograms), the only 0-cocycle is the exponential function, and the 1-cocycles are generalized multinomial coefficients (Fontené-Ward). There is an asymptotic relation between the combinatorial and probabilistic cocycles.Part III studies in detail the q-multinomial coefficients, showing that their growth rate is connected to Tsallis 2-entropy (quadratic entropy). When q is a prime power, these q-multinomial coefficients count flags of finite vector spaces with prescribed length and dimensions. We obtain a combinatorial explanation for the nonadditivity of the quadratic entropy and a frequentist justification for the maximum entropy principle with Tsallis statistics. We introduce a discrete-time stochastic process associated to the q-binomial probability distribution that generates finite vector spaces (flags of length 2). The concentration of measure on certain typical subspaces allows us to extend Shannon's theory to this setting.Part IV discusses the generalization of information cohomology to continuous random variables. We study the functoriality properties of conditioning (seen as disintegration) and its compatibility with marginalization. The cohomological computations are restricted to the real valued, gaussian case. When coordinates are fixed, the 1-cocycles are the differential entropy as well as generalized moments. When computations are done in a coordinate-free manner, with the so-called grassmannian categories, we recover as the only degree-one cohomology classes the entropy and the dimension. This constitutes a novel algebraic characterization of differential entropy.
Complete list of metadata

Cited literature [104 references]  Display  Hide  Download

Contributor : Abes Star :  Contact
Submitted on : Monday, September 28, 2020 - 4:54:08 PM
Last modification on : Saturday, December 4, 2021 - 4:04:56 AM
Long-term archiving on: : Tuesday, December 29, 2020 - 7:03:05 PM


Version validated by the jury (STAR)


  • HAL Id : tel-02951504, version 1


Juan Pablo Vigneaux. Topology of statistical systems : a cohomological approach to information theory. Information Theory [cs.IT]. Université Sorbonne Paris Cité, 2019. English. ⟨NNT : 2019USPCC070⟩. ⟨tel-02951504⟩



Record views


Files downloads