Skip to Main content Skip to Navigation
New interface

Context-aware person recognition in TV programs

Abstract : The automatic recognition and retrieval of faces can be a useful tool for exploiting and promoting large datasets, such as the archival collection of TV shows stored by INA. Although face recognition solutions have improved dramatically in the last decade, they unfortunately remain prone to mistakes, more especially with a large number of faces and a large number of different identities. The various TV shows are however quite standardised, meaning that it is most of the time easy for anyone to tell what a TV show is about in a glimpse, be it a sport show, an entertainment show or a newscast. Though implicit, this standardisation of the TV shows applies in numerous ways, from the visual appearance of the show to the broadcast time. Moreover, we also know that the contextual information plays a major role in helping the human brain recognizing people, and that, in fact, we seldom recognize people based on their facial appearance only. This also applies to TV shows, where the various contextual information can help us identify who is likely or not to appear in a given show. The goal of this thesis is to identify and to exploit the contextual modalities available and potentially useful for the identification of the people appearing in TV shows. For each one of these modalities, we extract the information as a feature descriptor which can be combined to the facial feature descriptor to either retrieve other instances of the same person or to identify them. More especially, we focus on how the social relationships of the people appearing in the shows make them more likely to appear with some people than with others. We introduce an unsupervised method for identifying simultaneously the participants of a TV show, by estimating their probably to appear together based on previous unannotated observations. We also study the visual context of the shows and we highlight how the background and other visual cues can help to successfully identify difficult faces. Finally, we explore how useful can be the contextual modalities such as the time of broadcast or the thematic tags assigned to each show, by evaluating the improvement they bring on the face recognition task and how redundant they can be with the other modalities.
Document type :
Complete list of metadata
Contributor : ABES STAR :  Contact
Submitted on : Monday, October 24, 2022 - 6:23:20 PM
Last modification on : Tuesday, October 25, 2022 - 3:57:49 AM


Version validated by the jury (STAR)


  • HAL Id : tel-03827700, version 1


Thomas Petit. Context-aware person recognition in TV programs. Artificial Intelligence [cs.AI]. Université de Lyon, 2022. English. ⟨NNT : 2022LYSEI049⟩. ⟨tel-03827700⟩



Record views


Files downloads