Discovery of human activities in video

Guido Thomas Pusiol 1
1 STARS - Spatio-Temporal Activity Recognition Systems
CRISAM - Inria Sophia Antipolis - Méditerranée
Abstract : The main objective of the thesis is to propose a complete framework for the automatic activity discovery, modelling and recognition using video information. The framework uses perceptual information (e.g. trajectories) as input and goes up to activities (semantics). The framework is divided into five main parts: 1) We break the video into chunks to characterize activities. We propose different techniques to extract perceptual features from the chunks. This way, we build packages of perceptual features capable of describing activity occurring in small periods of time. 2) We propose to learn the video contextual information. We build scene models by learning salient perceptual features. The models end up containing interesting scene regions capable of describing basic semantics (i.e. region where interactions occur). 3) We propose to reduce the gap between low-level vision information and semantic interpretation, by building an intermediate layer composed of primitive Events. The proposed representation for primitive events aims at describing the meaningful motions over the scene. This is achieved by abstracting perceptual features using contextual information in an unsupervised manner. 4) We propose a pattern-based method to discover activities at multiple resolutions (i.e. activities and sub-activities). Also, we propose a generative method to model multi-resolution activities. The models are built as a flexible probabilistic framework easy to update. 5) We propose an activity recognition method that finds in a deterministic manner the occurrences of modelled activities in unseen datasets. Semantics are provided by the method under user interaction. All this research work has been evaluated using real datasets of people living in an apartment (home-care application) and elder patients in a hospital. The work has also been evaluated for other types of applications such as sleeping monitoring.
Document type :
Complete list of metadatas

Cited literature [101 references]  Display  Hide  Download
Contributor : Guido Thomas Pusiol <>
Submitted on : Monday, February 10, 2014 - 6:35:53 PM
Last modification on : Thursday, January 11, 2018 - 4:21:59 PM
Long-term archiving on : Monday, May 12, 2014 - 1:10:11 PM


  • HAL Id : tel-00944617, version 1



Guido Thomas Pusiol. Discovery of human activities in video. Artificial Intelligence [cs.AI]. Université Nice Sophia Antipolis, 2012. English. ⟨tel-00944617⟩



Record views


Files downloads