Multi-frame sequence generator of 4D human body motion

Mathieu Marsot (1) , Stefanie Wuhrer (1) , Jean-Sébastien Franco (1) , Stephane Durocher (2)

1 MORPHEO - Capture and Analysis of Shapes in Motion
2 University of Manitoba [Winnipeg]

We examine the problem of generating temporally and spatially dense 4D human body motion. On the one hand generative modeling has been extensively studied as a per time-frame static fitting problem for dense 3D models such as mesh representations, where the temporal aspect is left out of the generative model. On the other hand, temporal generative models exist for sparse human models such as marker-based capture representations, but have not to our knowledge been extended to dense 3D shapes. We propose to bridge this gap with a generative auto-encoder-based framework, which encodes morphology, global locomotion including translation and rotation, and multi-frame temporal motion as a single latent space vector. To assess its generalization and factorization abilities, we train our model on a cyclic locomotion subset of AMASS, leveraging the dense surface models it provides for an extensive set of motion captures. Our results validate the ability of the model to reconstruct 4D sequences of human locomotions within a low error bound, and the meaningfulness of latent space interpolation between latent vectors representing different multi-frame sequences and locomotion types. We also illustrate the benefits of the approach for 4D human motion prediction of future frames from initial human locomotion frames, showing promising abilities of our model to learn realistic spatio-temporal features of human motion. We show that our model allows for data completion of both spatially and temporally sparse data.

Domains: Computer Vision and Pattern Recognition [cs.CV]

Mathieu Marsot, Stefanie Wuhrer, Jean-Sébastien Franco, Stephane Durocher. Multi-frame sequence generator of 4D human body motion. 2021. ⟨hal-03250297v1⟩