Skip to Main content Skip to Navigation
New interface

Morphologically Plausible Deformation Transfer

Jean Basset 1, 2, 3 
2 MORPHEO - Capture and Analysis of Shapes in Motion
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
3 MIMETIC - Analysis-Synthesis Approach for Virtual Human Simulation
UR2 - Université de Rennes 2, Inria Rennes – Bretagne Atlantique , IRISA-D5 - RÉALITÉ VIRTUELLE, HUMAINS VIRTUELS, INTERACTIONS ET ROBOTIQUE
Abstract : With the advances of 3D content generation and capture, and the recent popularity of immersive virtual environments, producing realistic animations of 3D virtual characters has seen an increasing demand. More specifically, automatically applying existing animations on new characters with different body shapes would represent a significant gain of both time and resources for animators. Traditional methods transfer the pose in each frame of the animation to the new character. However, this implies being able to define what makes two poses equivalent. This is not straightforward as poses tend to change depending on the morphology of the character performing them, and as their meaning is highly contextual. In this manuscript, we propose new approaches that transform the identity of a character into a new identity without modifying the character's pose, which does not require defining pose equivalences. When changing the identity of a character, some artifacts may appear, such as interpenetration or loss of self-contacts between body surfaces, e.g. the hands touching in a clapping pose. We study how to adapt our methods to correct these artifacts.We first propose a method that iteratively morphs the identity of a source character in a specific pose to match the identity of a target character. This method allows to naturally mimic the pose of the source character in our results, as the optimization directly starts from the wanted pose. In this method we only apply simple pose corrections in order to preserve self-contacts present in the source and avoid interpenetrations, which allows the method to adapt the results to extreme morphologies.We then present a deep encoder-decoder architecture that learns from data to predict the identity deformation of a character to match a target character's identity, without changing the pose. We propose self-supervised identity losses, that allow an inference time fine tuning step enabling transfer to identities far from the training set, such as casually clothed humans. Our model generalizes well to complex unseen poses.Finally, we study the impact of self-contacts between body surfaces on perceived pose equivalences. Indeed, we observed that some self-contacts were only present because of the morphology of the character and were not important to the pose. Preserving all self-contacts in our first method could therefore create artifacts in some cases where unnecessary self-contacts were preserved, significantly modifying the pose. We conduct a study where we present to observers two models of a character mimicking the pose of a source character, one with the same self-contacts as the source, and one with one self-contact removed. We ask observers to select which model best mimics the source pose. We show that poses with different self-contacts are considered different by observers in most cases, and that this effect is stronger for self-contacts involving the hand than for those involving the arms.
Complete list of metadata
Contributor : ABES STAR :  Contact
Submitted on : Thursday, October 13, 2022 - 2:56:46 PM
Last modification on : Friday, November 18, 2022 - 2:26:17 PM


Version validated by the jury (STAR)


  • HAL Id : tel-03813699, version 1


Jean Basset. Morphologically Plausible Deformation Transfer. Machine Learning [cs.LG]. Université Grenoble Alpes [2020-..], 2022. English. ⟨NNT : 2022GRALM015⟩. ⟨tel-03813699⟩



Record views


Files downloads