Level of detail for granular audio-graphic rendering : representation, implementation, and user-based evaluation

Abstract : Real-time simulation of complex audio-visual scenes remains challenging due to the technically independent but perceptually related rendering process in each modality. Because of the potential crossmodal dependency of auditory and visual perception, the optimization of graphics and sound rendering, such as Level of Details (LOD), should be considered in a combined manner but not as separate issues. For instance, in audition and vision, people have perceptual limits on observation quality. Techniques of perceptually driven LOD for graphics have been greatly advanced for decades. However, the concept of LOD is rarely considered in crossmodal evaluation and rendering. This thesis is concentrated on the crossmodal evaluation of perception on audiovisual LOD rendering by psychophysical methods, based on that one may apply a functional and general method to eventually optimize the rendering. The first part of the thesis is an overview of our research. In this part, we review various LOD approaches and discuss concerned issues, especially from a crossmodal perceptual perspective. We also discuss the main results on the design, rendering and applications of highly detailed interactive audio and graphical scenes of the ANR Topophonie project, in which the thesis took place. A study of psychophysical methods for the evaluation on audio-visual perception is also presented to provide a solid knowledge of experimental design. In the second part, we focus on studying the perception of image artifacts in audio-visual LOD rendering. A series of experiments was designed to investigate how the additional audio modality can impact the visual detection of artifacts produced by impostor-based LOD. The third part of the thesis is focused on the novel extended-X3D that we designed for audio-visual LOD modeling. In the fourth part, we present a design and evaluation of the refined crossmodal LOD system. The evaluation of the audio-visual perception on crossmodal LOD system was achieved through a series of psychophysical experiments. Our main contribution is that we provide a further understanding of crossmodal LOD with some new observations, and explore it through perceptual experiments and analysis. The results of our work can eventually be used as the empirical evidences and guideline for a perceptually driven crossmodal LOD system.
Document type :
Theses
Complete list of metadatas

https://tel.archives-ouvertes.fr/tel-00913624
Contributor : Abes Star <>
Submitted on : Wednesday, December 4, 2013 - 10:22:32 AM
Last modification on : Monday, September 16, 2019 - 11:45:49 AM
Long-term archiving on : Tuesday, March 4, 2014 - 10:20:15 PM

Files

VA2_DING_HUI_30092013_synthese...
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-00913624, version 1

Collections

Citation

Hui Ding. Level of detail for granular audio-graphic rendering : representation, implementation, and user-based evaluation. Other [cs.OH]. Université Paris Sud - Paris XI, 2013. English. ⟨NNT : 2013PA112205⟩. ⟨tel-00913624⟩

Share

Metrics

Record views

608

Files downloads

549