Accounting for Room Acoustics in Audio-Visual Multi-Speaker Tracking

Yutong Ban (1) , Xiaofei Li (1) , Xavier Alameda-Pineda (1) , Laurent Girin (2, 1) , Radu Horaud (1)

1 PERCEPTION - Interpretation and Modelling of Images and Videos
2 GIPSA-CRISSP - GIPSA - Cognitive Robotics, Interactive Systems, & Speech Processing

Multiple-speaker tracking is a crucial task for many applications. In real-world scenarios, exploiting the complementarity between auditory and visual data enables to track people outside the visual field of view. However, practical methods must be robust to changes in acoustic conditions, e.g. reverberation. We investigate how to combine state-of-the-art audio-source localization techniques with Bayesian multi-person tracking. Our experiments demonstrate that the performance of the proposed system is not affected by changes in the acoustic environment.

Keywords: sound-source localization; multi-speaker tracking; audio-visual fusion; dereverberation

Domains: Computer Vision and Pattern Recognition [cs.CV]; Signal and Image Processing; Machine Learning [cs.LG]; Sound [cs.SD]

Yutong Ban, Xiaofei Li, Xavier Alameda-Pineda, Laurent Girin, Radu Horaud. Accounting for Room Acoustics in Audio-Visual Multi-Speaker Tracking. ICASSP 2018 - IEEE International Conference on Acoustics, Speech and Signal Processing, Apr 2018, Calgary, Alberta, Canada. pp.6553-6557, ⟨10.1109/ICASSP.2018.8462100⟩. ⟨hal-01718114⟩