Reflection-Aware Sound Source Localization

by Inkyu An, Myungbae Son, Dinesh Manocha, and Sung-Eui Yoon
IEEE International Conference on Robotics and Automation (ICRA), 2018

Our robot, equipped with a cube-shaped microphone array, localizes a sound source position in the 3D space. Our formulation takes both direct and indirect sounds into account. Direct acoustic rays (shown in green) are propagated using backward ray tracing based on received signals using a TDOA-based method. Reflected (or indirect) rays (shown in red) are then generated once they hit the boundaries of the reconstructed scene. The blue disk, which is very close to the ground truth, represents a 95% confidence ellipse for the estimated sound source, computed by our method. The use of reflected rays improves the localization accuracy by 40% over only using direct rays.

This figure shows an overview of our reflection-aware sound source localization approach. Two highlighted modules are our main contributions. Please refer to the paper for details.

To demonstrate the benefits of our reflection-aware method, we test our approach in three different testing scenarios in a classroom environment. (a) shows our tested robot with the cube-shaped microphone array. (b) and (c) show our testing environments for static and dynamically moving sound sources, respectively.For the moving sound, it generates sounds, only when it is on the violet part of its trajectory. Please refer to the paper and the video for the results.

The working video.
ICRA 2018 spotlight video.


We present a novel, reflection-aware method for 3D sound localization in indoor environments. Unlike prior approaches, which are mainly based on continuous sound signals from a stationary source, our formulation is designed to localize the position instantaneously from signals within a single frame. We consider direct sound and indirect sound signals that reach the microphones after reflecting off surfaces such as ceilings or walls. We then generate and trace direct and reflected acoustic paths using inverse acoustic ray tracing and utilize these paths with Monte Carlo localization to estimate a 3D sound source position. We have implemented our method on a robot with a cube-shaped microphone array and tested it against different settings with continuous and intermittent sound signals with a stationary or a mobile source. Across different settings, our approach can localize the sound with an average distance error of 0.8 m tested in a room of 7 m by 7 m area with 3 m height, including a mobile and non-line-of-sight sound source. We also reveal that the modeling of indirect rays increases the localization accuracy by 40% compared to only using direct acoustic rays.


Paper: PDF
Poster: PDF
Source Code: ZIP file / Github Page
     : You can run our project without any other additional project (It contains all needed data).

Dept. of Computer Science
373-1 Guseong-dong, Yuseong-gu, Daejeon, 305-701
South Korea
sglabkaist dot gmail dot com