Our robot, equipped with a cube-shaped microphone array, localizes a sound source position in the 3D space.
Our formulation takes both direct and indirect sounds into account.
Direct acoustic rays (shown in green) are propagated using backward ray tracing based on received signals using a TDOA-based method. Reflected (or indirect) rays (shown in red) are then generated once they hit the boundaries of the reconstructed scene.
The blue disk, which is very close to the ground truth, represents a 95% confidence ellipse for the estimated sound source, computed by our method.
The use of reflected rays improves the localization accuracy by 40% over only using direct rays.
This figure shows an overview of our reflection-aware sound source localization approach. Two highlighted modules are our main contributions.
Please refer to the paper for details.
To demonstrate the benefits of our reflection-aware method, we test our approach in three different testing scenarios in a classroom environment.
(a) shows our tested robot with the cube-shaped microphone array. (b) and (c) show our testing environments for static and dynamically moving sound sources, respectively.For the moving sound, it generates sounds, only when it is on the violet part of its trajectory.
Please refer to the paper and the video for the results.
The working video.
ICRA 2018 spotlight video.
Abstract
We present a novel, reflection-aware method for
3D sound localization in indoor environments. Unlike prior
approaches, which are mainly based on continuous sound
signals from a stationary source, our formulation is designed to
localize the position instantaneously from signals within a single
frame. We consider direct sound and indirect sound signals
that reach the microphones after reflecting off surfaces such
as ceilings or walls. We then generate and trace direct and
reflected acoustic paths using inverse acoustic ray tracing and
utilize these paths with Monte Carlo localization to estimate a
3D sound source position. We have implemented our method
on a robot with a cube-shaped microphone array and tested
it against different settings with continuous and intermittent
sound signals with a stationary or a mobile source. Across
different settings, our approach can localize the sound with an
average distance error of 0.8 m tested in a room of 7 m by 7 m
area with 3 m height, including a mobile and non-line-of-sight
sound source. We also reveal that the modeling of indirect rays
increases the localization accuracy by 40% compared to only
using direct acoustic rays.
Contents
Paper: PDF
Poster: PDF
Presentation slides: PPT
Source Code: ZIP file / Github Page
: You can run our project without any other additional project (It contains all needed data).
Dept. of Computer Science
KAIST
373-1 Guseong-dong, Yuseong-gu, Daejeon, 305-701
South Korea
sglabkaist dot gmail dot com
