Audio-Material Reconstruction for Virtualized Reality Using a Probabilistic Damping Model

Auston Sterling, Nicholas Rewkowski, Roberta L. Klatzky, and Ming C. Lin

Virtualized Objects: A virtual environment with interactive objects whose impact sounds use parameters estimated from real objects

Audio-Material Reconstruction for Virtualized Reality Using a Probabilistic Damping Model
IEEE Conference on Virtual Reality and 3D User Interfaces 2019
IEEE Transactions on Visualization and Computer Graphics
[bibtex] [DOI: 10.1109/TVCG.2019.2898822]
Modal sound synthesis has been used to create realistic sounds from rigid-body objects, but requires accurate real-world material parameters. These material parameters can be estimated from recorded sounds of an impacted object, but external factors can interfere with accurate parameter estimation. We present a novel technique for estimating the damping parameters of materials from recorded impact sounds that probabilistically models these external factors. We represent the combined effects of material damping, support damping, and sampling inaccuracies with a probabilistic generative model, then use maximum likelihood estimation to fit a damping model to recorded data. This technique greatly reduces the human effort needed and does not require the precise object geometry or the exact hit location. We validate the effectiveness of this technique with a comprehensive analysis of a synthetic dataset and a perceptual study on object identification. We also present a study establishing human performance on the same parameter estimation task for comparison.

Download mp4 (122 MB)
Introduction and demonstrational video

Supplemental Material
Contains user study results, additional figures, and an additional document containing more detailed results.
This research is supported in part by the National Science Foundation and the UNC Arts and Sciences Foundation.
Related Links
The GAMMA group at UNC
Sound research at GAMMA