Example-Guided Physically Based Modal Sound Synthesis
ABSTRACT
Linear modal synthesis methods have often been used to generate sounds for rigid bodies. One of the key challenges in widely adopting such techniques is the lack of automatic determination of satisfactory material parameters that recreate realistic audio quality of sounding materials. We introduce a novel method using pre-recorded audio clips to estimate material parameters that capture the inherent quality of recorded sounding materials. Our method extracts perceptually salient features from audio examples. Based on psychoacoustic principles, we design a parameter estimation algorithm using an optimization framework and these salient features to guide the search of the best material parameters for modal synthesis. We also present a method that compensates for the differences between the real-world recording and sound synthesized using solely linear modal synthesis models to create the final synthesized audio. The resulting audio generated from this sound synthesis pipeline well preserves the same sense of material as a recorded audio example. Moreover, both the estimated material parameters and the residual compensation naturally transfer to virtual objects of different sizes and shapes, while the synthesized sounds vary accordingly. A perceptual study shows the results of this system compares well with real-world recordings in terms of material perception.
PUBLICATION
Example-Guided Physically Based Modal Sound Synthesis
ACM Transactions on Graphics (TOG), SIGGRAPH 2013
Volume 32 Issue 1, January 2013 Article No. 1
Zhimin Ren, Hengchin Yeh, and Ming C. Lin
Auditory Perception of Geometry-Invariant Material Properties
IEEE Transactions on Visualization and Computer Graphics (TVCG)
IEEE VR 2013 (Honorable Mention)
Volume 19, Issue 4, April 2013, pp. 557-566
Zhimin Ren, Hengchin Yeh, Roberta Klatzky, and Ming C. Lin
AudioCloning: Extracting Material Fingerprints from Example Audio Recording
SIGGRAPH 2012 Studio Talk
DEMO VIDEO