Temporal Integration of Multiple Silhouette-based Body-part Hypotheses


A method for temporally integrating appearance-based body-part labelling is presented.  We begin by modifying the silhouette labelling method of Ghost ; that system first determines which posture best describes the person currently and then uses posture-specific heuristics to generate labels for head, hands, and feet. Our approach is to assign a posture probability and then estimate body part locations for all possible postures. Next we temporally integrate these estimates by finding a best path through the posture-time lattice. A density-sampling propagation approach is used that allows us to model the multiple hypotheses resulting from consideration of different postures.  We show quantitative and qualitative results where the temporal integration solution improves the instantaneous estimates.  This method can be applied to any system that inherently has multiple methods of asserting instantaneous properties but from which a temporally coherent interpretation is desired.

Related publication:

Temporal Integration of Multiple Silhouette-based Body-part Hypotheses
Vivek Kwatra
, Aaron F. Bobick and Amos Y. Johnson
Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2001)
| BibTex

Goals :
  • Estimation of human body-part locations from a video.
  • Start from a silhouette-based instantaneous body-part labeling technique - Ghost , which first determines body  posture and then applies a posture-based heuristic to determine body-part locations in each frame.
  • Introduce a framework for temporal integration of  posture and body-part labels generated by Ghost .
How Ghost works:


Our Approach:

Sample-based Density



Comparison videos: