Leveraging Depth Cameras and Wearable Pressure Sensors for Full-body Kinematics and Dynamics Capture

 

SIGGRAPH ASIA 2014

 

Peizhao Zhang1                Kristin Siu2                Jianjie Zhang1

C. Karen Liu2                Jinxiang Chai1

 

1Texas A&M University                2Georgia Institute of Technology

 

 

teaser6_new_short

 

Abstract

 

We present a new method for full-body motion capture that uses input data captured by three depth cameras and a pair of pressure sensing shoes. Our system is appealing because it is low-cost, nonintrusive and fully automatic, and can accurately reconstruct both full-body kinematics and dynamics data. We first introduce a novel tracking process that automatically reconstructs 3D skeletal poses using input data captured by three Kinect cameras and wearable pressure sensors. We formulate the problem in an optimization framework and incrementally update 3D skeletal poses with observed depth data and pressure data via iterative linear solvers. The system is highly accurate because we integrate depth data from multiple depth cameras, foot pressure data, detailed full-body geometry, and environmental contact constraints into a unified framework. In addition, we develop an efficient physics-based motion reconstruction algorithm for solving internal joint torques and contact forces in the quadratic programming framework. During reconstruction, we leverage Newtonian physics, friction cone constraints, contact pressure information, and 3D kinematic poses obtained from the kinematic tracking process to reconstruct full-body dynamics data. We demonstrate the power of our approach by capturing a wide range of human movements and achieve state-of-the-art accuracy in our comparison against alternative systems.

Video

 

 

Downloads 

 

paper_front

 

Leveraging Depth Cameras and Wearable Pressure Sensors for Full-body Kinematics and Dynamics Capture, ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia), 2014

Paper PDF

 

video_front

Youtube , Video (mov, 189 M)

Bibtex

 

@article{Zhang_sa2014,

  title   = {Leveraging Depth Cameras and Wearable Pressure Sensors for Full-body Kinematics and Dynamics Capture},

  author  = {Peizhao Zhang, Kristin Siu, Jianjie Zhang, C. Karen Liu, Jinxiang Chai},

  journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2014)},

  volume  = {33},

  issue   = {6},

  pages   = {},

  year    = {2014},

}

 

Acknowledgement

 

This work is supported in part by the NSF under Grants No. IIS-1055046, IIS-1065384, and IIS-1064983.