Real-time Hand Tracking Using a Sum of Anisotropic Gaussians Model

International Conference on 3D Vision (3DV) 2014, Tokyo, Japan


Real-time marker-less hand tracking is of increasing importance in human-computer interaction. Robust and accurate tracking of arbitrary hand motion is a challenging problem due to the many degrees of freedom, frequent self-occlusions, fast motions, and uniform skin color. In this paper, we propose a new approach that tracks the full skeleton motion of the hand from multiple RGB cameras in real-time. The main contributions include a new generative tracking method which employs an implicit hand shape representation based on Sum of Anisotropic Gaussians (SAG), and a pose fitting energy that is smooth and analytically differentiable making fast gradient based pose optimization possible. This shape representation, together with a full perspective projection model, enables more accurate hand modeling than a related baseline method from literature. Our method achieves better accuracy than previous methods and runs at 25 fps. We show these improvements both qualitatively and quantitatively on publicly available datasets.


BibTeX, 1 KB

author = {Sridhar, Srinath and Rhodin, Helge and Seidel, Hans-Peter and Oulasvirta, Antti and Theobalt, Christian},
title = {Real-time Hand Tracking Using a Sum of Anisotropic Gaussians Model},
booktitle = {Proceedings of the International Conference on 3D Vision ({3DV})},
url = {},
numpages = {8},
month = Dec,
year = {2014}

Related Pages

  • Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data, ICCV 2013 (webpage)
  • Dexter 1: Dataset for Evaluating Hand Tracking Algorithms (webpage)
  • HandSonor: A Customizable Vision-based Control Interface for Musical Expression, Extended Abstracts, CHI 2013 (webpage)


This work was supported by the ERC Starting Grant CapReal. We would like to thank James Tompkin and Danhang Tang.


Srinath Sridhar

This page is Zotero translator friendly. Page last updated 20-Jan-2015. Imprint/Impressum.