the HandVu
vision-based hand gesture interface

Mathias Kölsch, Matthew Turk, Tobias Höllerer


This software collection implements a vision-based hand gesture interface and is available for download here. Called HandVu, it detects the hand in a standard posture, then tracks it and recognizes key postures - all in real-time and without the need for camera or user calibration. The technological novelties are mainly its speed and robustness, allowing deployment without constraining the visual scene. It works with unaugmented hands, moving cameras, dynamic backgrounds, changing lighting conditions and much more.

The software package consists of the main HandVu library and several applications that demonstrate video capture with OpenCV's highgui, DirectShow, and soon the ARtoolkit. If your application includes video processing, you would just make a few library calls and hand gesture recognition results will be available to you. If your application does not use camera input already, you can run one of the sample programs on the side and interface to your application with a networking interface.


Please visit the main HandVu web site for more information and to download the library.


Mathias Kölsch
Vision Based Hand Gesture Interfaces for Wearable Computing and Virtual Environments. 
Ph. D. Dissertation
, August 2004.

Mathias Kölsch and Matthew Turk 
Analysis of Rotational Robustness of Hand Detection with Viola&Jones' Method. 
In Proc. ICPR, 2004.

Mathias Kölsch, Matthew Turk, and Tobias Höllerer
Vision-Based Interfaces for Mobility. 
Intl. Conference on Mobile and Ubiquitous Systems (MobiQuitous), August 2004.

Mathias Kölsch and Matthew Turk
Fast 2D Hand Tracking with Flocks of Features and Multi-Cue Integration. 
In IEEE Workshop on Real-Time Vision for Human-Computer Interaction (at CVPR), 2004. 
This video (4MB, WMV) accompanies the paper. Best Paper award!

Mathias Kölsch and Matthew Turk
Robust Hand Detection. 
In Proc. IEEE Intl. Conference on Automatic Face and Gesture Recognition, May 2004.


mouse-based interaction
hand tracking
interaction with virtual objects