Hand gestures

We develop a simple game with multiple levels where you guide a character through a maze using your hands to determine the direction. The predetermined hand gestures are captured using a simple low-end webcam, and work at an acceptable success rate.

The main challenge consists in choosing suitable hand gestures that are easy to recognize by applying several operators on the captured image, retaining robust performance under varying light intensity, background color, skin color and hand size.

We approach this by first creating a black and white representation of the captured image by filtering certain colors and applying various dilation and erosion operations on the resulting image. With this representation, we can determine the gesture by measuring the width of the hand at certain heights, or counting the holes in the hand at certain horizontal lines to count the fingers. By carefully choosing which gestures were the easiest to distinguish, we manage to create an algorithm that works correctly under most circumstances.

This project was implemented using Matlab with the DIPimage toolbox and Java.