We delivered a simple game with multiple levels where you guide a character through a maze using your hands to determine the direction. The pre-determined hand gestures were captured using a simple low-end webcam, and worked at a decent success rate.
The main challenge consisted in choosing suitable hand gestures that were easy to recognize by applying several operators on the captured image, retaining robust performance under varying light intensity, background color, skin color and hand size.
We approached this by first creating a black-white representation of the captured image by filtering certain colors and applying various dilations and erosions on the resulting image. With this representation, we could determine the gesture by measuring the width of the hand at certain heights, or counting the holes in the hand at certain horizontal lines to count the fingers. By carefully choosing which gestures were the easiest to distinguish, we managed to create an algorithm that worked correctly under most circumstances.
This project was implemented using Matlab with the DIPimage toolbox and Java.