Hi There!

I'm Dan Schlegel, an Assistant Professor in the Computer Science Department at SUNY Oswego


One of the dreams of nearly all computer users is to be able to sit back in their chair, point at their screen and be able to interact with the machine using the least effort possible. We have created a vision system which uses finger tracking and gesture recognition to begin to make this dream a reality. The cameras used for this are off-the-shelf webcams and laptops built in webcams.

The user can point at the screen to click something, or use two fingers to scroll easily up and down a page. As has been discovered by many others creating novel interfaces it is very difficult to adapt a system which is defined for the very fine control of a mouse pointer. For this reason it is important to have an expressive set of gestures. Currently we only have gestures for moving the mouse pointer, scrolling and clicking, but these alone have made web browsing possible using this interface.

Our gesturing system is novel because we have the users gesture using what we are calling a plane of interaction (the AirTouch Plane). Gestures have different functions when they are in or out of this plane. This is the equivalent to the touching of a touch screen, a very popular interface metaphor.

The gesture system uses a homography to map the gestures given as input to the camera to a space behind the screen of the computer so that the user points to the place they mean on the screen and not to a place related to it on the camera.

For tracking we are using a modified version of the Continuously Adaptive Mean-Shift algorithm. We have made adjustments to the algorithm to automatically re-acquire a target when it is lost due to very quick movement and leaving/re-entering the frame.

This project was started as a group term project for Dr. Jason Corso‘s “Vision for HCI” course and was continued for about a year thereafter. Our group contained myself and Jeff Delmerico. AirTouch is no longer under active development.

Read More about AirTouch