Vision based multi-user human computer interaction

Abstract

This paper proposes a vision-based Multi-user Human Computer Interaction (HCI) method for creating augmented reality user interfaces. In the HCI session, one of the users' hands is selected as the active hand. The fingers of the active hand are employed as input devices to trigger functionalities of the application program. To share the token of interaction among the users, the HCI session is modeled as a Finite State Machine (FSM). The FSM is composed of the initial and steady states. In the initial state, the FSM identifies the active hand by tracking the hand with the maximum moving speed. Then the FSM enters the steady state to carry out the HCI session. At the end of each individual HCI cycle, the FSM polls requests from other hands for acquiring the role of active hand. If such requests are sensed, the FSM returns to the initial state to track a new active hand. Otherwise, the HCI session is continuously carried out by the current active hand. Test results show that the resultant user interface is efficient, flexible and practical for users with problems on using ordinary input devices. In a desk-top computer equipped with a 640 x 480 resolution web-camera, the HCI session can be successfully conducted when the operation distance ranges from 30 to 90 cm.