The recognition and comprehension of hand gestures - a review and research agenda

Timo Sowa

pp. 38-56

in: Ipke Wachsmuth, Günther Knoblich (eds), Modeling communication with robots and virtual humans, Berlin, Springer, 2008

Abstract

In this paper I review current and past approaches towards the use of hand gesture recognition and comprehension in human-computer interaction. I point out properties of natural coverbal gestures in human communication and identify challenges for gesture comprehension systems in three areas. The first challenge is to derive the meaning of a gesture given that its semantics is defined in three semiotic dimensions that have to be addressed differently. A second challenge is the spatial composition of gestures in imagistic spaces. Finally, a third technical challenge is the development of an integrated processing model for speech and gesture.