InfoMus Lab

Analysis of Expressive Gesture in Human Full-body Movement

 

Our research on expressive gesture is trying to find possible answers to questions such as the following ones: which are the features in expressive gesture that are mainly responsible of conveying expressive content? How can they be measured? How is the temporal dynamics of such features related to the communication of different expressive contents? Is it possible to build a classifier able to automatically classify expressive gestures on the basis of the expressive content they convey? Are the outputs of the automatic classifier consistent with spectators' perception of expressive gestures?   

Human full-body movement is as a first-class conveyor of expressive content in non-verbal interaction based on expressive gesture. It can therefore be considered as a suitable test-bed for studying the mechanisms underlying communication of expressive content to expressive gesture.

In this perspective our research focused on individuating, extracting, and analyzing features in human full-body movement that are related with communication of expressive content.

Research is inspired to several sources ranging from approaches grown in the traditional fields of science (e.g., psychology) and engineering (e.g., biomechanics) to approaches derived from theories from art and humanities (e.g., choreography, music composition), to approaches related to theories on KANSEI and KANSEI Information Processing. Rudolf Laban's Theory of Effort, Pierre Schaeffer's Sound Morphology, the psychological researches of Wallbot, Argyle, Boone and Cunningham are some examples of sources on which our research is grounded.

This approach implies a strict cooperation with choreographers and dancers (we work closely with professional choreographer and dance teacher Giovanni Di Cicco), aiming at (i) better understanding of expressive content in movement through discussion with experts of the field and (ii) recording a reference archive of significant movement patterns as a test-bed for analysis algorithms.

Research objectives include:

  • The definition of a methodology for individuating the relevant features and their role in communication of expressive content in human full-body movement.
  • The development of algorithms for measuring global expressive cues from human full-body movement, the analysis of such cues, and their use for attempting an automatic classification of dance fragments in term of conveyed expressive content.
  • The design of experiments for empirical validation of the models and algorithms.

Concrete outcomes are:

  • A collection of software module for analysis of expressive gesture in human-full body movement: such modules have been integrated in the EyesWeb open platform and are included in the EyesWeb Expressive Gesture Processing Library.
  • A collection of public events in which the developed models and algorithms have  been employed in real performances.
  • A collection of scientific papers describing our research in details. Some of them are listed here below.

 

Main references

A. Camurri, B. Mazzarino, M. Ricchetti, R. Timmers, G. Volpe 
Multimodal analysis of expressive gesture in music and dance performances, in A. Camurri, G. Volpe (Eds.), "Gesture-based Communication in Human-Computer Interaction", LNAI 2915, Springer Verlag, 2004.

A. Camurri, B. Mazzarino, G. Volpe
Analysis of Expressive Gesture: The EyesWeb Expressive Gesture Processing Library, in A. Camurri, G. Volpe (Eds.), "Gesture-based Communication in Human-Computer Interaction", LNAI 2915, Springer Verlag, 2004.

A. Camurri, B. Mazzarino, G. Volpe, P Morasso, F. Priano, C. Re Application of multimedia techniques in the physical rehabilitation of Parkinson's patients, Journal of Visualization and Computer Animation, In press.

A. Camurri, B. Mazzarino, G. Volpe
Expressive interfaces, Cognition, Technology & Work, Springer-Verlag, Published on line, December 2003.

A. Camurri, I. Lagerlöf, G. Volpe 
Recognizing Emotion from Dance Movement: Comparison of Spectator Recognition and Automated Techniques, International Journal of Human-Computer Studies, 59(1-2), pp. 213-225, Elsevier Science, July 2003.

 

Back to previous page