InfoMus Lab |
||
Expressive Gesture
One of the main goals of our research is to explore paradigms of interaction between humans and machines in the framework of multimodal environments for music, theatre, museum exhibits, and art installations. We deem that the process of designing interactive systems can benefit by adding a new channel: expressiveness. This would allow the design and development of novel expressive interfaces taking explicitly into account the processing and communication of information related to the emotional, affective sphere. In this perspective, the modeling of expressive gesture, considered as a main non-verbal communication channel, is a central issue in our work. In artistic contexts and in particular in the field of performing arts, gesture is often not intended to denote things or to support speech as in the traditional framework of natural gesture, but the information it contains and conveys is related to the affective/emotional domain. From this point of view, gesture can be considered "expressive" depending on the kind of information it conveys: expressive gesture carries what Cowie and colleagues call "implicit messages", and what Hashimoto calls KANSEI. That is, expressive gesture is the responsible of the communication of information that we call expressive content. Expressive content is different and in most cases independent from, even if often superimposed to, possible denotative meaning. Expressive content concerns aspects related to feelings, moods, affect, intensity of emotional experience. Our concept of expressive gesture is extended with respect to the traditional concept of gesture (e.g., as defined by Kurtenbach and Hulteen), since it considers also cases in which, with the aid of technology, communication of expressive content takes place even without an explicit movement of the body, or, at least, the movement of the body is only indirectly involved in the communication process (e.g., the allusion at movement in musical signals). Our research on expressive gesture is finalized to the development of interactive multimedia systems based on novel interaction paradigms enabling a deeper experience and participation of the user by explicitly observing and processing his/her (multimodal) expressive gesture. Since artistic performance uses non-verbal communication mechanisms to convey expressive content, we focused on performing arts, and in particular on dance and music, as the main test-beds where computational models of expressive gesture and algorithms for expressive gesture processing can be developed, studied, and tested. In a sense,
our work can be considered as an attempt to bridge the gap between these
science and humanities toward the common goal of understanding expressive
gesture and exploiting its communicative power under a scientific perspective
(i.e., a deeper understanding of non-verbal communication channels), an
engineering perspective (i.e., building enhanced and effective interactive
systems for several different application domains), and an artistic perspective
(i.e., exploiting the means technology provides in order to enrich language and
to pioneer novel art forms). In this scenario our research investigated some peculiar aspects of expressive gesture modeling and processing:
Main references A. Camurri, G. Volpe (Eds.) A. Camurri, B. Mazzarino, M. Ricchetti, R. Timmers,
G. Volpe A. Camurri, B. Mazzarino, G. Volpe A. Camurri, I. Lagerlöf, G. Volpe |
||