InfoMus Lab

Multimedia techniques for therapy and rehabilitation

The idea of aesthetic resonance is to give patients a visual and acoustic feedback depending on a qualitative analysis of their (full-body) movement, in order to evoke ludic aspects (and consequently introduce emotional-motivational elements) without the need of the rigid standardization required in typical motion analysis labs, or invasive techniques: the subjects are free to move in 3D with no sensors/markers on body.

The concept of aesthetic resonance was worked out in the framework of the EU-IST project CARE HERE and experiments were carried out aiming at planning, developing and validating aesthetically resonant environments for different types of sensorimotor impairments. 

From a technical point of view the aim was twofold:

• To develop a computational open architecture enabling (i) the integration of modules for gesture analysis and recognition and for real-time generation of multimedia feedback, (ii) the design of dynamic and interactive therapeutic exercises, and (iii) the performance of  the exercises in real-time.
• To develop algorithms for real-time motion analysis that, despite the limitation given by the lack of on-body markers/sensors, are reliable and precise enough (i) to enable the generation of suitable audio and visual feedbacks in aesthetically resonant environments, and (ii) to allow the therapist to evaluate the progress of the therapy by monitoring the measures associated to a collection of motion features the algorithms provide him.   

In particular a pilot experiment was carried out in collaboration with the Centro di Bioingegneria at Ospedale La Colletta - ASL 3 Genovese in order to test the developed techniques on patients with Parkinson's disease. The EyesWeb open software platform was adopted as the basic framework in which the motion analysis techniques were integrated and the therapeutic exercises developed.

Several exercises were developed for experiments with Parkinson's  patients. One of them, for one patient at a time, allows the subject to paint using his/her body (see Figure above). The patient sees himself on a large screen painting in real-time through his/her motion in the space. The interaction is based on some measured movement cues. For example, the color may depend on fluency; the amount of detected motion may be associated to intensity of the color trace; pauses in movement allow restarting the process and re-assigning/adapting interaction mappings. In this way, by a careful choice of colors, e.g., by creating "pleasant" color associations/mappings with fluent and non-hesitating movements, it is possible to create a sort of visual feedback encouraging improvement of movement in patients. During this exercise the subject looks at the picture painted on the monitor and continuously changes it while moving.

Main reference

A. Camurri, B. Mazzarino, G. Volpe, P Morasso, F. Priano, C. Re
Application of multimedia techniques in the physical rehabilitation of Parkinson's patients, Journal of Visualization and Computer Animation, 14(5), pp. 269-278, December 2003

Back to previous page