InfoMus Lab

The EyesWeb Open Platform

 

The EyesWeb open platform (free download here) has been originally conceived for supporting research on multimodal expressive interfaces and multimedia interactive systems. EyesWeb has also been widely employed for designing and developing real-time dance, music, and multimedia applications.
It supports the user in experimenting computational models of non-verbal expressive communication and in mapping gestures from different modalities (e.g., human full-body movement, music) onto multimedia output (e.g., sound, music, visual media). It allows fast development and experiment cycles of interactive performance set-ups by including a visual programming language allowing mapping, at different levels, of movement and audio into integrated music, visual, and mobile scenery.

EyesWeb has been designed with a special focus on the analysis and processing of expressive gesture in movement,  midi, audio, and music signals. It was the basic platform of the EU-IST Project MEGA and it has been employed in many artistic performances and interactive installations. However, use of EyesWeb is not limited to performing arts. Museum installations, entertainment, edutainment, therapy and rehabilitation are just some of a wide number of different application domains that can benefit of using it. For example, EyesWeb has been adopted as standard in other EU IST projects such as MEDIATE and CARE HERE in the therapy and rehabilitation field, and EU TMR MOSART. In the EU Sixth Framework Program it has been adopted as standard in the IST TAI-CHI project and it is employed in the HUMAINE and ENACTIVE Networks of Excellence. EyesWeb users include universities, public and private research centers, companies, and private users.

EyesWeb consists of a number of integrated hardware and software modules which can be easily interconnected and extended. The EyesWeb software includes a development environment and a set of libraries of reusable software components (blocks) which can be assembled by the user in a visual language to build patches as in common computer music languages inspired to analog synthesizers. A patch can be used as a module in a higher-level patch. 

EyesWeb includes a software Wizard enabling users to extend the system with new modules, datatypes, and libraries. 

The software runs on Win32 and is based on the Microsoft COM/DCOM standard. It supports Steinberg VST and ASIO, OSC (Open SoundControl), and FreeFrame plugins.

Automation support is provided. This let developers to invoke EyesWeb from other languages, such as Microsoft Visual Basic. The execution of a patch can be controlled from external applications. Any scripting language that supports automation can be used for this purpose (e.g., Phyton, VBScript, JavaScript, etc.).

Modules include passive modules (e.g., filters) and active modules, e.g., sources and modules with an internal dynamics (i.e., modules which receive inputs as any other module but may send outputs asynchronously with respect to their inputs). 

EyesWeb libraries include (since version 2.5):

  1. Input: support for frame grabbers (from webcams to professional videocameras), wireless on-body sensors (e.g., accelerometers), audio and MIDI input, serial, network, keyboard, mouse;
  2. Math and filters (e.g., operations with scalars and matrices);
  3. Imaging (processing and conversions of images)
  4. Sound and MIDI Libraries
  5. Communication (e.g., MIDI, OSC, TCP/IP, serial, DCOM);
  6. Motion Analysis: motion trackers (e.g., feature tracking, tracking of multiple colored blobs), modules for extraction of global features from movement (e.g. amount of detected motion), analysis of the use of the space. 
  7. Mapping of extracted features in real-time generation of audio and visual outputs.
  8. Output: visual, audio, MIDI, serial, network.

Our sensor systems are integrated in the EyesWeb environment and can be used for movement analysis. 

EyesWeb also supports multiple videocameras and various sensors, e.g., on-body wireless sensor systems, according to the requirements of the specific movement analysis task.

Previous and current versions of the EyesWeb system have been used in various artistic performances (e.g., in Luciano Berio's Cronaca Del Luogo - Salzburg Festival 1999), and museum exhibits (e.g., in 12 interactive exhibits at Cittą della Scienza science center, Napoli, Italy).

See the publications section for recent papers on EyesWeb platform and research.

 

Back to previous page