Real-time, low-latency sonification of complex movements

The research task of the IMS within the low-latency movement sonification project is the analysis of multiple hardware platforms. In general, a mobile hardware platform has to be established. The image above shows a healthy subject wearing sensors while performing exercise.

The data-flow comprises upper-body motion capturing by inertial mesurement units (IMU), the computation of movement features based on the IMU data and the synthesis of a movement dependent and pleasant sound.

Within the project real-time demonstrator based on various different hardware platforms have been established:

• General Purpose Processor (GPP)
  • Desktop PC, notebbok
• Reduced Instruction Set Processor (RISC)
  • Texas Instrumens Beagle Bone, BlackBerry Z10 smartphone
• RISC / DSP (Digital Signal Processor)
  • TI Integra development board
• Field Programmable Gate Array (FPGA)
  • Emulation of a Tensilica Xtensa LX4 customizable processor (ASIP) comprising a customized instruction set and dedicated hardware accelerators
  • Altera Nios II softcore processor comprising a customized instruction set and dedicated hardware accelerators

Besides the commercial Xsens MTx IMU a custom wireless IMU platform called IMS IMU has been established.

Participating institutes and companies are the Institute of Sports Science of the Leibniz Universität Hannover (Prof. Dr. Effenberg) and the Institute of Music Physiology and Musicians' Medicine of the Hanover University of Music, Drama and Media.

A short video clip published as "Experiment der Woche" of the Leibniz Universität Hannover shows the results of perliminary studies.