Video Analysis of Music-related Body Movement in Matlab

Description

The Musical Gestures Toolbox (MGT) was developed in the graphical programming environment Max for analysing music-related body movement. Now we are looking for a student to port the existing code to Matlab, and to extend the toolbox further. This includes the following features:

• Easy import and playback of a large variety of video formats
• Creation of various types of visualisations: motion history images, motion history keyframe images, motiongrams, etc.
• Calculation of various types of global motion features: Quantity of Motion, Area of Motion, Direction of Motion, etc.
• Integration with MoCap Toolbox and MIR Toolbox for easy analysis of video together with motion capture and sound data
• Classification of motion data together with sound data

Background

The project will build on these dissertations:

• A. R. Jensenius. Action–Sound : Developing Methods and Tools to Study Music-Related Body Movement. PhD thesis, University of Oslo, 2007.
• H. Kirkerød. Optical flow applied to infant movement. Master’s thesis, Norwegian University of Science and Technology (NTNU), Department of Engineering Cybernetics, 2010.
• S. F. Olsen. Automatisk segmentering av bevegelsesdata. Master’s thesis, Norwegian University of Science and Technology (NTNU), Department of Engineering Cybernetics, 2010.

and these existing toolboxes:

• Musical Gestures Toolbox (for Max) (no longer available)
• Mocap Toobox (for Matlab)
• MIR Toolbox (for Matlab)

Necessary background knowledge:

• Matlab
• Digital video processing and/or computer vision
• Preferably also digital signal processing and/or computer audition