Background
The intersection of biosignal processing and music technology offers exciting possibilities for innovative human-computer interactions. Biosignals such as actigraphy (movement data), electrocardiography (heart activity) and muscle activity measured through electromyography (EMG) provide rich, real-time insights into the body's dynamics. Translating these signals into auditory outputs can extend musical expression, allowing users to create and manipulate music through natural movements and muscle engagement. This approach not only enhances artistic performance but also holds potential for therapeutic applications, where music and movement play crucial roles in rehabilitation and well-being.
Problem Statement
Traditional musical instruments rely on physical manipulation to produce sound, which can limit the scope of interaction and expression. This presents an opportunity to explore how physiological signals—such as heart rate, muscle activity, or movement—can serve as a medium for musical creation. By harnessing these biosignals, we can develop instruments that allow users to interact with music through their body's natural functions, enhancing expressiveness and accessibility. This project aims to design an instrument that translates physiological signals into sound, opening up new possibilities in how we create and experience music.
Objective and Methodology
The objective of this project is to design and develop a musical controller that utilizes various physiological biosignals to generate and manipulate music in real time. By capturing data such as heart rate, muscle activity, and movement through wearable sensors, the system will translate physiological states and natural human motions into auditory outputs, enabling users to produce music through their physiological signal.
The methodology involves recruiting participants to collect a comprehensive dataset of biosignals during various activities and states. Wearable devices equipped with ECG, EMG, accelerometers, and gyroscopes will capture the necessary data. The collected signals will undergo preprocessing to reduce noise and standardize the data across individuals. Feature extraction techniques will identify meaningful attributes from the raw data, such as heart rate variability, muscle activation levels, movement intensity, and temporal patterns.
Machine learning algorithms will be employed to map these features to musical parameters like pitch, volume, tempo, and modulation effects. An intuitive software platform will be developed to process the biosignals in real time and interface with digital audio workstations or synthesizers. User testing will be conducted to evaluate the system's usability and expressiveness, with iterative refinements made based on feedback.
Data Validation and Significance
Validation will focus on the effectiveness of the auditory mappings and the user experience. Technical assessments will ensure that the biosignals are accurately captured and translated into sound in real time. User evaluations will provide insights into how intuitive and expressive the instrument is, and how well it enables users to explore new musical possibilities. By successfully demonstrating that physiological signals can be harnessed for musical creation in an accessible and engaging way, the project will highlight the potential of biosignal-based instruments to transform musical expression.
Expected Outcomes
- A functional prototype of a musical controller that translates various physiological biosignals into real-time auditory outputs.
- A software platform that processes biosignals and allows users to customize how their physiological states and movements map to musical parameters.
- Contributions to the fields of biosignal processing and music technology through documentation and potential publications.
- A foundation for future applications in areas such as rehabilitation therapy, interactive performances, and educational tools.