Background
Electromyography (EMG) measures the electrical activity generated by muscle contractions. When implemented via multi-channel armbands, EMG enables non-invasive, human-computer interface allowing to control robotic arms, prosthetic or even play video games. Intracranial EEG (iEEG) is a method that uses electrodes placed directly on the surface of or deep within the brain's cortex. This technique allows for recordings of neural signals related to motor planning and execution with very high temporal (millisecond-level) and spatial resolution. This allows for high temporal (millisecond-level) and spatial resolution recordings of neural signals related to motor planning and execution. By integrating cortical activity with peripheral muscle signals, a multimodal system can contextually enrich muscle recordings, reducing ambiguity due to fatigue or signal overlap, and improving overall gesture recognition performance.
Problem Statement
Current EMG armband systems excel at classifying gestures in controlled conditions but often face declining accuracy in realistic usage scenarios characterized by co-contraction, electrode shifts, or muscle fatigue. Although iEEG has been used independently for motor decoding, it remains less explored for gesture recognition. Few studies have systematically fused EMG and iEEG (or EEG) using regression approaches for continuous direct control (i.e. where the system predict not only the gesture, but the actual angle of each joint in the arm to be reproduce by the robotic). This leaves a gap in developing robust, low-latency gesture recognition systems that leverage cortical augmentation to enhance peripheral EMG signals in dynamic, practical environments.
Objective and Methodology
This project's primary objective is to develop and validate a multimodal brain-computer interface (BCI) that integrates surface electromyography (sEMG) from a wearable armband with intracranial EEG (iEEG) signals. The goal is to improve the continuous decoding of hand gestures, aiming for significant enhancements in accuracy, latency, and robustness over systems that rely on sEMG alone. A core challenge to be addressed is the development of a machine learning framework that is resilient to the high inter-subject variability of iEEG signals, which results from clinically-driven electrode placements in epilepsy patients.
Methodology
This project will take place over three semesters (long thesis):
Semester 1: Data Acquisition and Foundational Review. Synchronized multi-channel sEMG and iEEG data will be acquired from patients undergoing pre-surgical evaluation for epilepsy, in collaboration with Rikshospitalet and the University of Oslo's Faculty of Medicine. It is a critical constraint of this study that iEEG electrode placement is determined solely by clinical requirements for seizure localization. This results in varied cortical coverage between subjects, demanding a highly adaptable modeling approach. In parallel, a comprehensive literature review will be conducted to analyze sEMG and iEEG motor signal features, evaluate signal fusion strategies, and establish a clear rationale for augmenting EMG-based BCIs with cortical signals.
Semester 2: Signal Processing and Baseline Modeling. A dual pipeline for signal preprocessing and feature extraction will be developed. For the sEMG signals, features will be extracted from the time domain, frequency domain, and through image-based representations of muscle activity. For the iEEG signals, processing will focus on extracting neural correlates of motor intent, such as frequency-specific band power and event-related potentials. To establish performance benchmarks, a suite of sEMG-only gesture recognition models, including ridge regression, random forests, and convolutional neural networks (CNNs), will be trained and evaluated.
Semester 3: Multimodal Fusion and System Validation. The final phase will concentrate on designing and implementing fusion models that effectively combine the two data streams. Several architectures will be investigated, ranging from early-stage feature concatenation to more sophisticated strategies like models with separate encoders and shared decision layers, or decision-level ensembling. Advanced networks utilizing attention mechanisms or transformer-based architectures will be explored to optimally integrate the disparate signal types. The performance of the fused models will be compared against the sEMG-only baselines. Ablation studies will be performed to quantify the performance contribution of the iEEG data. The project will culminate in the writing of a master's thesis and the preparation of a manuscript for submission to a peer-reviewed journal.
Expected Outcomes
The anticipated outcome is a multimodal regression model that outperforms EMG-only systems in gesture recognition accuracy, latency, and stability. The project also aims to produce an interpretable comparison of fusion strategies, open-source code, and scholarly publications. Clinically, this may enable progress toward reliable, real-time, neuroprosthetic control with hybrid neural-muscular inputs.