The research theme titled "AI for artistic performances" concentrates on how real-time interaction with AI systems transforms the creative process in applications such as music and art performance, interactive installations, and gaming. This area emphasizes the use of multiple and embodied AI agents to facilitate continuous interactions between humans and machine agents, supporting live, dynamic, and interactive performance in contrast to creating a fixed final creative product. To achieve personalized and novel outputs, this research theme balances data-based approaches with artists’ knowledge and search-based methods, with the ultimate goal being to create new AI algorithms, open-source software and hardware, and human-machine interaction methods for real-time artistic applications.
Example projects
We list some example projects within this topic, but feel free to suggest or ask for alternative themes.
Evolutionary search for novel solutions using deep learning models to evaluate aesthetic quality. The project would use some kind of quality-diversity (QD) algorithms to search for new and interesting artifacts (new sounds, images, robot designs, …), but instead of manually defining a measure of how interesting the artifacts are, we would like to explore using deep learning models, e.g. vision-language models (VLMs). One could explore how well these models work out of the box, or explore possibilities of fine-tuning models to individual preferences. Another interesting venue to explore in this context would be interactive quality-diversity algorithms, combining machine evaluation with human guidance.
IMPSY intelligent musical instruments. The project would use the IMPSY platform as a basis for developing intelligent instruments. Within this framework, there are several possibilities: a) Exploring algorithms for music generation and co-creativity with human performers. In a live performance setting, what kind of interaction strategies should the instruments have? Can the instruments adapt to the situation, sensing the performance and the intent of the human musician? b) Exploring multi-agent interaction between multiple instrument modules, with or without human interaction. What kind of algorithms / models are suitable for different types of performances, and what are the interaction dynamics?
This project could be executed with a physical setup, using embedded systems and small synthesizer modules, but simulations or fully laptop-based implementations are also possible.