The need for autonomous, robotic solutions is increasing along with a larger focus on sustainable ocean management and the emerging of marine industries such as offshore CO2 storage. Autonomous Underwater Vehicles (AUVs) have excellent marine monitoring capabilities but have an unfulfilled potential in their ability to use sensor input in real-time for autonomous and optimized mapping of emissions such as gas bubbles and dissolved gas.
The aim of this master project is to explore how RL can be used to train for environmental surveys, based on a dataset of high-resolution oceanographic simulations of gas emission. Each simulation constitutes a database where for each point in space and time, the concentration of dissolved gas or presence of gas bubbles can be queried. The simulation includes tidal currents, so the gas plumes will flow back and forth in the water. One can regard the simulation as a large, digital aquarium where a software agent can perform surveys with simulated sensors. By doing many surveys, it is possible to use RL to train neural models to guide an AUV during fully autonomous surveys.
Within the RL context there are several possible directions to take based on these environments. For instance hierarchical RL, with a meta-agent choosing between a suite of expert agents, or evaluate different neural models for goal achievement or training efficiency.
The project will be carried out in collaboration with the Norwegian Geotechnical Institute (NGI), which is involved in several projects related to environmental monitoring of gas leakage in the oceans.