Kai Olav Ellefsen

• Bruin, Ege de; Glette, Kyrre & Ellefsen, Kai Olav (2026). Lamarckian Inheritance in Dynamic Environments: How Key Variables Affect Evolutionary Dynamics. ALIFE : Proceedings of the Artificial Life Conference. isal2026. doi: https:/direct.mit.edu/isal/proceedings/isal2026/38/35/138242. Full text in Research Archive
• Kvalsund, Mia-Katrin Ose; Ellefsen, Kai Olav; Glette, Kyrre; Pontes-Filho, Sidney & Lepperød, Mikkel Elle (2026). Sensor movement drives emergent attention and scalability in active neural cellular automata. Neural Networks. ISSN 0893-6080. 200. doi: 10.1016/j.neunet.2026.108798. Full text in Research Archive
• Lømo, Tobias; Baselizadeh, Adel; Ellefsen, Kai Olav & Tørresen, Jim (2026). Dual Process Dreamer: Fast and Slow Decision-Making with World Models. Proceedings of the International Conference on Agents and Artificial Intelligence (ICAART). ISSN 2184-3589. 2, p. 1230–1241. doi: 10.5220/0014243200004052. Full text in Research Archive Show summary

  Most robot systems are based on a single decision-making process. This process needs to balance time, energy, and accuracy in every situation. However, according to ”dual process theory” (DPT) from cognitive psychology, this is not how humans work. Depending on the situation, we have the ability to switch between two thinking methods, a fast system 1 (S1) and a slower system 2 (S2). In this paper, we propose a novel approach to a dual process architecture for robots and agents. Our method, called Dual Process Dreamer (DPDreamer), is a combination of a reinforcement learning policy network, a planning algorithm, and a learned world model. The world model allows the parts of DPDreamer to work together and create a more integrated system compared to previous proposals of DPT systems. DPDreamer was tested in a puzzle game called Sokoban, and by balancing the use of S1 and S2, DPDreamer managed a success rate similar to S2 while using S1 most of the time, showing the benefit of using a more adaptable system.

• Watanabe, Shin; Horn, Geir; Tørresen, Jim & Ellefsen, Kai Olav (2025). Integrating Bilevel Planning and Offline Skill Learning for Enhancing Mobile Manipulation, 2025 IEEE 21st International Conference on Automation Science and Engineering (CASE). Institute of Electrical and Electronics Engineers (IEEE). ISSN 9798331522476. p. 2275–2280. doi: 10.1109/CASE58245.2025.11163881. Full text in Research Archive Show summary

  Solving complex robotic mobile manipulation tasks requires both planning the sequence of skills to execute and learning how to robustly execute each skill. Planning-based approaches such as task and motion planning (TAMP) can help train skills more efficiently through demonstrations, while learning-based approaches such as reinforcement learning (RL) can help plan tasks more quickly through heuristics. This paper presents a novel approach to generalizing mobile manipulation tasks by synergistically combining sampling-based TAMP and value-based RL. The TAMP solver first generates suboptimal demonstration trajectories of a particular skill, from which an offline RL algorithm distills a robust policy and a value function, the latter serving as a skill feasibility classifier. The policy and the classifier are both fed back into the TAMP workflow not only to improve the skill success rate but also to speed up the planner by sampling robot configurations for which the skill is likely to succeed. We evaluate the approach on a simulated block-pushing domain. Re-purposing a byproduct of an offline skill-learning process leads to an integrated planning and learning system that exploits the awareness of its own skill competence.

• Bruin, Ege de; Glette, Kyrre & Ellefsen, Kai Olav (2025). Integrating Sample Inheritance into Bayesian Optimization for Evolutionary Robotics. In Witkowski, Olaf; Adams, Alyssa M.; Sinapayen, Lana; Baltieri, Manuel & Khosravy, Mahdi (Ed.), ALIFE 2025: Ciphers of Life: Proceedings of the Artificial Life Conference 2025. MIT Press. doi: 10.1162/ISAL.a.866. Full text in Research Archive Show summary

  In evolutionary robotics, robot morphologies are designed automatically using evolutionary algorithms. This creates a body-brain optimization problem, where both morphology and control must be optimized together. A common approach is to include controller optimization for each morphology, but starting from scratch for every new body may require a high controller learning budget. We address this by using Bayesian optimization for controller optimization, exploiting its sample efficiency and strong exploration capabilities, and using sample inheritance as a form of Lamarckian inheritance. Under a deliberately low controller learning budget for each morphology, we investigate two types of sample inheritance: (1) transferring all the parent’s samples to the offspring to be used as prior without evaluating them, and (2) reevaluating the parent’s best samples on the offspring. Both are compared to a baseline without inheritance. Our results show that reevaluation performs best, with prior-based inheritance also outperforming no inheritance. Analysis reveals that while the learning budget is too low for a single morphology, generational inheritance compensates for this by accumulating learned adaptations across generations. Furthermore, inheritance mainly benefits offspring morphologies that are similar to their parents. Finally, we demonstrate the critical role of the environment, with more challenging environments resulting in more stable walking gaits. Our findings highlight that inheritance mechanisms can boost performance in evolutionary robotics without needing large learning budgets, offering an efficient path toward more capable robot design.

• Nergård, Katrine Linnea; Ellefsen, Kai Olav & Tørresen, Jim (2025). Fast or Slow: Adaptive Decision Making in Reinforcement Learning with Pre-Trained LLMs. In Ugur, Emre; Sciutti, Alessandra & Rohlfing, Katharina (Ed.), 2025 IEEE International Conference on Development and Learning (ICDL). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798331543433. doi: 10.1109/ICDL63968.2025.11204357. Full text in Research Archive
• Waarum, Ivar-Kristian; Blomberg, Ann Elisabeth Albright; Krogstad, Thomas Røbekk; Dewar, Marius & Ellefsen, Kai Olav (2025). Optimal Sampling Patterns for Robotic Environmental Monitoring. International Conference on Control, Automation and Robotics (ICCAR). ISSN 2251-2446. 2025(2025), p. 163–169. doi: 10.1109/iccar64901.2025.11073028. Full text in Research Archive Show summary

  Cost-efficient methods and technologies for environmental monitoring is an enabler for holistic decision making and sustainable management of natural resources. The information value from monitoring surveys with robotic sensor platforms such as AUVs and UAVs can increase if domain knowledge is exploited when selecting the sampling locations. We investigate how to maximize the information value of surveys targeting dispersible substances by adjusting the survey pattern in accordance with the direction and strength of the wind or water currents. Samples are acquired with a range of different patterns, deployed in a large number of randomly generated emission scenarios in a Monte Carlo framework. The information value from each survey is used as a performance metric to identify opportune pattern configurations. Among the results is the impact that the orientation of the survey pattern relative to the direction of the wind has on the information value, and that given proper orientation the line spacing can be wider. The results are used to suggest guidelines for robotic environmental monitoring surveys.

• Bruin, Ege de; Glette, Kyrre & Ellefsen, Kai Olav (2025). Generational Replacement and Learning for High-Performing and Diverse Populations in Evolvable Robots. In Obafemi-Ajayi, Tayo (Eds.), 2025 IEEE Symposium on Computational Intelligence in Artificial Life and Cooperative Intelligent Systems. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798331508388. doi: 10.1109/alife-cis64968.2025.10979828. Full text in Research Archive Show summary

  Evolutionary Robotics offers the possibility to design robots to solve a specific task automatically by optimizing their morphology and control together. However, this co-optimization of body and control is challenging, because controllers need some time to adapt to the evolving morphology - which may make it difficult for new and promising designs to enter the evolving population. A solution to this is to add intra-life learning, defined as an additional controller optimization loop, to each individual in the evolving population. A related problem is the lack of diversity often seen in evolving populations as evolution narrows the search down to a few promising designs too quickly. This problem can be mitigated by implementing full generational replacement, where offspring robots replace the whole population. This solution for increasing diversity usually comes at the cost of lower performance compared to using elitism. In this work, we show that combining such generational replacement with intra-life learning can increase diversity while retaining performance. We also highlight the importance of performance metrics when studying learning in morphologically evolving robots, showing that evaluating according to function evaluations versus according to generations of evolution can give different conclusions.

• Fatima, Safia; Ellefsen, Kai Olav & Moonen, Leon (2025). Self Healing of a Mixed Autonomy Traffic System Using Reinforcement Learning and Attention. IEEE Open Journal of Intelligent Transportation Systems. 6, p. 1200–1220. doi: 10.1109/OJITS.2025.3606539. Full text in Research Archive Show summary

  As urban traffic becomes increasingly complex with the integration of connected and autonomous vehicles alongside human-driven vehicles, there is a critical need for adaptive traffic management systems capable of self-healing in response to disruptions. This paper introduces TS2RLA (“Traffic System Recovery using Reinforcement Learning and Attention”), a novel framework for self-healing in mixed-autonomy traffic systems by combining deep reinforcement learning with an attention mechanism to optimize traffic flow and recover from faults in various scenarios in a mixed-autonomy traffic environment. We evaluated TS2RLA in four complex traffic scenarios: bottleneck, figure-eight, grid, and merge. Our results demonstrate significant improvements over the baseline model, showing an average of 86.74% reduction in crashes, 71% improvement in speed and traffic throughput, and robust performance under diverse and complex traffic conditions. Moreover, our experiments show that TS2RLA leads to a significant reduction in CO2 emissions and fuel consumption. TS2RLA’s attention-based approach shows particular benefits in bottleneck and figure-eight scenarios, demonstrating its ability to adapt to complex, multi-factor traffic situations. For scenarios that TS2RLA had not been trained on before, it performs even more favorably than the baseline, with a 96.8% crash reduction and 95.3% throughput improvement. This shows its ability to adapt effectively to new traffic conditions. Overall, we conclude that TS2RLA could significantly improve the safety, efficiency, and capacity of real-world traffic systems, particularly in dynamic urban environments. As such, our work contributes to the field of intelligent transportation systems by offering a versatile self-healing framework capable of managing the complexities of mixed-autonomy traffic.

• Waarum, Ivar-Kristian; Hove, Alouette van; Krogstad, Thomas Røbekk; Ellefsen, Kai Olav & Blomberg, Ann Elisabeth Albright (2025). Mixtures of Gaussian processes for robotic environmental monitoring of emission sources. Environmental Monitoring and Assessment. ISSN 0167-6369. 197(6). doi: 10.1007/s10661-025-14059-6. Full text in Research Archive Show summary

  Emission of greenhouse gases such as methane and carbon dioxide is a known driver of atmospheric heating. Traditional and emerging industries need innovative solutions to comply with increasingly strict sustainability demands and document environmental impact. Mobile sensor platforms such as aerial or underwater vehicles with a high degree of autonomy present a cost-efficient option for environmental monitoring. Autonomous vehicles commonly use Gaussian processes (GPs) for online statistical modelling of concentrations of environmental features. Emission sources in the monitoring area introduce a complication, since the variance is likely heterogeneous between areas dominated by influx and areas with background concentrations. Mixtures of GPs have previously been demonstrated to be effective in such scenarios. Mixture methods distinguish between the natural background concentration and emission to improve model performance when predicting concentrations and variance at unsampled locations. The mixing of GP models allows for nonstationarity and anisotropy in the modelled spatial dynamics, which is desirable for emission modelling in environments with advective forces such as wind or water current. In this paper, we compare different approaches to spatial concentration modelling that accommodate heterogeneous dynamics, based on mixtures of GPs. Distinction of background and emission is either data-driven or derived from domain knowledge. The predictive performance of different mixture methods is demonstrated on field measurements near emissions and compared in an online path planning context. We identify and discuss important trade-offs between data-driven and knowledge-based clustering of measurements. Results show that mixture methods give realistic variance estimates, suitable for online planning.

• Watanabe, Shin; Horn, Geir; Tørresen, Jim & Ellefsen, Kai Olav (2024). Improving Robot Skills by Integrating Task and Motion Planning with Learning from Demonstration. In Capi, Genci (Eds.), 2024 6th International Conference on Control and Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798331518158. doi: 10.1109/iccr64365.2024.10927043. Full text in Research Archive
• Reilstad, Didrik Spanne & Ellefsen, Kai Olav (2024). Cultivating Diversity: a Comparison of Diversity Objectives in Neuroevolution, Applications of Evolutionary Computation. Springer. ISSN 9783031568527. p. 19–35. doi: 10.1007/978-3-031-56855-8_2. Full text in Research Archive
• Thorsen, Ola; Esema, Emmanuel Joseph; Hemaz, Said; Ellefsen, Kai Olav; Herrebrøden, Henrik & Arnim, Hugh Alexander von [Show all 7 contributors for this article] (2024). Can machine learning help reveal the competitive advantage of elite beach volleyball players? In Westphal, Florian; Peretz-Andersson, Einav; Riveiro, Maria; Bach, Kerstin & Heintz, Fredrik (Ed.), 14th Scandinavian Conference on Artificial Intelligence SCAI 2024, June 10-11, 2024, Jönköping, Sweden. Swedish Artificial Intelligence Society. ISSN 9789180757096. p. 57–66. doi: 10.3384/ecp208007. Full text in Research Archive Show summary

  As the world of competitive sports increasingly embraces data-driven techniques, our research explores the potential of machine learning in distinguishing elite from semi-elite beach volleyball players. This study is motivated by the need to understand the subtle yet crucial differences in player movements that contribute to high-level performance in beach volleyball. Utilizing advanced machine learning techniques, we analyzed specific movement patterns of the motion of the torso during spikes, captured through vest-mounted accelerometers. Our approach offers novel insights into the nuanced dynamics of elite play, revealing that certain movement patterns are distinctly characteristic of higher skill levels. One of our key contributions is the ability to classify spiking movements at different skill levels with an accuracy rate as high as 87%. This current research provides a foundation of what separates elite players from their semi-elite counterparts.

• Nilsen, Mathias Kvamme; Nygaard, Tønnes & Ellefsen, Kai Olav (2023). Reward tampering and evolutionary computation: a study of concrete AI-safety problems using evolutionary algorithms. Genetic Programming and Evolvable Machines. ISSN 1389-2576. 24(2). doi: 10.1007/s10710-023-09456-0. Full text in Research Archive Show summary

  Abstract Reward tampering is a problem that will impact the trustworthiness of the powerful AI systems of the future. Reward Tampering describes the problem where AI agents bypass their intended objective, enabling unintended and potentially harmful behaviours. This paper investigates whether the creative potential of evolutionary algorithms could help ensure trustworthy solutions when facing this problem. The reason why evolutionary algorithms may help combat reward tampering is that they are able to find a diverse collection of different solutions to a problem within a single run, aiding the search for desirable solutions. Four different evolutionary algorithms were deployed in tasks illustrating the problem of reward tampering. The algorithms were designed with varying degrees of human expertise, measuring how human guidance influences the ability to discover trustworthy solutions. The results indicate that the algorithms’ ability to find and preserve trustworthy solutions is very dependent on preserving diversity during the search. Algorithms searching for behavioural diversity showed to be the most effective against reward tampering. Human expertise also showed to improve the certainty and quality of safe solutions, but even with only a minimal degree of human expertise, domain-independent diversity management was found to discover safe solutions.

• Strand, Ørjan; Reilstad, Didrik Spanne; Wu, Zhenying; Silva, Bruno Castro da; Tørresen, Jim & Ellefsen, Kai Olav (2023). RADAR: Reactive and Deliberative Adaptive Reasoning - Learning When to Think Fast and When to Think Slow, 2022 IEEE International Conference on Development and Learning (ICDL). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665413121. p. 184–189. doi: 10.1109/ICDL53763.2022.9962202. Full text in Research Archive Show summary

  When designing and deploying Reinforcement Learning (RL) algorithms, one typically selects a single value for the discount rate, which results in an agent that will always be equally reactive or deliberative. However, similarly to humans, RL agents can benefit from adapting their planning horizon to the current context. To enable this, we propose a novel algorithm: RADAR: Reactive and Deliberate Adaptive Reasoning. RADAR enables an agent to learn to adaptively choose a level of deliberation and reactivity according to the state it is in, given that there are cases where one mode of operation is better than the other. Through experiments in a grid world, we verify that the RADAR agent has the capability to adapt its reasoning modality to the current context. In addition, we observe that the RADAR agent exhibits different preferences regarding its thinking modes when a penalty for mental effort is included in its mathematical formulation.

• Norstein, Emma Stensby; Veenstra, Frank; Ellefsen, Kai Olav; Nygaard, Tønnes & Glette, Kyrre (2023). Effects of compliant and structural parts in evolved modular robots. In Iizuki, Hiroyuki; Suzuki, Keisuke; Uno, Ryoko; Damiano, Luisa; Spychala, Nadine; Aguilera, Miguel; Izquierdo, Eduardo; Suzuki, Reiji & Baltieri, Manuel (Ed.), ALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artificial Life Conference. MIT Press. doi: 10.1162/isal_a_00689. Full text in Research Archive Show summary

  A striking difference between animals and traditional robots is that the latter usually have rigid and non-flexible bodies. Animals, on the other hand, exhibit highly adapted traits, such as elastic tendons. The tendons work as springs, storing and releasing kinetic energy during an animal’s gait cycle. Springs have been used in some hand designed robots for similar benefits. However, little research has been done on springs in robots with evolving morphology. We examine the use of compliant and structural modules in modular robots, using a standard evolutionary algorithm. We also look at connections between spring stiffness and robot size using the quality diversity algorithm MAP-Elites. We found that the modular robots evolved to use elastic actuators, and that structural modules enabled morphologies that use less actuators, but still achieve the same walking speed as the robots with actuators in every module. We also observe some indications that larger robots may require lower elasticity.

• Norstein, Emma Stensby; Ellefsen, Kai Olav; Veenstra, Frank; Nygaard, Tønnes & Glette, Kyrre (2023). Evolution of linkages for prototyping of linkage based robots, Proceedings of the 2022 IEEE Symposium Series on Computational Intelligence (SSCI 2022). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665487689. p. 1283–1290. doi: 10.1109/SSCI51031.2022.10022197. Full text in Research Archive Show summary

  Prototyping robotic systems is a time consuming process. Computer aided design, however, might speed up the process significantly. Quality-diversity evolutionary approaches optimise for novelty as well as performance, and can be used to generate a repertoire of diverse designs. This design repertoire could be used as a tool to guide a designer and kick-start the rapid prototyping process. This paper explores this idea in the context of mechanical linkage based robots. These robots can be a good test-bed for rapid prototyping, as they can be modified quickly for swift iterations in design. We compare three evolutionary algorithms for optimising 2D mechanical linkages: 1) a standard evolutionary algorithm, 2) the multi-objective algorithm NSGA-II, and 3) the quality-diversity algorithm MAP-Elites. Some of the found linkages are then realized on a physical hexapod robot through a prototyping process, and tested on two different floors. We find that all the tested approaches, except the standard evolutionary algorithm, are capable of finding mechanical linkages that creates a path similar to a specified desired path. However, the quality-diversity approaches that had the length of the linkage as a behaviour descriptor were the most useful when prototyping. This was due to the quality-diversity approaches having a larger variety of similar designs to choose from, and because the search could be constrained by the behaviour descriptors to make linkages that were viable for construction on our hexapod platform.

• Teigen, Bjørn Ivar; Davies, Neil; Ellefsen, Kai Olav; Skeie, Tor & Tørresen, Jim (2022). Protocol Violations in the WiFi Distributed Coordination Function of Raspberry Pi 4, 2022 17th Wireless On-Demand Network Systems and Services Conference (WONS). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9783903176461. doi: 10.23919/WONS54113.2022.9764440. Full text in Research Archive
• Teigen, Bjørn Ivar; Davies, Neil; Thompson, Peter; Ellefsen, Kai Olav; Skeie, Tor & Tørresen, Jim (2022). ΔQ Generative Models: Modeling Time-Variation in Network Quality, Proceedings of the 2022 18th International Conference of Network and Service Management. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9783903176515. p. 233–239. doi: 10.23919/CNSM55787.2022.9965128. Full text in Research Archive
• Teigen, Bjørn Ivar; Davies, Neil; Ellefsen, Kai Olav; Skeie, Tor & Tørresen, Jim (2022). Quantifying the Quality Attenuation of WiFi, Proceedings of the 47th IEEE Conference on Local Computer Networks. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665480017. p. 189–197. doi: 10.1109/LCN53696.2022.9843690. Full text in Research Archive
• Norstein, Emma Stensby; Ellefsen, Kai Olav & Glette, Kyrre (2022). Open-Ended Search for Environments and Adapted Agents Using MAP-Elites, Applications of Evolutionary Computation. EvoApplications 2022.. Springer. ISSN 9783031024610. p. 651–666. doi: 10.1007/978-3-031-02462-7_41. Full text in Research Archive
• Gorton, Patrick & Ellefsen, Kai Olav (2021). Evaluating Predictive Deep Learning Models. In Yayilgan, Sule Yildirim; Bajwa, Imran Sarwar & Sanfilippo, Filippo (Ed.), Proceeding of the 3rd International Conference on Intelligent Technologies and Applications (INTAP). Springer. ISSN 9783030717117. p. 139–150. doi: 10.1007/978-3-030-71711-7_12. Full text in Research Archive Show summary

  Predicting the future using deep learning models is a research field of increasing interest. However, there is a lack of established evaluation methods for assessing their predictive abilities. Images and videos are targeted towards human observers, and since humans have individual perceptions of the world, evaluation of videos should take subjectivity into account. In this paper, we present a framework for evaluating predictive models using subjective data. The methodology is based on a mixed methods research design, and is applied in an experiment to measure the realism and accuracy of predictions of a visual traffic environment. Our method is shown to be uncorrelated with the predominant approach for evaluating predictive models, which is a frame-wise comparison between predictions and ground truth. These findings emphasise the importance of using subjective data in the assessment of predictive abilities of models and open up a new direction for evaluating predictive deep learning models.

• Teigen, Bjørn Ivar; Ellefsen, Kai Olav; Skeie, Tor & Tørresen, Jim (2021). Known Performance Issues Are Prevalent in Consumer WiFi Routers, 17th International Conference on Network and Service Management, CNSM 2021. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9783903176362. p. 254–258. doi: 10.23919/CNSM52442.2021.9615514. Full text in Research Archive
• Barnes, Chloe M; Ekárt, Anikó; Ellefsen, Kai Olav; Glette, Kyrre; Lewis, Peter R. & Tørresen, Jim (2021). Behavioural Plasticity Can Help Evolving Agents in Dynamic Environments but at the Cost of Volatility. ACM Transactions on Autonomous and Adaptive Systems. ISSN 1556-4665. 15(4). doi: 10.1145/3487918. Full text in Research Archive Show summary

  Neural networks have been widely used in agent learning architectures; however, learnings for one task might nullify learnings for another. Behavioural plasticity enables humans and animals alike to respond to environmental changes without degrading learned knowledge; this can be achieved by regulating behaviour with neuromodulation—a biological process found in the brain. We demonstrate that by modulating activity-propagating signals, neurally trained agents evolving to solve tasks in dynamic environments that are prone to change can expect a significantly higher fitness than non-modulatory agents and also achieve their goals more often. Further, we show that while behavioural plasticity can help agents to achieve goals in these variable environments, this ability to overcome environmental changes with greater success comes at the cost of highly volatile evolution.

• Norstein, Emma Stensby; Ellefsen, Kai Olav & Glette, Kyrre (2021). Co-optimising Robot Morphology and Controller in a Simulated Open-Ended Environment. In Castillo, Pedro A. & Laredo, Juan Luiz Jiménez (Ed.), Applications of Evolutionary Computation. Springer. ISSN 9783030726997. doi: 10.1007/978-3-030-72699-7_3. Full text in Research Archive Show summary

  Designing robots by hand can be costly and time consuming, especially if the robots have to be created with novel materials, or be robust to internal or external changes. In order to create robots automatically, without the need for human intervention, it is necessary to optimise both the behaviour and the body design of the robot. However, when co-optimising the morphology and controller of a locomoting agent the morphology tends to converge prematurely, reaching a local optimum. Approaches such as explicit protection of morphological innovation have been used to reduce this problem, but it might also be possible to increase exploration of morphologies using a more indirect approach. We explore how changing the environment, where the agent locomotes, affects the convergence of morphologies. The agents’ morphologies and controllers are co-optimised, while the environments the agents locomote in are evolved open-endedly with the Paired Open-Ended Trailblazer (POET). We compare the diversity, fitness and robustness of agents evolving in environments generated by POET to agents evolved in handcrafted curricula of environments. Our agents each contain of a population of individuals being evolved with a genetic algorithm. This population is called the agent-population. We show that agent-populations evolving in open-endedly evolving environments exhibit larger morphological diversity than agent-populations evolving in hand crafted curricula of environments. POET proved capable of creating a curriculum of environments which encouraged both diversity and quality in the populations. This suggests that POET may be capable of reducing premature convergence in co-optimisation of morphology and controllers.

• Nordmoen, Jørgen Halvorsen; Veenstra, Frank; Ellefsen, Kai Olav & Glette, Kyrre (2021). MAP-Elites Enables Powerful Stepping Stones and Diversity for Modular Robotics. Frontiers in Robotics and AI. 8. doi: 10.3389/frobt.2021.639173. Full text in Research Archive Show summary

  In modular robotics modules can be reconfigured to change the morphology of the robot, making it able to adapt to specific tasks. However, optimizing both the body and control of such robots is a difficult challenge due to the intricate relationship between fine-tuning control and morphological changes that can invalidate such optimizations. These challenges can trap many optimization algorithms in local optima, halting progress towards better solutions. To solve this challenge we compare three different Evolutionary Algorithms on their capacity to optimize high performing and diverse morphologies and controllers in modular robotics. We compare two objective-based search algorithms, with and without a diversity promoting objective, with a Quality Diversity algorithm—MAP-Elites. The results show that MAP-Elites is capable of evolving the highest performing solutions in addition to generating the largest morphological diversity. Further, MAP-Elites is superior at regaining performance when transferring the population to new and more difficult environments. By analyzing genealogical ancestry we show that MAP-Elites produces more diverse and higher performing stepping stones than the two other objective-based search algorithms. The experiments transitioning the populations to new environments show the utility of morphological diversity, while the analysis of stepping stones show a strong correlation between diversity of ancestry and maximum performance on the locomotion task. Together, these results demonstrate the suitability of MAP-elites for the challenging task of morphology-control search for modular robots, and shed light on the algorithm’s capability of generating stepping stones for reaching high-performing solutions.

• Barnes, Chloe M; Ekárt, Anikó; Ellefsen, Kai Olav; Glette, Kyrre; Lewis, Peter R. & Tørresen, Jim (2021). Evolving Neuromodulated Controllers in Variable Environments, 2021 IEEE International Conference on Autonomic Computing and Self-Organizing Systems. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665412612. p. 164–169. doi: 10.1109/ACSOS52086.2021.00037. Full text in Research Archive Show summary

  Modern technical systems are increasingly composed of heterogeneous components that are situated in variable environments. In nature, organisms can temporarily adapt their behaviour to novel stimuli with behavioural plasticity; this can be achieved with neuromodulation, a biological process that modulates synaptic activity in the brain. We explore how neuromodulation affects goal-achievement in evolved neural controllers for artificial agents in variable environments. As variability can arise from the actions of others, we show that the benefit of plasticity can increase with variability, as agents can temporarily change their phenotype within their lifetime. Further, we show that cooperation can emerge between plastic agents that cannot perceive one another in highly variable environments.

• Barnes, Chloe M; Ekárt, Anikó; Ellefsen, Kai Olav; Glette, Kyrre; Lewis, Peter R. & Tørresen, Jim (2020). Coevolutionary Learning of Neuromodulated Controllers for Multi-Stage and Gamified Tasks, 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781728172781. p. 129–138. doi: 10.1109/ACSOS49614.2020.00034. Full text in Research Archive
• Nordmoen, Jørgen Halvorsen; Veenstra, Frank; Ellefsen, Kai Olav & Glette, Kyrre (2020). Quality and Diversity in Evolutionary Modular Robotics, 2020 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781728125473. p. 2109–2116. doi: 10.1109/SSCI47803.2020.9308511. Full text in Research Archive
• Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 9783030356644. p. 58–68. doi: 10.1007/978-3-030-35664-4_6. Full text in Research Archive
• Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-adapting Goals Allow Transfer of Predictive Models to New Tasks, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 9783030356644. p. 28–39. doi: 10.1007/978-3-030-35664-4_3. Full text in Research Archive
• Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives. Evolutionary Computation. ISSN 1063-6560. 28(1), p. 115–140. doi: 10.1162/evco_a_00250. Full text in Research Archive Show summary

  The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modular neural networks are beneficial. However, apart from objectives aiming to make networks more modular, such structural objectives have not been widely explored. We propose two new structural objectives and test their ability to guide evolving neural networks on two problems which can benefit from decomposition into subtasks. The first structural objective guides evolution to align neural networks with a user-recommended decomposition pattern. Intuitively, this should be a powerful guiding target for problems where human users can easily identify a structure. The second structural objective guides evolution towards a population with a high diversity in decomposition patterns. This results in exploration of many different ways to decompose a problem, allowing evolution to find good decompositions faster. Tests on our target problems reveal that both methods perform well on a problem with a very clear and decomposable structure. However, on a problem where the optimal decomposition is less obvious, the structural diversity objective is found to outcompete other structural objectives—and this technique can even increase performance on problems without any decomposable structure at all.

• Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion. In López-Ibáñez, Manuel (Eds.), GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 9781450361118. p. 133–141. doi: 10.1145/3321707.3321762. Full text in Research Archive Show summary

  Overcoming robotics challenges in the real world requires resilient control systems capable of handling a multitude of environments and unforeseen events. Evolutionary optimization using simulations is a promising way to automatically design such control systems, however, if the disparity between simulation and the real world becomes too large, the optimization process may result in dysfunctional real-world behaviors. In this paper, we address this challenge by considering embodied phase coordination in the evolutionary optimization of a quadruped robot controller based on central pattern generators. With this method, leg phases, and indirectly also inter-leg coordination, are influenced by sensor feedback. By comparing two very similar control systems we gain insight into how the sensory feedback approach affects the evolved parameters of the control system, and how the performances differ in simulation, in transferal to the real world, and to different real-world environments. We show that evolution enables the design of a control system with embodied phase coordination which is more complex than previously seen approaches, and that this system is capable of controlling a real-world multi-jointed quadruped robot. The approach reduces the performance discrepancy between simulation and the real world, and displays robustness towards new environments.

• Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781538681299. p. 189–194. doi: 10.1109/DEVLRN.2019.8850685. Full text in Research Archive

View all works in NVA

• Watanabe, Shin; Horn, Geir; Tørresen, Jim & Ellefsen, Kai Olav (2025). 2025 IEEE 21st International Conference on Automation Science and Engineering (CASE). Institute of Electrical and Electronics Engineers (IEEE). ISBN 9798331522476. doi: https:/ieeexplore.ieee.org/xpl/conhome/11163731/proceeding. Full text in Research Archive
• Ellefsen, Kai Olav & Rohlfing, Katharina J. (2019). Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISBN 9781538681299. 340 p. Full text in Research Archive

View all works in NVA

• Ellefsen, Kai Olav; Bruin, Ege de; Glette, Kyrre & Luck, Kevin (2026). Special Session: Co-design of Robot Morphology and Control. doi: https:/www.mn.uio.no/ifi/english/people/aca/kaiolae/alife-2026-special-session-on-co-design-of-robot-m.html. Full text in Research Archive
• Ellefsen, Kai Olav (2026). Fem grunner til at det ikke finnes virkelig intelligente roboter - Titan.uio.no. Titan.uio.no. Full text in Research Archive Show summary

  I film og litteratur går robotene omkring, utfører alle mulige oppgaver og har interessante personligheter. Hvor blir det av dem?

• Ellefsen, Kai Olav (2026). Towards more intelligent robots, that balance deep reasoning and rapid response. Full text in Research Archive Show summary

  In this talk, I first introduce broadly the research done in the group for Robotics and Intelligent Systems (ROBIN) at the University of Oslo, where we work at the intersection of robotics and artificial intelligence both in applied research and in studies aiming to better understand learning and intelligence in nature. After a broad introduction, I go more in depth on a specific research project we are currently enganged in: Predictive and Intuitive Robot Companion (PIRC). In this project, we take inspiration from human reasoning to develop AI agents that are able to balance deep reasoning with rapid response, allowing them to react quickly when needed, and to reason more carefully for cognitively demanding tasks. Dual Process Theory proposes that humans have two modes of reasoning. A fast, instinctive mode (often called System 1), which is active for instance when we quickly respond to dangers, or when we perform routine tasks; and a slow mode (often called System 2), characterized by deep reasoning and focused attention - which is active when we face new challenges or solve hard problems. Most Deep Learning systems are similar to System 1: They operate quickly, detecting patterns in inputs that are similar to what was present in their training data - a process similar to a human "instinct". In the project PIRC (Predictive and Intuitive Robot Companion) at the University of Oslo, we do research on how to perform System 2-style reasoning in robots and AI agents, and on how to implement meta-reasoning: The process of selecting between the two reasoning modes to ensure rapid, "instinctive" response when possible, and deep reasoning when required. https://www.uio.no/ritmo/english/projects/pirc/

• Ellefsen, Kai Olav (2026). Hvor blir det av de virkelig intelligente robotene? doi: https:/www.mn.uio.no/ifi/om/aktuelt/arrangementer/andre/ki-dagen-2026.html. Full text in Research Archive
• Fatima, Safia; Ellefsen, Kai & Moonen, Leon (2025). Replication Package for "Self Healing of a Mixed Autonomy Traffic System Using Reinforcement Learning and Attention". doi: 10.5281/zenodo.17018040. Full text in Research Archive Show summary

  This repository contains the replication package for the paper "Self Healing of a Mixed Autonomy Traffic System Using Reinforcement Learning and Attention," published in the IEEE Open Journal of Intelligent Transportation Systems (OJ-ITS) with DOI 10.1109/OJITS.2025.3606539.

• Ellefsen, Kai Olav (2025). Seminar: "Fast and Slow Thinking AI". Full text in Research Archive
• Ellefsen, Kai Olav (2025). Neuroevolution - What, Why and How? doi: https:/smash.ung.si/events/seminars/seminar-with-prof-kai-olav-ellefsen-neuroevolution-optimizing-neural-networks-with-evolutionary-algorithms/. Full text in Research Archive
• Veenstra, Frank; Bruin, Ege de; Glette, Kyrre & Ellefsen, Kai Olav (2025). Co-Optimization of Robot Bodies and Brains with Evolutionary Algorithms. Full text in Research Archive Show summary

  Read this story on the University of Oslo's website.

• Wiik, Linn & Ellefsen, Kai Olav (2025). 22. juli-hets, Ekstremvær, Jobbintervju på kafé - Nyhetsmorgen - NRK Radio. [Radio]. NRK. Full text in Research Archive Show summary

  Ungdomspolitikere utsettes for hets og trusler knyttet til Utøya. 14 år siden terrorangrepet i Norge, i dag. Har satt inn krisestab på grunn av ekstremvær i Dannmark. Norges rolle i KI-kappløpet, når Europa henger etter.

• Carnevali, Alice & Ellefsen, Kai Olav (2025). Will robots do our dishes by 2030? |Euronews Tech Talks. [Internet]. EuroNews. Full text in Research Archive Show summary

  Robots are slowly making their way into agriculture, healthcare, and our homes. Should we be worried?

• Nergård, Katrine Linnea; Ellefsen, Kai Olav & Tørresen, Jim (2025). Fast or Slow: Adaptive Decision Maiking in Reinfocement Learning with Pre-Trained LLMs. Full text in Research Archive
• Ellefsen, Kai Olav (2025). Fast and Slow Thinking AI. Full text in Research Archive
• Ellefsen, Kai Olav (2024). Forskere er på sporet av en ny metode for å oppdage når snakkerobotene hallusinerer. Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Ellefsen, Kai Olav (2024). KI-algoritmer lærer mindre effektivt - fordi de aldri har vært babyer. Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Ellefsen, Kai Olav (2024). For KI er to pluss to like tidkrevende som å løse klimakrisen. Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Ellefsen, Kai Olav (2024). Humanoidene kommer! Men må de på død og liv likne oss mennesker? Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Ellefsen, Kai Olav (2024). Det går fort galt når KI-modeller trenes på data de selv har laget. Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Kjørstad, Elise; Børnich, Bernt; Ellefsen, Kai Olav; Goodwin, Morten & Zouganeli, Evi (2024). Roboter som ligner oss: – Det er utrolig spennende det som skjer nå. [Internet]. forskning.no. Full text in Research Archive
• Ellefsen, Kai Olav (2024). Forskningsmaskinene kommer! Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Ellefsen, Kai Olav (2024). Forsøker å forstå hva som foregår på innsiden av kraftige KI-algoritmer. Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Ellefsen, Kai Olav & Bendiksen, Alexander (2024). Bendiksen Airways podkast - Episode #44 - Kai Olav Ellefsen - kunstig intelligens og robotikk. [Radio]. https://podcasts.apple.com/no/podcast/44-kai-olav-ellefsen-k. Full text in Research Archive
• Goodwin, Morten; Ellefsen, Kai Olav; Kjørstad, Elise & Zouganeli, Evi (2024). En robot som assistent? Dette er hindrene, sier forskere. [Internet]. forskning.no. Full text in Research Archive
• Ellefsen, Kai Olav (2024). Hva er Kunstig intelligens? Full text in Research Archive
• Glette, Kyrre; Ellefsen, Kai Olav; Norstein, Emma Stensby & Bruin, Ege de (2024). Automatic Design of Robot Bodies and Brains with Evolutionary Algorithms - Tutorial. Full text in Research Archive Show summary

  The evolution of robot bodies and brains allows researchers to investigate which building blocks are interesting for evolving Artificial Life, and how controllers and morphologies can be shaped together for automated robot design. This tutorial aims to introduce evolution of robot body and control, and some of the key challenges one faces when doing experiments in Evolutionary Robotics. These include finding good ways to represent robots (genotypic encodings), challenges related to co-optimizing morphology and control, how environments shape body and control, and selecting the right physical substrate for evolving robots. After introducing these challenges and showing relevant examples from our own and other labs’ research, we will present a short demo of how to run Evolutionary Robotics experiments in practice, with the Unity ML-Agents framework.

• Kvalsund, Mia-Katrin; Ellefsen, Kai; Glette, Kyrre; Pontes-Filho, Sidney & Lepperød, Mikkel (2023). The distributed neocortex: How neuroscience can inspire distributed AI systems. Full text in Research Archive
• Waarum, Ivar-Kristian; Krogstad, Thomas Røbekk; Ellefsen, Kai Olav & Blomberg, Ann Elisabeth Albright (2023). Belief State Models for Autonomous Marine Monitoring. Full text in Research Archive
• Ellefsen, Kai Olav (2023). Hva er Kunstig Intelligens? Full text in Research Archive
• Ellefsen, Kai Olav (2023). Evolutionary Robotics. Full text in Research Archive
• Ellefsen, Kai Olav (2023). Kunstig intelligens: Verden sett gjennom en maskins øyne. Full text in Research Archive
• Karlsen, Einar; Tørresen, Jim; Ellefsen, Kai Olav; Badescu, Claudia Andreea; Otterdijk, Marieke van & Watanabe, Shin [Show all 9 contributors for this article] (2023). Møte den nye helsearbeideren. [Journal]. Elektronikk. Full text in Research Archive
• Strand, Ørjan; Reilstad, Didrik Spanne; Wu, Zhenying; Silva, Bruno Castro da; Torresen, Jim & Ellefsen, Kai Olav (2023). Learning When to Think Fast and When to Think Slow. Full text in Research Archive
• Ellefsen, Kai Olav (2023). More human robot brains with inspiration from biology, psychology and neuroscience. Full text in Research Archive
• Bogen, Annika Celin & Ellefsen, Kai Olav (2023). Hvorfor mener noen at kunstig intelligens er farlig? [Internet]. ung.forskning.no. Full text in Research Archive
• Ellefsen, Kai Olav (2023). Forskningsmiljøene er splittet i synet på KI som eksistensiell trussel. Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Ellefsen, Kai Olav (2023). Er Minecraft et steg på veien mot roboter som kan tømme oppvaskmaskinen? Morgenbladet. ISSN 0805-3847. Full text in Research Archive
• Ellefsen, Kai Olav; Dautenhahn, Kerstin & Tørresen, Jim (2022). Guest Editorial Special Issue on Development and Learning and on Epigenetic Robotics. IEEE Transactions on Cognitive and Developmental Systems. ISSN 2379-8920. 14(4), p. 1302–1305. doi: 10.1109/TCDS.2022.3210665. Full text in Research Archive
• Ellefsen, Kai Olav (2022). Towards more Human Robot Brains. Full text in Research Archive Show summary

  Despite many large breakthroughs in Artificial Intelligence in the last decade, robots are still struggling to solve tasks that we as humans take for granted, like emptying a dishwasher or learning multiple skills in a sequence. In this talk, Kai Olav Ellefsen will talk about work he and colleagues at the group of Robotics and Intelligent Systems (ROBIN), University of Oslo, have done with the goal of making robots more robust and better learners, by taking inspiration from how humans and animals learn.

• Reilstad, Didrik Spanne; Strand, Ørjan; Wu, Zhenying; Silva, Bruno Castro da; Torresen, Jim & Ellefsen, Kai Olav (2022). RADAR: Reactive and Deliberative Adaptive Reasoning – Learning When to Think Fast and When to Think Slow. Full text in Research Archive
• Ellefsen, Kai Olav & Time, Jon Kåre (2020). Han lærer roboter hvordan verden fungerer. [Journal]. https://morgenbladet.no/aktuelt/2020/02/han-laerer-roboter-h. Full text in Research Archive Show summary

  Kai Olav Ellefsen irriterer seg over de mange overdrivelsene om hva kunstig intelligens kan utrette. Selv prøver han å lære maskinene å forutse konsekvensene av sine handlinger.

• Ellefsen, Kai Olav; Nygaard, Tønnes Frostad & Kjørstad, Elise (2020). Her er den første roboten som er laget av levende celler. [Internet]. forskning.no. Full text in Research Archive
• Ellefsen, Kai Olav (2020). Hva kan intelligente maskiner lære av biologisk liv? Biolog. ISSN 0801-0722. p. 16–19. Full text in Research Archive
• Lehman, Joel; Clune, Jeff; Misevic, Dusan; Adami, Christopher; Altenberg, Lee & Beaulieu, Julie [Show all 52 contributors for this article] (2020). The surprising creativity of digital evolution: A collection of anecdotes from the evolutionary computation and artificial life research communities. Artificial Life. ISSN 1064-5462. 26(2). doi: 10.1162/artl_a_00319. Full text in Research Archive
• Gorton, Patrick Ribu & Ellefsen, Kai Olav (2020). Evaluating Predictive Deep Learning Models. Full text in Research Archive
• Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives. Full text in Research Archive
• Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Intelligent, Adaptive Robots in Real-World Scenarios. Full text in Research Archive
• Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Adaptive Robot Body and Control for Real-World Environments. Full text in Research Archive
• Ellefsen, Kai Olav; Bakken, Marianne & Grzegorz, Kajda (2025). Gaze-Informed Prediction of Plant Interaction Hotspots in Egocentric Videos. Universitetet i Oslo. Full text in Research Archive Show summary

  Understanding where humans interact with objects in egocentric video is essential for action anticipation and human machine collaboration. While gaze has proven beneficial as a supervisory signal in indoor environments, its effectiveness in outdoor agricultural settings remains unexplored. This thesis investigates whether gaze can enhance prediction of plant interaction hotspots in greenhouse environments by adapting CNN-LSTM architectures with attention consistency modules to integrate gaze supervision. Using a newly annotated dataset of 46 egocentric videos with synchronized gaze recordings captured during plant manipulation tasks, I evaluate model performance through classification metrics and spatial attention metrics (KLD, SIM, AUC-Judd). The results reveal that gaze heatmaps exhibit strong alignment with ground-truth interaction regions and differ significantly from center-biased baselines, suggesting gaze contains task-relevant spatial information. However, fundamental dataset limitations - including structural ambiguity of plant components, class imbalance, and insufficient visual separability - prevent model form leveraging this signal effectively during training. Edge detection and semantic segmentation analyses demonstrate that CNNs struggle to distinguish visually similar plant parts, leading to poor classification performance across all model variants. Consequently, this work does not provide conclusive evidence for the benefit of gaze supervision or knowledge transfer from indoor to agricultural environments. Nevertheless, the quantitative and qualitative analyses establish that gaze is informative for interaction prediction, and the identified dataset challenges provide crucial insight for advancing egocentric vision research in agricultural domains.

• Torheim, Kevin Tran & Ellefsen, Kai Olav (2025). Exploring fast and slow thinking in artificial intelligence. Universitetet i Oslo. Full text in Research Archive
• Hasle, Viktor Ringvold & Ellefsen, Kai Olav (2025). Fast and Slow Reasoning with Model-Based Reinforcement Learning. Universitetet i Oslo. Full text in Research Archive Show summary

  After Daniel Kahneman authored the pop-psychology book Thinking, Fast and Slow, the psychological theory of viewing human reasoning as a dual-process system has gained popularity. The dual-process theory states that human reasoning can be divided into two main modes of thinking: System 1, which is fast, intuitive, and automatic, and System 2, which is slow, deliberate, and analytical. This thesis investigates applying a dual-system approach to solving Sokoban puzzles. Sokoban is a logic puzzle game that involves moving blocks onto targets. To model these systems computationally, a learned World Model is developed to predict the consequences of future actions within the Sokoban environment. A World Model is a generative model that learns to model the dynamics of the environment. This model is used in two distinct ways: (1) to train an Actor-Critic policy that acts reflexively within the environment, representing the intuitive behavior of System 1; and (2) as a component of a planning algorithm that embodies the deliberative reasoning of System 2. A higher-level Meta-Agent is trained to dynamically arbitrate between these two modes, selecting how much planning is optimal in any state. The Meta-Agent enables a nuanced allocation of cognitive resources by dynamically adjusting the extent of System 2 processing. This novel approach represents the System 1 and System 2 dichotomy as a spectrum, rather than a binary. Sokoban puzzles are notoriously hard to solve, both computationally as well as for humans. Because of this difficulty, a high degree of exactness is required to solve them reliably. As such, one would expect the most effective reasoning strategy would be to maximize the amount of System 2 thinking. The thesis finds that this is suboptimal, and strategically planning in critical states improves the performance. The high-level Meta-Agent performs better than any pure planning approaches by minimizing the likelihood of the learned World Model generating unwanted errors during planning. This finding suggests that optimal problem-solving in complex environments benefits from a hybrid strategy that balances intuitive and analytical reasoning.

• Kajda, Grzegorz & Ellefsen, Kai Olav (2024). Deep learning methods for robotic self-learning using egocentric vision and gaze tracks. Universitetet i Oslo. Full text in Research Archive
• Aas, Bendik Møller & Ellefsen, Kai Olav (2024). Minding the Gaps: Examining the Relationship Between World Modeling and Generalization in Deep Reinforcement Learning. Universitetet i Oslo. Full text in Research Archive Show summary

  A long-standing challenge within the machine learning paradigm of Reinforcement Learning (RL) is to develop algorithms that are highly capable of successfully transferring their current knowledge to hitherto unseen situations. In the last decade, RL algorithms have superseded human performance in arcade games, strategic board games like chess and Go, as well as visually complex realtime strategy games like StarCraft and Dota. Meanwhile, the robustness and generalizability of these problem-solving algorithms are called into question, as many of these models have proven surprisingly vulnerable to small changes in the environment they are trained on. Here, a wide gap in performance remains to be bridged. Loosely inspired by how humans and other higher-order non-human animals develop predictive models of their environment, the field of Model-based Reinforcement Learning (MBRL) trains machine learning agents that learn and leverage internal representations of how the environment behaves. This thesis explores the hypothesis that learning such predictive models of the environment can facilitate more robust generalization. Explicitly learning to anticipate and forecast the outcomes of one’s actions presumably requires and fosters a richer understanding of the causal dynamics that govern the environment.

• Orten, Kristine & Ellefsen, Kai Olav (2024). Exploring Dual-System Reasoning through Model-Free and Model-Based RL. Universitetet i Oslo. Full text in Research Archive Show summary

  This thesis explores dual-system cognition within the context of reinforcement learning (RL). In his book "Thinking, Fast and Slow", Kahneman popularized the idea of categorizing human reasoning as two different systems: a fast, instinctive system - System 1 (S1), and a more deliberate and effortful system - System 2 (S2). S1 is often employed for simple, routine tasks, while S2 is used to solve more complex problems and to plan ahead. Inspired by this concept, researchers have implemented similar adaptive reasoning systems in machine learning (ML), developing models that possess the capacity to predict future events and dynamically respond to the environment. The essence of human prediction, suggested by Craik in "The Nature of Explanation", stems from our internal model of reality. Similarly to humans, S2-like capabilities in ML (e.g. planning and generalization) necessitate a world model. S2-reasoning is therefore commonly implemented through "model-based" RL - algorithms with access to a model of the environment. Contrastingly, S1-like reasoning is modeled through model-free RL. To govern when to use S1 and S2, a third system (System 0) can be employed. In RL, this higher-order system is often referred to as a meta-agent. The thesis objective was to investigate the dual-system approach within a meta-agent context, examining a novel area through the integration of model-free and model-based RL techniques. The focus was on examining the impact of sub-system properties, determining the conditions under which a meta-agent can outperform its sub-systems, and identifying patterns in how the sub-systems were utilized by the meta-agent. The overarching goal was to compare the performance of the sub-systems to that of the meta-agent to evaluate the benefits of an adaptive reasoning system. The thesis advances the field of adaptive reasoning by examining the advantages of combining a model-free and model-based RL in a meta-agent framework. It reveals patterns in how the meta-agent selects between fast and slow agents, identifying which types of states best suit each type of reasoning strategy. The research also investigates how the quality of sub-systems affects the performance of the meta-agent, with a focus on quality similarity among sub-systems and their overall performance level. Furthermore, the thesis strives to deepen our comprehension of adaptive reasoning systems by implementing dual-system meta-agents (S1 and S2), as well as meta-agents employing homogeneous sub-systems (two different S1-agents), and meta-agents that randomly deploy sub-systems (S1 and S2 without a learned meta-agent). The results of the thesis indicated that the meta-agent struggled to exceed the performance of high-quality sub-systems. This is presumably due to the fact that high-quality agents present a smaller scope for enhancement compared to their lower-quality counterparts. In addition, the findings showed that meta-agents are capable of intelligently leveraging diverse reasoning strategies, realizing a performance increase over the individual sub-systems. These results affirm that adaptive reasoning can indeed confer significant advantages, validating the potential of dual-system approaches.

• Kocan, Danielius & Ellefsen, Kai Olav (2023). Attention-Guided Explainable Reinforcement Learning: Key State Memorization and Experience-Based Prediction. Universitetet i Oslo. Full text in Research Archive Show summary

  As autonomous vehicles and robotic systems progressively permeate our everyday lives, the transparency of the Reinforcement Learning (RL) algorithms guiding their actions is essential. In high-stakes domains like autonomous driving or robotic surgery, understanding, interpreting, and predicting an RL agent's decisions is crucial. Despite their vast potential, the intrinsic complex nature of RL models often results in a lack of transparency, making their decision-making processes challenging to interpret and trust. Although progress has been made in explaining the actions of autonomous agents, notable deficiency persists in the realm of visualization explanation techniques. This thesis attempts to address the transparency issue of these RL agents by introducing an attention-guided explanation technique. This method is designed to offer a visualized prediction of the expected and adverse events an agent seeks to achieve or avoid based on prior experience. This process involves identifying key situations, or 'key states', and their outcomes using the self-attention mechanism. The results demonstrate that attention serves as a vital tool not only in identifying key states used in the construction of the experience database but also in explaining the agent's decision-making. Furthermore, the proposed model allows for a real-time interaction between the user and the selected agent, where the agent can perform actions in the given environment and the user can request an explanation in the form of a prediction visualization at any given time. Although the simulation environment underpinning the results may be insufficient for real-world problems, the method presented in this thesis may serve as a foundation for future research in the field of attention-guided memory collection.

• Taye, Eyosiyas Bisrat & Ellefsen, Kai Olav (2023). Accountability Module: Increasing Trust in Reinforcement Learning Agents. Universitetet i Oslo. Full text in Research Archive Show summary

  Artificial Intelligence requires trust to be fully utilised by users and for them to feel safe while using them. Trust, and indirectly, a sense of safety, has been overlooked in the pursuit of more accurate or better-performing black box models. The field of Explainable Artificial Intelligence and the current recommendations and regulations around Artificial Intelligence require more transparency and accountability from governmental and private institutes. Creating a self-explainable AI that can be used to solve a problem while explaining its reasoning is challenging to develop. Still, it would be unable to explain all the other AIs without self-explainable abilities. It would likely not function for different problem domains and tasks without extensive knowledge about the model. The solution proposed in this thesis is the Accountability Module. It is meant to function as an external explanatory module, which would be able to function with different AI models in different problem domains. The prototype was inspired by accident investigations regarding autonomous vehicles but was created and implemented for a simplified simulation of vehicles driving on a highway. The prototype's goal was to attempt to assist an investigator in understanding why the vehicle crashed. The Accountability Module found the main factors in the decision that resulted in an accident. It was also able to facilitate the answering of whether the outcome was avoidable and if there were inconsistencies with the agent's logic by examining different cases against each other. The prototype managed to provide useful explanations and assist investigators in understanding and troubleshooting agents. The thesis and the Accountability Module indicate that a similar explanatory module is a robust direction to explore further. The chosen explainability methods and techniques were highly connected to the problem domain and limited by the scope of the thesis. Therefore, a more extensive test of the prototype with different problems needs to be performed to check the system's rigidity and versatility as well as the significance of the results. Nevertheless, in a collaboration between an Accountability Module expert and a domain expert, I expect a modular explainability solution to create more insight into an AI model and its significant incidents.

• Høvin, Mats Erling; Ellefsen, Kai Olav & Skjeltorp, Ole Edvin (2022). 3D Neural Cellular Automata - Simulating morphogenesis: Shape, color and behavior of three-dimensional structures. Universitetet i Oslo. Full text in Research Archive Show summary

  Plants, fungi, humans and all other multicellular organisms go through the same process of growing step by step. Starting as a single cell with a genome containing all the genetic information of the organism, they grow into the shape encoded in their genome with stunning accuracy. Not only do they grow into a shape, but a complex composition of cell types. Organisms also know when to stop growing, and some even have abilities to regrow damaged cells. The study of this process, called developmental biology, can provide insight useful for a range of disciplines, such as medicine and artificial intelligence. Computer science has a long history of benefiting from mimicking models of biology, and modern computing power provides tools to simulate biological models in ways that may benefit both fields. Simulations can provide insight and observations that are hard to catch otherwise. This thesis contributes to the tools capable of providing such insights, and aims to simulate morphogenesis by growing a single cell into a three-dimensional colored shape. The framework extends recent work simulating 2D morphogenesis using machine learning combined with an abstract computational system called cellular automatas (CA). In addition to the added dimensionality, we further extend the framework and propose a novel solution allowing guidance of the morphogenesis through certain checkpoints during training. We also experiment with a novel approach of training a simple 3D model to exhibit an oscillating motion, with promising results laying the foundation for future work exceeding past simulation of just morphogenesis. A formula for estimating a hyperparameter, the minimum number of updates a CA needs during training, is derived to provide a basis for future work on 3D neural cellular automatas (3D NCA). The framework is successfully adapted to the higher dimensionality and three-dimensional morphogenesis is simulated with high precision on a range of models covering different geometrical challenges. Both shape and color is correctly grown from a single cell, smaller models are indistinguishable from their targets, while larger models tend to have a few cells misplaced. We observe a significant increase in computational cost with the three-dimensional simulations, indicating that optimisation measures would be critical if using the framework on large scale simulations. In terms of simulating morphogenesis, the framework matches the performance of similar work published while this thesis was written, in this relatively narrow but fast evolving field.

• Ellefsen, Kai Olav & Phan, Tommy (2022). Exploring the Potential of Hierarchical Quality-Diversity Algorithms for Robot Navigation. Universitetet i Oslo. Full text in Research Archive Show summary

  This thesis expands upon the concept of these Hierarchical Quality-Diversity algorithms; specifically on previous research, where these algorithms were used to draw digits from zero to nine. While a promising start, these tasks do not properly assess the potential of these algorithms when tasked with generating generalised robot behaviours. If these algorithms can produce comparatively similar results to traditional algorithms when applied to the most common robotics task, further optimisations would allow them to surpass traditional solutions, either through optimisations to its evolutionary process, or the discovery of benefits that are inherent to hierarchical QD-algorithms.

• Gran, Kristian Roa & Ellefsen, Kai Olav (2022). Learning to drive by predicting the future: Direct Future Prediction. Universitetet i Oslo. Full text in Research Archive Show summary

  The use of artificial intelligence in systems for autonomous vehicles is growing in popularity [1, 2]. Following the rapid development of deep learning techniques over the past years, reinforcement learning has enabled automating the learning of prediction abilities. Controlling an autonomous vehicle with reinforcement learning is typically done by either learning a direct mapping from observations to actions, or by learning a model of the environment and using the model to make decisions. Model-free approaches have previously seen the most success as errors can easily propagate in an inaccurate model. The predictive reinforcement learning algorithm «Direct Future Prediction» (DFP) won the Visual Doom AI competition in 2016 with results 50% better than the second best submission [3]. By learning a simpler model of the environment that only focuses on a few measurable quantities this approach can efficiently solve challenging control tasks. Prior to this thesis, the utility of the method had not yet been tested for relevant real world tasks, such as sensorimotor control of an autonomous vehicle. DFP is tested on a variety of traffic scenarios with the aim of investigating the potential for predictive deep learning algorithms to learn to control an autonomous vehicle. The more classical reinforcement learning algorithm «Deep Q-Networks» (DQN) is also trained and tested on the same scenarios, and is used as a reference for determining the performance of DFP. Experiments are conducted in a more difficult version of the CarRacing simulator from OpenAI gym, where DQN has previously performed well [4, 5]. DFP is able to solve all of the traffic scenarios from the conducted experiments, and is able to drive around a challenging track while avoiding cleverly placed obstacles. The performance of DFP is equal to, or better, in every experiment when compared to DQN. The driving style of the DFP agents are calm and controlled, which is further highlighted by the sporadic driving of the DQN agents. Performance is strong also in previously unseen environments, indicating that the method has good generalization abilities, which is further explained when visualizing the future DFP predictions.

• Bordvik, David Andreas; Ellefsen, Kai Olav & Riemer-Sørensen, Signe (2022). Forecasting regulation market balancing volumes from market data and weather data using Deep Learning and Transfer Learning. Universitetet i Oslo. Full text in Research Archive
• Ølberg, Eirik & Ellefsen, Kai Olav (2022). CHILD: Predicting simulation outcome in open-ended co-evolution. Universitetet i Oslo. Full text in Research Archive Show summary

  In this thesis, we present CHILD, a method to reduce computational costs by predicting simulation outcomes in problem-solution co-evolution and open-ended learning. In recent years, great promise has been shown by open-ended evolutionary algorithms to produce complex and generalized solutions to difficult and unseen problems. This branch of evolutionary algorithms are characterized by their ability to innovate indefinitely, often by finding and solving ever changing new problems. But as with any method of artificial intelligence, they are hindered by the computational resources they demand. CHILD detects difficulties in a task, and predicts the outcome of simulations. By doing this, CHILD attempts to reduce the costs of the algorithm by omitting simulation where the result would be either severe failure, or effortless success. Since open-ended algorithms are designed to innovate indefinitely, they are greatly limited by the computational resources they consume, which is tied to the efficiency of their algorithm. Alleviating this is the primary motivation for the CHILD method.

• Thoresen, Sindre & Ellefsen, Kai Olav (2021). Solving Long Term Planning Problems with Direct Future Prediction. Universitetet i Oslo. Full text in Research Archive Show summary

  Reinforcement learning is a rapidly growing field in machine learning that has the potential to foster algorithms that can solve a wide variety of problems. Two major classes of reinforcement learning are model-free and model-based methods. Model-based methods try to learn predictive models of the environment to achieve better efficiency and reliability. However, learning good models is difficult, as minor inaccuracies compound over time and make long-term planning difficult. Because of this and other practical problems, model-based methods have not seen the same success as model-free methods. Direct Future Prediction (DFP) is a reinforcement learning algorithm that won a VizDoom AI competition in 2016 and received significant attention in the reinforcement learning community. Interestingly, it falls somewhere in between model-based and model-free methods. This is because DFP does not try to learn detailed models that simulate the environment. Instead, it only tries to predict the agent's effect on a few high-level features. The idea is that this can avoid the problem of compounding errors by focusing on predicting just what is important. DFP has so far only been tested on a first-person shooter environment that is reactive and offers few long-term planning challenges. This thesis focuses on testing DFP in other environments that explicitly test long term-planning and exploration. These are challenges found in many reinforcement learning problems, and documenting DFP's ability to solve them could give insight into the role of predictive modeling in reinforcement learning. The thesis tests DFP in two new environments and examines its performance. Results show that DFP can fail to solve certain long-term planning problems, but also that it can be surprisingly effective with the right configuration. The thesis also proposes a new method that combines DFP with a recent technique in intrinsic motivation, a field in reinforcement learning that focuses on effective exploration. The new method uses the variance over an ensemble of DFP models as an uncertainty signal to guide exploration. The method is tested on hard exploration and multi-task problems. Results are promising on some problems, where the new method was able to find several solutions, while DFP found none. However, problems with the ensemble approach are also discussed, which may hinder scaling to more complex environments.

• Ellefsen, Kai Olav & Bjørsvik, Vegard (2021). Solving Sparse Reward Environments Using Go-Explore with Learned Cell Representation. Universitetet i Oslo. Full text in Research Archive Show summary

  Traditional reinforcement learning methods still struggle to solve environments that provide simple and intuitive rewards, such as a distance reward from goal or only a reward when the task is completed, as they give sparse and deceptive feedback. That is, feedback that can easily lead the agent towards locally optimal solutions that do not get it closer to actually solving the problem. Solving such problems requires a thorough exploration of the environment, and a new family of algorithms, Go-Explore, tries to solve that problem. Go-Explore has proven to be able to solve known hard-to-explore environments such as Montezuma's Revenge and Pitfall by following two principles of explicitly remembering promising states and first returning to such states before exploring. While it has proven to solve such environments, it has not been proven to solve a more general range of environments. An integral part of Go-Explore is the state-to-cell mapping. The cells are a collection of similar states, which in the original Go-Explore method needs a manually designed mapping for the cells. Designing such a mapping for every environment manually would mean it is unsure how general the method is. This thesis proposes a new method for the state-to-cell mapping of the Go-Explore algorithm that will work by learning new mappings for every environment. It does so by implementing an autoencoder network to learn the observed states from the environment and creates a grouping of the latent values to map the states to cells. It is tested on an environment with known sparse rewards to show that it can solve a similar category of environments as the original method, but importantly, without any manual design of the state-to-cell mapping. The thesis also tests whether the state-to-cell mapping is essential for Go-Explore to operate effectively. By comparing different state-to-cell mapping methods to a random state-to-cell mapping, it tries to grasp the importance of the mapping. From the test, it is able to show a significant difference between the random mapping and the original mapping used by Go-Explore. This shows that a good mapping is vital for the efficiency of the Go-Explore algorithm.

• Nordmoen, Jørgen; Glette, Kyrre & Ellefsen, Kai Olav (2021). Enhancing MAP-Elites to overcome challenges in Evolutionary Robotics. University of Oslo. Full text in Research Archive
• Gorton, Patrick & Ellefsen, Kai Olav (2020). Backpropagating to the Future: Evaluating Predictive Deep Learning Models. Universitetet i Oslo. Full text in Research Archive
• Sørensen, Scott Andreas Fiskerstrand & Ellefsen, Kai Olav (2020). Comparing Model-Free and Model-Based Reinforcement Learning for Collision Avoidance. Universitetet i Oslo. Full text in Research Archive Show summary

  Autonomous cars are increasingly utilizing artificial intelligence in their systems. The problem of collision avoidance for autonomous cars can be approached using reinforcement learning (RL). In this thesis we look at two approaches to RL; one where an RL agent learns a direct mapping from observations to actions called model-free RL, and one where an RL agent learns to act by using a separate, learned predictive model of the environment called model-based RL. Both model-based and model-free RL for collision avoidance has been researched and shown to be useful and effective solutions to the problem of collision avoidance. However, research comparing these two methods for collision avoidance in a controlled and systematic manner seems to be lacking. In this thesis, the costs and benefits of model-free reinforcement learning versus model-based reinforcement learning for predicting and avoiding collisions in traffic situations are investigated.

View all works in NVA