Homa Tarigopula

• Tarigopula, Homa; Malthe-Sørenssen, Anders & Lepperød, Mikkel Elle (2023). Model-based meta-reinforcement learning for alchemy. Full text in Research Archive Show summary

  Meta-learning emphasises the ‘learning to learn’ paradigm where the similar characteristics between the tasks generate a common basis bearing relevance that extends beyond the trained tasks and acts as an inductive bias that helps the learning agent to quickly generalize to unseen tasks from the same distribution. For example during navigation, understanding the underlying structure of the environment can aid inferences when going back to an explored state from an unexplored point of view. Recent work by Whittington et al 2020 has shown that the underlying graph defining the structure once learnt is reusable across different environments consisting of different sensory observations, thus forming an invariant feature basis. This can be extended to the relational inference scenarios encountered by reinforcement learning (RL) agents. A recently introduced benchmark ‘Alchemy' aims at challenging meta-RL by providing both a convenient interface and an optimal solution for assessing agent efficiency. The meta-RL agent navigating the alchemy environment is challenged with understanding the latent causal structure to excel at this task with maximum reward. We aim to train model-based meta-RL agents to solve alchemy, by introducing useful inductive bias for capturing the underlying structure of explored environments. We hypothesize that training such an agent, in addition to improving the meta-RL knowledge base, may help shed light on the interactions between Prefrontal cortex and Entorhinal cortex. Here, we present preliminary results from initial studies that show promising performance on the task compared to the two deep RL agents (VMPO, IMPALA) previously deployed to solve Alchemy.

• Tarigopula, Homa; Malthe-Sørenssen, Anders & Lepperød, Mikkel Elle (2022). Model based meta reinforcement learning for alchemy. doi: https:/fensforum.org/wp-content/uploads/2023/01/FENS2022.pdf. Full text in Research Archive Show summary

  Meta-learning emphasises the ‘learning to learn’ paradigm where the similar characteristics between the tasks generate a common basis bearing relevance that extends beyond the trained tasks and acts as an inductive bias that helps the learning agent to quickly generalize to unseen tasks from the same distribution. For example during navigation, understanding the underlying structure of the environment can aid inferences when going back to an explored state from an unexplored point of view. Recent work by Whittington et al 2020 has shown that the underlying graph defining the structure once learnt is reusable across different environments consisting of different sensory observations, thus forming an invariant feature basis. This can be extended to the relational inference scenarios encountered by reinforcement learning (RL) agents. A recently introduced benchmark ‘Alchemy' aims at challenging meta-RL by providing both a convenient interface and an optimal solution for assessing agent efficiency. The meta-RL agent navigating the alchemy environment is challenged with understanding the latent causal structure to excel at this task with maximum reward. We aim to train model-based meta-RL agents to solve alchemy, by introducing useful inductive bias for capturing the underlying structure of explored environments. We hypothesize that training such an agent, in addition to improving the meta-RL knowledge base, may help shed light on the interactions between Prefrontal cortex and Entorhinal cortex. Here, we present preliminary results from initial studies that show promising performance on the task compared to the two deep RL agents (VMPO, IMPALA) previously deployed to solve Alchemy.

View all works in NVA