Predictive and Intuitive Robot Assistant (PIRC)

In these master projects we target a psychology-inspired computing breakthrough through research combining insight from cognitive psychology with computational intelligence to build models that forecast future events and respond dynamically. The systems will be aware and alert for how to best act given their knowledge about themselves and perception of their environment. Humans anticipate future events more effectively than computers. We combine sensing across multiple modalities with learned knowledge to predict outcomes and choose the best actions. Can we transfer these skills to intelligent systems in human-robot interaction scenarios?

Bilde av robot-assistent

In 2021, the ROBIN group and RITMO centre start the new Research Council of Norway funded project – Predictive and Intuitive Robot Companion (PIRC, 2020–2025) ­– to contribute to making robot assistants work safely and effectively at the same time. We will apply our machine learning and robotics expertise, and collaborate with researchers in cognitive psychology, to apply recent models of human prediction to perception-action loops of future intelligent robot companions. The goal of the work will be to allow such robots to adapt and act more seamlessly with their environment than the current technology. We will equip the robots with these new skills and in addition, provide them with the knowledge that users they are interacting with, apply the same mechanisms. This will include mechanisms for adaptive response time from quick and intuitive to slower and well-reasoned. The models will be applied in two robotics applications with potential for very wide impact: physical rehabilitation and home care robot support for older people. You find a presentation of the project here. ROBIN plan to, during spring 2021, to purchase a relevant robot like TIAGo from PAL Robotics or Lio from F&P Robotics that will be applied in the master projects below.

We would like to address the above mentioned challenges through three different master projects:

Predictive and Alert Systems 

Humans are superior to computers and robots when it comes to anticipating future events by applying multimodal sensing together with learned knowledge in choosing the best actions. Are we able to transfer these prediction skills into intelligent systems and also apply that knowledge in how the systems interact with human users?

That is what we would like to address in one or more mater projects; through collaboration with researchers in cognitive psychology who we collaborate with in the RITMO Centre of Excellence, we aim at applying recent understanding within the field to develop models to be applied in perception-action loops of future intelligent systems including smartphones and robots.

The project is concerned with introducing new technology by applying cognitive psychology concepts for shifting between instinctive reactions and slower well-reasoned response using prediction mechanisms. Human capabilities like prediction and instinctive reaction – even in a complex situation, are features that are important for humans as for future robots to be able to offer intelligent and effective human interaction. Developing such capabilities within robotics and smartphones is the overall goal of this project. This includes demonstrating how a system with the capability of being alert would increase user friendliness.

Cognitive control has been explored in cognitive psychology and neuroscience by referring to processes that allow information processing and behaviour to vary adaptively from moment to moment depending on current goals, rather than remaining rigid and inflexible. Cognitive control enables humans to flexibly switch between different thoughts and actions. Models with two modes of thought have been proposed in psychology; one being fast and instinctive and another being slower and with more reasoning involved (Kahneman, 2011). One may think that the latter is to be preferred but humans almost entirely use the instinctive one in their daily life. That is, we do not think much about what to do when we walk or cycle. The benefit is that we do tasks effectively and with little cognitive effort. However, if we fail in doing something, we have to slow down and try again with more reasoning until can manage it. Next time we may be able to do the same task in an intuitive way using that earlier learned knowledge (Li, 2016). 

References

[Kahneman 2011] D. Kahneman, “Thinking, Fast and Slow”, Penguin UK, 2011.

[Li 2016] T. H. S. Li et al., "Robots That Think Fast and Slow: An Example of Throwing the Ball Into the Basket," in IEEE Access, vol. 4, no. , pp. 5052-5064, 2016. 

[Norstein 2020] Thinking fast and slow in intelligent systems, Master thesis. https://www.duo.uio.no/handle/10852/79608 

The tasks of the project:

  1. Get an overview of theory behind the two model system in humans and work undertaken on emulating it in artificial systems.
  2. Define an own experimental setup invoking a robot to work on developing a demonstrator of the thinking and fast and slow concept.
  3. Compare various implementations of the system.
  4. Write master thesis report

Fast and Slow Reasoning with Model-Based Reinforcement Learning

Model-based Reinforcement Learning aims to make AI agents learn more robust and transferrable knowledge by first learning to "understand" the world around them, and later using that understanding to learn how to act (whereas model-free RL agents learn how to act directly while interacting with an environment). This learned model also has the advantage that it can allow the agent to plan ahead, by "imagining" the outcome of its actions (Ha, 2018).

As described above, humans have been proposed to have two main forms of reasoning: One fast and instinctive, and another slower and with more reasoning involved (Kahneman, 2011). If balanced properly, these two forms of reasoning can allow fast response when needed, and more carefully planned responses when necessary (or when there is plenty of time).

This master's project proposes to combine the two concepts above: Can the ability to predict outcomes of actions from model-based RL and the ability to switch between slow and fast reasoning be combined? For instance, can we learn when to use (slow) predicted outcomes from the predictive model, and when to instead choose a (fast) "instinctive" action? The idea could take the following form:

  1. Train a predictive model of an environment, for instance using this approach.
  2. Train two RL models: One model-based that predicts future states and uses them for acting, and a model-free one that acts directly from current environment inputs.
  3. Train an agent to act optimally in the environment by using the two models above at the right moments: Can it learn to evaluate if a situation calls for immediate/instinctive responses, or if there is a need to think ahead to make the right decision?
    • Learning here could for instance take as input the current state (e.g. an image of the situation the agent finds itself in) and produce as output the binary choice between the "fast" and the "slow" model. Can the agent via Reinforcement Learning learn to choose the right model at the right moment?

References

[Kahneman 2011] D. Kahneman, “Thinking, Fast and Slow”, Penguin UK, 2011.

[Ha 2018] Ha, David, and Jürgen Schmidhuber. "Recurrent world models facilitate policy evolution." Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018.

[Norstein 2020] Thinking fast and slow in intelligent systems, Master thesis. https://www.duo.uio.no/handle/10852/79608 

The tasks of the project:

  1. Get an overview of theory behind the two model system in humans and work undertaken on emulating it in artificial systems.
  2. Define your own experimental setup including a problem to test the algorithm on and your own selection of model-free and model-based RL algorithms.
  3. Analyze the results: Did the agent learn to properly select between "fast" and "slow" thinking? Is there any regularity with regards to in which situations the agent chooses each form of reasoning?
  4. Write master thesis report

Robot Sensing and Control for Physical Rehabilitation at Home

The objective of this project is to apply the developed models for prediction and adaptive response for a robot supporting in rehabilitation in a home environment. Thus, there has to be undertaken implementation and research within robot perception and control relevant for rehabilitation training tasks. In addition, user studies through human robot interaction experiments are to be performed. The project will be undertaken in collaboration with Sunnaas Rehabilitation Hospital.

References

A popular science introduction to physical therapy.

Fazekas, Gábor. (2013). Robotics in rehabilitation: Successes and expectations. International journal of rehabilitation research. 36. 10.1097/MRR.0b013e32836195d1. 

Gassert, R., Dietz, V. Rehabilitation robots for the treatment of sensorimotor deficits: a neurophysiological perspective. J NeuroEngineering Rehabil 15, 46 (2018). https://doi.org/10.1186/s12984-018-0383-x https://doi.org/10.1186/s12984-018-0383-x 

A master project undertaken i collaboration with Sunnaas: Haakon Drews: Classification of Error Types in Physiotherapy Exercises (2017)

The tasks of the project:

  1. Get an overview of theory behind predictive models and the two model system in humans and work undertaken on emulating it in artificial systems.
  2. Define an own experimental setup to work on developing a robot demonstrator of the psychology concepts. Relevance to rehabilitation should be taken into account.
  3. Compare various implementations of the system.
  4. Write master thesis report

 

Home Care Robot Support

The objective of this master project is to apply the developed models for prediction and adaptive response for a robot contributing to home care. The research will be focused on two different tasks: preparing food in the kitchen and/or interacting with a human with regards to bringing food etc and returning remains. Thus, there has to be undertaken implementation and research within robot perception and control, as well as user studies, relevant for the home care tasks.

Reinforcement learning will be relevant, including learning a library of standard robot tasks. That is, a library of basic behaviours are trained (e.g. “move arm 10 cm to the left”), and then higher-level behaviours (e.g. “pick up object”) are learned as combinations of more basic behaviours. Specifying each behaviour in such a library manually is time consuming and risks not identifying all relevant sub-behaviours. Automatic methods for building hierarchies of behaviours therefore exist (see Cully & Demiris below). However, these have so far mainly been tested on simpler robot interaction scenario. A valuable contribution of this work would therefore be to suggest and test how to extend such techniques to learning complex and useful behaviours for home care robots.

Bildet kan inneholde: tekst, linje, gul, organisme, grafikk.
Image from "Hierarchical behavioral repertoires with unsupervised descriptors" illustrating that learned motion primitives (to the right) can be combined to form more complex, learned motions.

 

References

Robinson, Hayley & Macdonald, Bruce & Broadbent, Elizabeth. (2014). The Role of Healthcare Robots for Older People at Home: A Review. International Journal of Social Robotics. 6. 575-591. 10.1007/s12369-014-0242-2. 

Van Aerschot, L., Parviainen, J. Robots responding to care needs? A multitasking care robot pursued for 25 years, available products offer simple entertainment and instrumental assistance. Ethics Inf Technol 22, 247–256 (2020). https://doi.org/10.1007/s10676-020-09536-0 

A. Cully and Y. Demiris, “Hierarchical behavioral repertoires with unsupervised descriptors,” in Proceedings of the Genetic and Evolutionary Computation Conference, Kyoto, Japan, Jul. 2018, pp. 69–76, doi: 10.1145/3205455.3205571.

The tasks of the project:

  1. Get an overview of theory behind the two psychology models and state-of-art within service robots applied in human-robot interaction scenarios.
  2. Get an overview of techniques for learning hierarchies of robot skills, enabling the robot to learn complex tasks as sequences of simpler ones.
  3. Define an own experimental setup to work on developing a demonstrator incorporating the psychology concepts where the robot is able to learn selected human supportive tasks, as well as demonstrating the learning of complex tasks as a combination of simpler ones.
  4. Compare various implementations of the system.
  5. Write master thesis report

 

The RITMO - Centre for Interdisciplinary Studies of Rhythm, Time and Motion started in 2018. Researchers in the network come from the ROBIN group (Informatics), FRONT Neurolab (Psychology), and the Department of Musicology, and have access to state-of-the-art laboratories.

 

 

 
Publisert 15. okt. 2020 08:19 - Sist endret 29. juni 2021 11:15

Omfang (studiepoeng)

60