Predictive and Intuitive Robot Assistant (PIRC)

In 2021, the ROBIN group and RITMO centre start the new Research Council of Norway funded project – Predictive and Intuitive Robot Companion (PIRC, 2020–2025) ­– to contribute to making robot assistants work safely and effectively at the same time. We will apply our machine learning and robotics expertise, and collaborate with researchers in cognitive psychology, to apply recent models of human prediction to perception-action loops of future intelligent robot companions. The goal of the work will be to allow such robots to adapt and act more seamlessly with their environment than the current technology. We will equip the robots with these new skills and in addition, provide them with the knowledge that users they are interacting with, apply the same mechanisms. This will include mechanisms for adaptive response time from quick and intuitive to slower and well-reasoned. The models will be applied in two robotics applications with potential for very wide impact: physical rehabilitation and home care robot support for older people. You find a presentation of the project here. ROBIN has the assistant robot TIAGo from PAL Robotics that will be applied in the master projects below.

We would like to address the above mentioned challenges through these master projects:

Predictive and Alert Systems 

Humans are superior to computers and robots when it comes to anticipating future events by applying multimodal sensing together with learned knowledge in choosing the best actions. Are we able to transfer these prediction skills into intelligent systems and also apply that knowledge in how the systems interact with human users?

That is what we would like to address in one or more mater projects; through collaboration with researchers in cognitive psychology who we collaborate with in the RITMO Centre of Excellence, we aim at applying recent understanding within the field to develop models to be applied in perception-action loops of future intelligent systems including smartphones and robots. 

The project is concerned with introducing new technology by applying cognitive psychology concepts for shifting between instinctive reactions and slower well-reasoned response using prediction mechanisms. Human capabilities like prediction and instinctive reaction – even in a complex situation, are features that are important for humans as for future robots to be able to offer intelligent and effective human interaction. Developing such capabilities within robotics and smartphones is the overall goal of this project. This includes demonstrating how a system with the capability of being alert would increase user friendliness.

Cognitive control has been explored in cognitive psychology and neuroscience by referring to processes that allow information processing and behaviour to vary adaptively from moment to moment depending on current goals, rather than remaining rigid and inflexible. Cognitive control enables humans to flexibly switch between different thoughts and actions. Models with two modes of thought have been proposed in psychology; one being fast and instinctive and another being slower and with more reasoning involved (Kahneman, 2011). One may think that the latter is to be preferred but humans almost entirely use the instinctive one in their daily life. That is, we do not think much about what to do when we walk or cycle. The benefit is that we do tasks effectively and with little cognitive effort. However, if we fail in doing something, we have to slow down and try again with more reasoning until can manage it. Next time we may be able to do the same task in an intuitive way using that earlier learned knowledge (Li, 2016). Two earlier ROBIN master students have done projects on developing systems using the two modes, see [Norstein 2020] and [Tønnessen 2021].

See alternative variants of this project under Fast and Slow Thinking and Acting.

References

[Kahneman 2011] D. Kahneman, “Thinking, Fast and Slow”, Penguin UK, 2011.

[Li 2016] T. H. S. Li et al., "Robots That Think Fast and Slow: An Example of Throwing the Ball Into the Basket," in IEEE Access, vol. 4, no. , pp. 5052-5064, 2016. 

[Norstein 2020] Thinking fast and slow in intelligent systems, ROBIN Master thesis. https://hdl.handle.net/10852/79608 

[Tønnessen 2021] Thinking Fast and Slow – PINE: Planning and Identifying Neural Network, ROBIN Master thesis https://www.duo.uio.no/handle/10852/90312 

The tasks of the project

1. Get an overview of theory behind the two model system in humans and work undertaken on emulating it in artificial systems.
2. Define an own experimental setup invoking a robot to work on developing a demonstrator of the thinking and fast and slow concept.
3. Compare various implementations of the system.
4. Write master thesis report

Model-based Reinforcement Learning for Tiago

Much of the progress in recent years has been from the area of model-free Reinforcement Learning, that is, the idea of trying to learn to solve a problem without forming an explicit internal model of the environment. Model-based RL attempts to solve two key challenges with model-free approaches: 1) They require enormous amounts of training data, and 2) there is no straightforward way to transfer a learned policy to a new task in the same environment. To do this, model-based RL takes the approach of first learning a predictive model of the environment, before using this model to make a plan that solves the problem. Despite these advantages of model-based algorithms, model-free RL has so far been most successful for complex environments. A key reason for this is that model-based RL is likely to produce very bad policies if the learned predictive model is imperfect, which it will be for most complex environments.

A way to alleviate this problem, making predictive models more feasible to learn, and thus more useful, is to keep the predictive model as small as possible, trying to learn only exactly what is necessary to solve a problem. For instance, if you are trying to model a car-driving scenario, it is not important to perfectly model the flight of birds you see in the distance or the exact colour of the sky. To drive safely and efficiently, we need to model a few key effects, such as the effect of our actions (speed/brake/turn) on the distance to pedestrians, on the speed of the car, and so on. This insight is the idea behind a recent technique called "Direct Future Prediction (DFP)", which solves RL problems by learning to predict how possible actions affect the most important metrics.

In this project, we aim to apply model-based Reinforcement Learning to training the assistive robot TIAGo. In doing so, it is important to explore how to best model TIAGo and his environment: If we model it in too much detail, learning will be too complex. Too little detail, and the learned environment model is of limited use. Finding an intermediate spot, such as the representation used in "Direct Future Prediction (DFP)" can help us keep learning efficient while still providing a useful environment model.

The tasks of the project

1. Read up on model-based RL, in particular methods applied to robots, and methods that try to balance the needs for a complete model with having a low model complexity.
2. Identify a task that TIAGo can carry out where a model of his environment can be useful
3. Set up an experiment, testing the benefit of TIAGo having a model of his environment when learning how to do his task, and perhaps also comparing models at different detail level/ complexity level.
4. Write master thesis report.

Robot Sensing and Control for Physical Rehabilitation at Home

The objective of this project is to apply the developed models for prediction and adaptive response for a robot supporting in rehabilitation in a home environment. Thus, there has to be undertaken implementation and research within robot perception and control relevant for rehabilitation training tasks. In addition, user studies through human robot interaction experiments are to be performed. The project will be undertaken in collaboration with Sunnaas Rehabilitation Hospital.

References

A popular science introduction to physical therapy.

Fazekas, Gábor. (2013). Robotics in rehabilitation: Successes and expectations. International journal of rehabilitation research. 36. 10.1097/MRR.0b013e32836195d1. 

Gassert, R., Dietz, V. Rehabilitation robots for the treatment of sensorimotor deficits: a neurophysiological perspective. J NeuroEngineering Rehabil 15, 46 (2018). https://doi.org/10.1186/s12984-018-0383-x https://doi.org/10.1186/s12984-018-0383-x 

A master project undertaken i collaboration with Sunnaas: Haakon Drews: Classification of Error Types in Physiotherapy Exercises (2017)

The tasks of the project

1. Get an overview of theory behind predictive models and the two model system in humans and work undertaken on emulating it in artificial systems.
2. Define an own experimental setup to work on developing a robot demonstrator of the psychology concepts. Relevance to rehabilitation should be taken into account.
3. Compare various implementations of the system.
4. Write master thesis report

Home Care Robot Support

The objective of this master project is to apply the developed models for prediction and adaptive response for a robot contributing to home care. The research will be focused on two different tasks: preparing food in the kitchen and/or interacting with a human with regards to bringing food etc and returning remains. Thus, there has to be undertaken implementation and research within robot perception and control, as well as user studies, relevant for the home care tasks.

Reinforcement learning will be relevant, including learning a library of standard robot tasks. That is, a library of basic behaviours are trained (e.g. “move arm 10 cm to the left”), and then higher-level behaviours (e.g. “pick up object”) are learned as combinations of more basic behaviours. Specifying each behaviour in such a library manually is time consuming and risks not identifying all relevant sub-behaviours. Automatic methods for building hierarchies of behaviours therefore exist (see Cully & Demiris below). However, these have so far mainly been tested on simpler robot interaction scenario. A valuable contribution of this work would therefore be to suggest and test how to extend such techniques to learning complex and useful behaviours for home care robots.

References

Robinson, Hayley & Macdonald, Bruce & Broadbent, Elizabeth. (2014). The Role of Healthcare Robots for Older People at Home: A Review. International Journal of Social Robotics. 6. 575-591. 10.1007/s12369-014-0242-2. 

Van Aerschot, L., Parviainen, J. Robots responding to care needs? A multitasking care robot pursued for 25 years, available products offer simple entertainment and instrumental assistance. Ethics Inf Technol 22, 247–256 (2020). 

A. Cully and Y. Demiris, “Hierarchical behavioral repertoires with unsupervised descriptors,” in Proceedings of the Genetic and Evolutionary Computation Conference, Kyoto, Japan, Jul. 2018, pp. 69–76, doi: 10.1145/3205455.3205571.

The tasks of the project

1. Get an overview of theory behind the two psychology models and state-of-art within service robots applied in human-robot interaction scenarios.
2. Get an overview of techniques for learning hierarchies of robot skills, enabling the robot to learn complex tasks as sequences of simpler ones.
3. Define an own experimental setup to work on developing a demonstrator incorporating the psychology concepts where the robot is able to learn selected human supportive tasks, as well as demonstrating the learning of complex tasks as a combination of simpler ones.
4. Compare various implementations of the system.
5. Write master thesis report

The RITMO - Centre for Interdisciplinary Studies of Rhythm, Time and Motion started in 2018. Researchers in the network come from the ROBIN group (Informatics), FRONT Neurolab (Psychology), and the Department of Musicology, and have access to state-of-the-art laboratories.