Multimodal Elderly Care Systems (MECS)

• Woll, Anita & Tørresen, Jim (2023). What is a Smart Hospital? A Review of the Literature. In Duffy, V.G.; Lehto, M.; Yih, Y. & Proctor, R.W. (Ed.), Human-Automation Interaction. Springer. ISSN 9783031107801. p. 145–165. Full text in Research Archive
• Tørresen, Jim; Saplacan, Diana; Baselizadeh, Adel & Mahler, Tobias (2023). Machine Excellence Tradeoffs to Ethical and Legal Perspectives, Proceedings of the 2023 IEEE Conference on Artificial Intelligence (IEEE CAI). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350339840. p. 237–240. doi: 10.1109/CAI54212.2023.00109. Full text in Research Archive Show summary

  Abstract—We appreciate well-functioning technology being able to also personalize its services. However, to protect privacy and avoid a potential misuse of personal data, we are encouraged to limit the amount of personal data we share through apps and Internet services. While some services do not really need all the data they ask us to provide, others depend on it to provide the best possible performance of its service. That regards systems that apply data in machine learning for tasks like medical diagnostics. Especially deep learning algorithms perform better by using a large amount of data and are now able to benefit from the large amount as well with limited training time given access to high-performance computing resources. This paper address and discuss the tradeoffs like the one we have between data sharing minimalization for increased privacy and data maximization for machine learning systems. Perspectives related to ethics, legal, and social issues are considered in the paper. There is no single conclusion on the challenge, but attention to it can increase the awareness that the best balance differs depending on the application addressed.

• Baselizadeh, Adel; Khaksar, Weria; Uddin, Md Zia; Saplacan, Diana & Tørresen, Jim (2023). Privacy-Preserving User Pose Prediction for Safe and Efficient Human-Robot Interaction, Proceedings of the 2023 19th IEEE International Conference on Automation Science and Engineering (CASE). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350320701. doi: 10.1109/CASE56687.2023.10260298. Full text in Research Archive Show summary

  Enhancing user privacy is crucial in improving the safety and efficiency of Human-Robot Interaction (HRI), as it is a key factor for establishing user trust in the robot. Using privacy-preserving sensors and local processing of the user's data are ways to enhance privacy in HRI. This paper presents a privacy-preserving sensing system for real-time tracking and predicting the user's movements in HRI. As privacy-preserving sensors, a thermal and a depth camera are used to monitor the user's movements and determine their current pose. In order to improve the robot's perception of the user's situation and enhance the quality of real-time user monitoring, a Deep Learning (DL) model has been developed to estimate the future poses of the user. The developed model is based on the Sequence to Sequence mechanism (Seq2Seq). Modifications have been made to Seq2Seq so it can be run locally on the robot. As a result of these modifications, the computational cost of the model has been reduced by 34%. Experimental studies have been conducted to evaluate the performance of the sensing system in tracking and predicting the user's movements in HRI. According to the test results, the proposed sensing system is able to track the movements of the user appropriately. Additionally, it is shown that the estimation of the user's movement through the proposed system with the prediction model can improve the safety and efficiency aspects of the HRI experiments by up to 24% and 17%, respectively.

• Otterdijk, Marieke van; Saplacan, Diana; Baselizadeh, Adel; Laeng, Bruno & Tørresen, Jim (2023). To Shake or Not to Shake: Intuitive Reactions of Senior Adults to a Robot Handshake in a Western Culture, Proceedings of the 32nd IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) 2023. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350336719. p. 890–896. doi: 10.1109/RO-MAN57019.2023.10309608. Full text in Research Archive
• Saplacan, Diana; Schulz, Trenton; Tørresen, Jim & Pajalic, Zada (2023). Health Professionals’ Views on the Use of Social Robots with Vulnerable Users: A Scenario-Based Qualitative Study Using Story Dialogue Method, Proceedings of the 32nd IEEE International Conference on Robot & Human Interactive Communication (RO-MAN) 2023. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350336719. p. 421–428. doi: 10.1109/RO-MAN57019.2023.10309644. Full text in Research Archive Show summary

  We used the story dialog method (SDM) to gather the viewpoints of health professionals about the use of social ro-bots in the home and healthcare services with vulnerable users. SDM consists of participants bringing stories that are discussed together. The aim of the study was to address universal design and accessibility issues with robots in specific use situations. Three social robots were used for this purpose in four stories: TIAGo, Romibo and robot companions. SDM method was used in two workshops with eight participants (33 invited, 12 re-cruited, and 8 participated in the end). The participants uncov-ered issues regarding ethics, responsibility, use of data, infra-structure, design, and user concerns based on provided stories and own experiences. These issues provide important aspects that should be considered when using robots with vulnerable us-ers and ensuring that a robot is usable by as many people as pos-sible. The main contribution of the paper is introducing the SDM method to Human-Robot Interaction community, as well as find-ings around universal design and accessibility of robots, in order to create more inclusive social robots.

• Saplacan, Diana & Tørresen, Jim (2022). Robots as Welfare Technologies to Reduce Falls Amongst Older Adults: An Explorative Study from Norway, Human Aspects of IT for the Aged Population. Technology in Everyday Living. Lecture Notes in Computer Science. LNCS 13331. Springer Nature Switzerland AG. ISSN 9783031056543. p. 88–106. doi: 10.1007/978-3-031-05654-3_6. Full text in Research Archive
• Soma, Rebekka; Bratteteig, Tone; Saplacan, Diana; Schimmer, Robyn; Camparo, Erik & Verne, Guri Birgitte (2022). Strengthening Human Autonomy. In the Era of Autonomous Technology. Scandinavian Journal of Information Systems (SJIS). ISSN 0905-0167. 34(2). Full text in Research Archive
• Bordvik, David Andreas; Hou, Jie; Noori, Farzan Majeed; Uddin, Md Zia & Tørresen, Jim (2022). Monitoring In-Home Emergency Situation and Preserve Privacy using Multi-modal Sensing and Deep Learning, 2022 International Conference on Electronics, Information, and Communication (ICEIC 2022). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665409353. doi: 10.1109/ICEIC54506.2022.9748829. Full text in Research Archive Show summary

  Videos and images are commonly used in home monitoring systems. However, detecting emergencies in-home while preserving privacy is a challenging task concerning Human Activity Recognition (HAR). In recent years, HAR combined with deep learning has drawn much attention from the general public. Besides that, relying entirely on a single sensor modal-ity is not promising. In this paper, depth images and radar presence data were used to investigate if such sensor data can tackle the challenge of a system's ability to detect abnormal and normal situations while preserving privacy. The recurrence plots and wavelet transformations were used to make a two-dimensional representation of the presence radar data. Moreover, we fused data from both sensors using data-level, feature-level, and decision-level fusions. The decision-level fusion showed its superiority over the other two techniques. For the decision-level fusion, a combination of the depth images and presence data recurrence plots trained first on convolutional neural networks (CNN). The output was fed into support vector machines, which yielded the best accuracy of 99.98%.

• Mantelli, Mathias; Pittol, Diego; Maffei, Renan; Tørresen, Jim; Prestes, Edson & Kolberg, Mariana (2021). Semantic Active Visual Search System Based on Text Information for Large and Unknown Environments. Journal of Intelligent & Robotic Systems. ISSN 0921-0296. 101(2). doi: 10.1007/s10846-020-01298-7. Full text in Research Archive
• Noori, Farzan Majeed; Uddin, Md Zia & Tørresen, Jim (2021). Ultra-Wideband Radar-Based Activity Recognition Using Deep Learning. IEEE Access. 9, p. 138132–138143. doi: 10.1109/ACCESS.2021.3117667. Full text in Research Archive Show summary

  With recent advances in the field of sensing, it has become possible to build better assistive technologies. This enables the strengthening of eldercare with regard to daily routines and the provision of personalised care to users. For instance, it is possible to detect a person’s behaviour based on wearable or ambient sensors; however, it is difficult for users to wear devices 24/7, as they would have to be recharged regularly because of their energy consumption. Similarly, although cameras have been widely used as ambient sensors, they carry the risk of breaching users’ privacy. This paper presents a novel sensing approach based on deep learning for human activity recognition using a non-wearable ultra-wideband (UWB) radar sensor. UWB sensors protect privacy better than RGB cameras because they do not collect visual data. In this study, UWB sensors were mounted on a mobile robot to monitor and observe subjects from a specific distance (namely, 1.5–2.0 m). Initially, data were collected in a lab environment for five different human activities. Subsequently, the data were used to train a model using the state-of-the-art deep learning approach, namely long short-term memory (LSTM). Conventional training approaches were also tested to validate the superiority of LSTM. As a UWB sensor collects many data points in a single frame, enhanced discriminant analysis was used to reduce the dimensions of the features through application of principal component analysis to the raw dataset, followed by linear discriminant analysis. The enhanced discriminant features were fed into the LSTMs. Finally, the trained model was tested using new inputs. The proposed LSTM-based activity recognition approach performed better than conventional approaches, with an accuracy of 99.6%. We applied 5-fold cross-validation to test our approach. We also validated our approach on publically available dataset. The proposed method can be applied in many prominent fields, including human–robot interaction for various practical applications, such as mobile robots for eldercare.

• Tørresen, Jim (2021). Undertaking Research with Humans within Artificial Intelligence and Robotics: Multimodal Elderly Care Systems. Technology Architecture + Design (TAD). ISSN 2475-1448. 5(2), p. 141–145. doi: 10.1080/24751448.2021.1967052. Full text in Research Archive
• Saplacan, Diana; Khaksar, Weria & Tørresen, Jim (2021). On Ethical Challenges Raised by Care Robots: A Review of the Existing Regulatory-, Theoretical-, and Research Gaps, Proceedings of the 2021 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO) Virtual Conference, July 8-10, 2021. IEEE Robotics and Automation Society. ISSN 9781665449526. p. 219–226. doi: 10.1109/ARSO51874.2021.9542844. Full text in Research Archive Show summary

  This paper presents some of the ethical challenges that current care robots raise in home- and healthcare services for senior adults (≥65 years). The paper is grounded in some of the state-of-the-art projects within the area of care robotics.Further, the paper identifies and discusses several central challenges raised by using robots as part of care services for the elderly people. The paper contributes to the ethical debate on the implications care robots may have for the practical context of healthcare. In addition, the paper summarizes the main lines of the EU legal approach to AI robotic technology, offering a comprehensive picture of the existing regulatory, theoretical and research gaps, compelling the need of an interdisciplinary ethical reflection on care robots. Finally, the discussion is then balanced by some of the opportunities the care robots may provide for the care services.

• Khaksar, Weria; Neggers, Margot; Barakova, Emilia Ivanova & Tørresen, Jim (2021). Generation Differences in Perception of the Elderly Care Robot. In Caleb-Solly, Praminda; Ghafurian, Moojan; Johnson, Michelle & Wada, Kazuyoshi (Ed.), Proceedings of the 30th IEEE International Conference on Robot & Human Interactive Communication (ROMAN). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781665404921. p. 551–558. doi: 10.1109/RO-MAN50785.2021.9515534. Full text in Research Archive
• Otterdijk, Marieke van; Neggers, Margot; Tørresen, Jim & Barakova, Emilia Ivanova (2021). Preferences of Seniors for Robots Delivering a Message With Congruent Approaching Behavior, Proceedings of the 2021 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO) Virtual Conference, July 8-10, 2021. IEEE Robotics and Automation Society. ISSN 9781665449526. doi: 10.1109/ARSO51874.2021.9542833. Full text in Research Archive Show summary

  Congruence of verbal and nonverbal behavior of a robot is essential to establish a seamless and natural interaction between humans and robots. We have investigated the preference and user experience of the senior participants that a robot approaches with expressive movement that is tailored or incongruent to the message the robot brings. Twelve elderly experienced three scenarios with different messages varying from good news, bad news, and functional news, accompanied by the corresponding approaching behavior and body posture. The user experience of these elderly was evaluated during these scenarios using a questionnaire and a semi-structured interview. The participants were interviewed about their experience, preference, and motivation for the different robot motion behaviors. The analysis showed no significant difference in user experiences in the good news and functional news scenarios. Also, no clear preference among the different robot behaviors was found. However, a significant preference for congruence in the sad news scenario was found with a clear preference for the sad approaching behavior.

• Herstad, Jo; Schulz, Trenton Wade & Saplacan, Diana (2021). T-able: An Investigation of Habituating Moving Tables at Home. In Verma, Ira (Eds.), Universal Design 2021: From Special to Mainstream Solutions. IOS Press. ISSN 9781643681900. p. 238–251. doi: 10.3233/SHTI210400. Full text in Research Archive Show summary

  This paper investigates the philosophical and designerly questions of how the concepts of familiarity, orientation, and habituation can be used to understand a selfmoving (semi-autonomous robot) table at home. Tables are familiar habituated objects in domestic settings for people with various abilities. We explore the idea of a selfmoving table through the lens of universal design. Phenomenology is applied to get a grip on ways of orienting and being oriented by such habituated familiar objects. Specifically, we investigate how the t-able is used as a telephone table, where the telephone is always charged and in a fixed place on the table. This is an attempt to make the telephone easier to use and relate to at home. The paper aims to inform future robots' design for the independently living elderly by designing robots mainly from natural materials, such as wood. We also discuss similarities and differences between the universal design of the built environment and ICT environments with this paper.

• Schulz, Trenton Wade; Soma, Rebekka & Holthaus, Patrick (2021). Movement acts in breakdown situations: How a robot's recovery procedure affects participants' opinions. Paladyn - Journal of Behavioral Robotics. ISSN 2080-9778. 12(1), p. 336–355. doi: 10.1515/pjbr-2021-0027. Full text in Research Archive Show summary

  Recovery procedures are targeted at correcting issues encountered by robots. What are people’s opinions of a robot during these recovery procedures? During an experiment that examined how a mobile robot moved, the robot would unexpectedly pause or rotate itself to recover from a navigation problem. The serendipity of the recovery procedure and people’s understanding of it became a case study to examine how future study designs could consider breakdowns better and look at suggestions for better robot behaviors in such situations. We present the original experiment with the recovery procedure. We then examine the responses from the participants in this experiment qualitatively to see how they interpreted the breakdown situation when it occurred. Responses could be grouped into themes of sentience, competence, and the robot’s forms. The themes indicate that the robot’s movement communicated different information to different participants. This leads us to introduce the concept of movement acts to help examine the explicit and implicit parts of communication in movement. Given that we developed the concept looking at an unexpected breakdown, we suggest that researchers should plan for the possibility of breakdowns in experiments and examine and report people’s experience around a robot breakdown to further explore unintended robot communication.

• Uddin, Md Zia; Noori, Farzan Majeed & Tørresen, Jim (2020). In-Home Emergency Detection Using an Ambient Ultra-Wideband Radar Sensor and Deep Learning. In Noori, Farzan Majeed (Eds.), IEEE 10th International Conference on Ultrawideband and Ultrashort Impulse Signals (UWBUSIS). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781728173146. p. 1089–1093. doi: 10.1109/UkrMW49653.2020.9252708. Full text in Research Archive Show summary

  Human behavior analysis in indoor environments from ambient sensor data is one of the most challenging human-machine interaction research topics. In this paper, we present a deep learning-based novel approach for human posture recognition and emergency detection using an ultra-wideband radar sensor. The strength of this sensor is that it collects less privacy-related information than a regular camera. The sensor is installed on a mobile robot to observe a subject from a short distance. At first, raw data is collected from the sensor. Then, the data is used to train a Recurrent Neural Network (RNN) for modeling of different human activities and conditions, including normal and emergency conditions. Finally, the trained model is used for testing the model on new inputs. The experiments were performed using two datasets recorded in lab environments, and the proposed approach produced better recognition performance than the conventional ones. The proposed method can be applied in many prominent research fields (e.g., human-robot interaction) for different practical applications such as mobile robots for eldercare.

• Saplacan, Diana; Herstad, Jo & Pajalic, Zada (2020). An Analysis of Independent Living Elderly’s Views on Robots: A Descriptive Study from the Norwegian Context. In Mauri, Lloret, James; Saplacan, Diana; Carcani, Klaudia; Ardiansyah, Prima Oky Dicky & Vasilache, Simona (Ed.), ACHI 2020, The Thirteenth International Conference on Advances in Computer-Human Interactions. International Academy, Research and Industry Association (IARIA). ISSN 9781612087610. p. 199–208. Full text in Research Archive Show summary

  This study illustrates the independent living elderly’s (≥65 years) views on robots. The data was documented through audio recordings of interviews, photos, and written logs. The analysis was done through qualitative manifest and latent content analysis. The results of the analysis were sorted into three categories: aging during the technological renaissance, domestic robots, and the elderly’s expectations of robots. The overall resulted theme was: integrating robots in the elderly’s everyday life. The results were discussed through the lenses of the Sense-of-Coherence (SOC) theoretical construct and its belonging elements: comprehensibility, manageability, and meaningfulness. The relevance of this paper contributes to giving an understanding of the domestic robots’ requirements specifications and the elderly’s expectation of human-robot interaction.

• Saplacan, Diana; Herstad, Jo & Schulz, Trenton Wade (2020). Situated Abilities within Universal Design – A Theoretical Exploration. International Journal on Advances in Intelligent Systems. 13(34), p. 278–291. Full text in Research Archive Show summary

  This paper investigates Universal Design (UD) through the idea of designing for situated abilities, rather than focusing on designing for disabled users. This shift in perspective from disabilities to abilities is explored through the design of a domestic robot that integrates into our homes, in a familiar way. We explore the concept of designing for situated abilities through a proof-of-concept robotic wooden table, the T-ABLE, as an alternative design for domestic robots. Finally, the paper identifies four dimensions of situated abilities.

• Noori, Farzan; Riegler, Michael; Uddin, Md Zia & Tørresen, Jim (2020). Human Activity Recognition from Multiple Sensors Data Using Multi-Fusion Representations and CNNs. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP). ISSN 1551-6857. 16(2). doi: 10.1145/3377882. Full text in Research Archive Show summary

  With the emerging interest in the ubiquitous sensing field, it has become possible to build assistive technologies for persons during their daily life activities to provide personalized feedback and services. For instance, it is possible to detect an individual’s behavioral pattern (e.g., physical activity, location, and mood) by using sensors embedded in smart-watches and smartphones. The multi-sensor environments also come with some challenges, such as how to fuse and combine different sources of data. In this article, we explore several methods of fusion for multi-representations of data from sensors. Furthermore, multiple representations of sensor data were generated and then fused using data-level, feature-level, and decision-level fusions. The presented methods were evaluated using three publicly available human activity recognition (HAR) datasets. The presented approaches utilize Deep Convolutional Neural Networks (CNNs). A generic architecture for the fusion of different sensors is proposed. The proposed method shows promising performance, with the best results reaching an overall accuracy of 98.4% for the Context-Awareness via Wrist-Worn Motion Sensors (HANDY) dataset and 98.7% for the Wireless Sensor Data Mining (WISDM version 1.1) dataset. Both results outperform previous approaches.

• Saplacan, Diana; Herstad, Jo; Tørresen, Jim & Pajalic, Zada (2020). A Framework on Division of Work Tasks between Humans and Robots in the Home. Multimodal Technologies and Interaction. 4(3). doi: 10.3390/mti4030044. Full text in Research Archive Show summary

  This paper analyzes work activity in the home, e.g., cleaning, performed by two actors, a human and a robot. Nowadays, there are attempts to automate this activity through the use of robots. However, the activity of cleaning, in and of itself, is not important; it is used instrumentally to understand if and how robots can be integrated within current and future homes. The theoretical framework of the paper is based on empirical work collected as part of the Multimodal Elderly Care Systems (MECS) project. The study proposes a framework for the division of work tasks between humans and robots. The framework is anchored within existing research and our empirical findings. Swim-lane diagrams are used to visualize the tasks performed (WHAT), by each of the two actors, to ascertain the tasks’ temporality (WHEN), and their distribution and transitioning from one actor to the other (WHERE). The study presents the framework of various dimensions of work tasks, such as the types of work tasks, but also the temporality and spatiality of tasks, illustrating linear, parallel, sequential, and distributed tasks in a shared or non-shared space. The study’s contribution lies in its foundation for analyzing work tasks that robots integrated into or used in the home may generate for humans, along with their multimodal interactions. Finally, the framework can be used to visualize, plan, and design work tasks for the human and for the robot, respectively, and their work division.

• Woll, Anita & Tørresen, Jim (2020). Bridging the User Barriers of Home Telecare. Advances in Intelligent Systems and Computing. ISSN 2194-5357. 1069, p. 816–835. doi: 10.1007/978-3-030-32520-6_59. Full text in Research Archive
• Bråthen, Heidi; Maartmann-Moe, Harald & Schulz, Trenton Wade (2019). The Role of Physical Prototyping in Participatory Design with Older Adults. In Kokil, Uttam & Ota, Tomoko (Ed.), ACHI 2019, The Twelfth International Conference on Advances in Computer-Human Interactions. International Academy, Research and Industry Association (IARIA). ISSN 9781612086866. p. 141–146. Full text in Research Archive Show summary

  How can older adults actively participate in the design processes of assistive robots designed for their homes? We have organized workshops with a group of older adults who worked actively with materials and physical prototyping to design a fetch robot for the home. We present the basics of the workshop materials, how the workshop was performed, and findings on the role of physical prototyping from the workshops.

• Saplacan, Diana & Herstad, Jo (2019). Understanding Robot Motion in Domestic Settings, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781538681299. p. 67–74. doi: 10.1109/DEVLRN.2019.8850695. Full text in Research Archive Show summary

  This paper is a conceptual, empirical, and philosophical exploration of the ways we experience the motion of things. The paper aims to understand better how humans understand the motion of semi-autonomous things. The paper focuses on two perspectives of understanding (robot) motion: 1) through familiarity and a phenomenological approach; and 2) through legibility as a qualitative attribute of motion. Finally, seeing motion through the lenses of these two concepts adds to the development of an understanding of communication or interaction between humans and moving things, and the human relationship to motion.

• Saplacan, Diana & Herstad, Jo (2019). An Explorative Study on Motion as Feedback: Using Semi-Autonomous Robots in Domestic Settings. International Journal On Advances in Software. 12(1&2), p. 68–90. Full text in Research Archive Show summary

  This paper presents motion as feedback. The study is based on empirical data from an explorative study of semiautonomous robots used in domestic settings. We explore feedback received from stationary technology, e.g., a smartphone, and technology that is self-propelled, e.g., a semiautonomous robot. The paper has its theoretical foundation in the familiarity concept used as a contextual and analytical tool for unpacking feedback. The data analysis is done through thematic analysis. The findings are structured in: feedback received from a smartphone app technology, feedback received from the robot-mediated via an app; and motion as feedback received from the robot. Motion as feedback is discussed in terms of: (a) what type of emotions feedback triggers in theusers, and (b) making sense of the motion as positive, negative,homeostatic, archival and transition feedback. We argue that having familiarity in mind when designing new technologies, can make it easier for the user to know-how to engage with the technology. Our conclusion is that: a semi-autonomous robot technology can become more familiar to the user if it triggers positive feelings, if its motion is coherent, if its navigation is appropriate to the situation, and if its motion is not disturbing or interrupting the user; and lastly, familiarity needs to be considered when designing for a robot for the elderly.

View all works in NVA

• Kuoppamäki, Sanna; Lindblom, Diana Saplacan & Pauletto, Sandra (2026). Presentation at the Welfare Technology Fair. Full text in Research Archive
• Saplacan, Diana (2023). Presentation of the paper "Health Professionals’ Views on the Use of Social Robots with Vulnerable Users: A Scenario-Based Qualitative Study Using Story Dialogue Method". Full text in Research Archive
• Saplacan, Diana (2023). Comment on "Vulnerability in Social and Health Robotics" - presentation by Rachele Carli (Univ. of Bologna and Univ. of Luxembourg), as part of the Vulnerability in Robot Society (VIROS) Annual Symposium, Session I on "Vulnerability and Cognitive Sovereignty". Full text in Research Archive
• Tørresen, Jim & Nakasawa, Atsushi (2022). Tutorial: Ethical Considerations in User Modeling and Personalization (ECUMAP). Full text in Research Archive
• Tørresen, Jim (2022). Research Ethics in AI and Robotics. Full text in Research Archive
• Tørresen, Jim (2022). Keynote: Multi-Modal Sensing for Care Robots for Older People. Full text in Research Archive
• Tørresen, Jim (2022). Tutorial: Ethical challenges for Autonomous and Multiagent Systems. Full text in Research Archive
• Cheung, Anne; Saplacan, Diana; Pietropaoli, Stefano & Weng, Yueh-Hsuan (2022). SORO – INTERDISCIPLINARY WORKSHOP ON THE GOVERNANCE FOR SOCIAL ROBOTS (HALF DAY). Full text in Research Archive
• Saplacan, Diana (2022). Ethics and Social Robots: A Universal Design Perspective - Presentation during Interdisciplinary Workshop on the Governance for Social Robots (SORO) at JURIX 2022 (The 35th International Conference on Legal Knowledge and Information Systems). Full text in Research Archive
• Anderson, Adrian Bergflødt,; Søyseth, Vegard & Saplacan, Diana (2022). OPEN ZONE FOR EXPERIMENTAL INFORMATICS (ÅPEN SONE FOR EKSPERIMENTELL INFORMATIKK) as part of "Signals and noise in artificial intelligence" - SENT Festival at the Technical Museum, Oslo, Norway. Full text in Research Archive
• Saplacan, Diana (2022). Ethics: Autonomous Technologies for All (ATA)? Full text in Research Archive
• Saplacan, Diana (2022). Presentation at The 2nd Workshop on Design-Centered HRI and Governance. Title: Making Care Robots Understandable: An Introduction to Universal Design Principles as Design and Ethical Guidelines of Social Assistive Robots. Full text in Research Archive
• Saplacan, Diana (2022). Robots as Welfare Technologies to Reduce Falls Amongst Older Adults: An Explorative Study from Norway. Full text in Research Archive
• Saplacan, Diana (2022). Ongoing research with Social and Assistive Robots - Presentation for the National Commitee in Research Ethics in Science and Technology (NENT). Full text in Research Archive
• Saplacan, Diana (2022). Tutorial presentation. Title: "Robots and Society: Challenges and Opportunities within social Human-Robot Interaction" at 31st IEEE International Conference on Robot & Human Interactive Communication. Full text in Research Archive
• Tørresen, Jim; Atsushi, Nazawa; Saplacan, Diana; Prestes, Edson; Mahler, Tobias & Weng, Yueh-Hsuan (2022). Tutorial at IROS 2022 - Ethical, Legal and User Perspectives on Robots and Systems (ELAUPORAS) – Assessments and Potential Measures. Full text in Research Archive
• Saplacan, Diana (2022). Ethical challenges and opportunities with (care) robots - Universal Design as an ethical charter for inclusive robot design and interaction. Full text in Research Archive
• Brinck, Ingar; Paiva, Ana; Saplacan, Diana & Bensch, Suna (2022). Debate during the symposium: "The Future of Human-Robot Interaction Science" at the Trandisciplinary AI for Good of All (TAIGA) Inauguration Conference, leaded by Prof. Thomas Hellström and Ass. Prof. Niclas Kaiser leading TAIGAS focus area on Embodied interactive AI, at Umeå University, Sweden. Full text in Research Archive
• Saplacan, Diana; Tørresen, Jim; Weng, Yueh-Hsuan & Li, Phoebe (2022). Tutorial: Robots and Society (RO-SO 2022) / Ethical perspectives and technical challenges and opportunities with care robots. Full text in Research Archive
• Tørresen, Jim (2022). Tutorial: Ethical Challenges in Computational Intelligence Research. Full text in Research Archive
• Saplacan, Diana (2022). Ethics: Research Ethics and Autonomous Technologies for User Diversity. Full text in Research Archive
• Saplacan, Diana (2022). What do we talk about when we talk about social robots vs. robot sociomorphism ? Empirical examples from previous and ongoing research. Full text in Research Archive
• Dahl, Heidi Elisabeth Iuell; Mehmandarov, Rustam Karim; Harkestad, Inge & Saplacan, Diana (2022). AI and Ethics workshop. Full text in Research Archive
• Tørresen, Jim (2022). Ethical Perspectives of Robotics and AI – How to develop preferable system? Full text in Research Archive
• Tørresen, Jim (2022). Research on health-related treatment and care technology – A technical and ethical view. Full text in Research Archive
• Tørresen, Jim (2022). Sensing, acting and adapting in the real world. Full text in Research Archive
• Saplacan, Diana (2022). TIAGo wishes you welcome to a debate on the theme: Should a robot be involved in care tasks? (Norwegian title: TIAGo ønsker velkommen til en debatt med tema: Bør en robot involveres i omsorgsoppgaver?). Full text in Research Archive
• Saplacan, Diana; Baselizadeh, Adel & Tørresen, Jim (2022). Robot Demonstration: Meet TIAGo the robot! The robot showcases several tasks: brushing hair, putting the lipstick on, using a (plastic) knife, moving an object, carrying a bag. Full text in Research Archive
• Saplacan, Diana (2022). Guest Lecture: Ongoing research with Social and Assistive Robots: Research projects, empirical examples, and theory. Full text in Research Archive
• Saplacan, Diana (2022). Ethical Issues in Mobile Robots - Guest Lecture for Norwegian University of Life Sciences (NMBU), at the Faculty of Science and Technology, part of Human-Robot Interaction Course, Advanced Topics in Mobile robots. Full text in Research Archive
• Saplacan, Diana (2022). The importance of standarization within new technological areas. Full text in Research Archive
• Saplacan, Diana (2022). Healthcare Professionals’ Attitudes towards the Organization of Care Services and the Adoption of Welfare Robots in Norway. Full text in Research Archive
• Saplacan, Diana (2022). Participatory Design and Human-Robot Interaction - An ethical and inclusive perspective on contemporary technologies. Full text in Research Archive
• Saplacan, Diana (2021). Ethics and Technology. Full text in Research Archive
• Noori, Farzan Majeed (2021). Recognition of Human Activities using UWB Radar and Deep Learning. Full text in Research Archive Show summary

  The population of older adults are increasing every day, and the trend is expected to continue. With the help of advanced assistive technologies, we can provide them better healthcare. In this work, we provide a novel sensing approach based on Ultra-wideband (UWB) sensors to recognize human activities and forecast future events. Previously, researchers focused on wearable or video sensors to detect a person’s behaviour. However, it is challenging for older people to wear devices 24/7. Similarly, vision-based sensors always carry privacy concerns. In contrast, a non-contact ambient sensor with no such privacy issues is XeThru ultra-wideband (UWB) radar. We collected the data using multiple modalities, such as UWB, depth images, thermal images, and actigraphy device, i.e. to collect heart rate (HR) for ground truth. The dataset comprised of the participants sitting on the sofa in normal situation. Afterwards, we recommended participants do some exercise for the sake of increasing HR. When the HR increased by more than 140 BPM, the users lie down on the floor in front of sensors until their HR reached a normal level. In this research, we classify normal vs abnormal situations using CNNs and LSTMs. Accuracy, precision, and recall are used as performance measures. We got 95% and 98% accuracies using CNNs and LSTMs, respectively. Furthermore, seven classes were introduced based on HR levels, as shown in Table1. We got promising results with LSTMs. A confusion matrix is shown in Figure 1. It was the first step towards classifying activities using UWB radar. In the future, we are planning to predict the future HR based only on UWB data, as it would be challenging for older people to wear actigraphy devices. Moreover, we will include other modalities which we did not include in our preliminary analysis.

• Saplacan, Diana (2021). Ethics and technology: Research With --, About --, and for Users. Full text in Research Archive
• Otterdijk, Marieke van (2021). De Kennis van Nu” (The knowledge of now). [TV]. Dutch national TV (NPO). Full text in Research Archive
• Tørresen, Jim (2021). How to achieve ethical artificial intelligence (AI) research and development? Full text in Research Archive
• Saplacan, Diana (2021). AI, Robots and Their Social Impact: An Introduction to AI, ML, and Robots. Full text in Research Archive
• Saplacan, Diana; Khaksar, Weria & Tørresen, Jim (2021). On Ethical Challenges Raised by Care Robots: A Review of the Existing Regulatory-, Theoretical-, and Research Gaps. Full text in Research Archive
• Tørresen, Jim (2021). Tutorial: Ethical Considerations in Robotics and Automation. Full text in Research Archive
• Tørresen, Jim (2021). Tutorial: Explainability, Trust and Ethics for Robots and Autonomous Systems. Full text in Research Archive
• Saplacan, Diana (2021). Challenges and Opportunities with AI-based Care Robots: An Ethical and Inclusive Approach. Full text in Research Archive
• Tørresen, Jim; Saplacan, Diana; Lintvedt, Mona Naomi; Mahler, Tobias & Fosch-Villaronga, Eduard (2021). Tutorial: Ethical and Legal Assessments Related to Robots and Systems. Full text in Research Archive
• Tørresen, Jim (2021). Tutorial: Intelligent System Research – AI Ethical Challenges and Opportunities. Full text in Research Archive
• Tørresen, Jim (2021). Tutorial: Ethical Considerations in User Modeling and Personalization. Full text in Research Archive
• Tørresen, Jim (2021). INTROMAT: INtroducing personalized TReatment Of Mental health problems using Adaptive Technology. Full text in Research Archive
• Tørresen, Jim (2021). Kunstig intelligens – allsidig i metoder og anvendelser. Full text in Research Archive
• Tørresen, Jim (2021). Introducing a Multimodal Elderly Care Safety Alarm Robot. Full text in Research Archive
• Tørresen, Jim (2021). Tutorial: Ethical Considerations in the Development and Use of Neural Information Processing Systems. Full text in Research Archive
• Saplacan, Diana (2021). Ethical Challenges and Opportunities with Care Robots Universal Design as An Ethical Charter for Inclusive Robot Design and Interaction. Full text in Research Archive
• Saplacan, Diana (2021). Care Robots and Universal Design - An Ethical Charter for Designing Care Robots and Human-Robot Interaction. Full text in Research Archive
• Saplacan, Diana; Tørresen, Jim; Mahler, Tobias & Fosch-Villaronga, Eduard (2021). Robots and Society: Ethical, Legal, and Technical Perspectives on Integrating Robots in the Home- and Healthcare Systems and Services (RO-SO). Full text in Research Archive Show summary

  The RO-SO tutorial is a continuation of the previous tutorials regarding robots, ethics, legal, and technical aspects. The tutorial is part of the 30th IEEE International Conference on Robot and Human Interactive Communication (ROMAN 2021). It focuses on providing an overview and discussing the ethical, legal, and technical perspectives, including challenges, dilemmas, and advantages and opportunities of integrating robots as part of home- and healthcare systems and services. The following two questions will guide the overall objective for this tutorial, but these will not limit the talks: How does the meaning of good care change when we integrate robots as part of the home- and healthcare systems and services? What does good care mean when we integrate assistive robots as part of the home- and healthcare services? What are some of the ethical, legal, and technical challenges and opportunities when integrating assistive care robots in the home- and healthcare services?

• Saplacan, Diana (2021). "How can I trust that you care about my care, RO-BOT?" - A talk about ethics in the context of care robots. Full text in Research Archive
• Saplacan, Diana (2021). Ethical challenges and opportunities with care robots (Talk at RO-MAN 2021, in PANEL I on Introduction on Ethics and Care Robots). Full text in Research Archive
• Saplacan, Diana (2021). Some experiences about analysis and writing it up - On qualitative data analysis methods. Full text in Research Archive
• Noori, Farzan Majeed; Uddin, Md Zia & Tørresen, Jim (2020). Presentation on In-Home Emergency Detection Using an Ambient Ultra-Wideband Radar Sensor and Deep Learning. Full text in Research Archive
• Tørresen, Jim; Kurazume, Ryo & Prestes, Edson (2020). Guest Editorial: Special Issue on Elderly Care Robotics – Technology and Ethics. Journal of Intelligent & Robotic Systems. ISSN 0921-0296. 98, p. 3–4. doi: 10.1007/s10846-020-01148-6. Full text in Research Archive
• Tørresen, Jim (2020). Ethical Consideration in Robotics and Intelligent Systems Research. Full text in Research Archive
• Tørresen, Jim (2020). Tutorial: Ethical Challenges and Opportunities within Computational Intelligence System Development. Full text in Research Archive
• Tørresen, Jim (2020). Addressing Ethical Challenges for Care Giving Robots. Full text in Research Archive
• Tørresen, Jim (2020). Ethical Considerations in User Modeling and Personalization: ACM UMAP 2020 Tutorial. In Kuflik, Tsvi; Torre, Ilaria & Burke, Robin (Ed.), Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. Association for Computing Machinery (ACM). ISSN 9781450368612. p. 390–391. Full text in Research Archive
• Tørresen, Jim (2020). Addressing ethical challenges within evolutionary computation applications: GECCO 2020 tutorial. In Coello, Carlos A. Coello (Eds.), GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery (ACM). ISSN 9781450371278. p. 1206–1223. Full text in Research Archive
• Tørresen, Jim (2020). Tutorial: Addressing Ethical Challenges within Evolutionary Computation Applications. Full text in Research Archive
• Tørresen, Jim (2020). Sensing Human State with Application in Older People Care and Mental Health Treatment. Full text in Research Archive
• Tørresen, Jim (2020). AI Ethics: How to achieve ethically good artificial intelligence research and development. Full text in Research Archive
• Tørresen, Jim (2020). Tutorial: Ethical Considerations in User Modeling and Personalization. Full text in Research Archive
• Tørresen, Jim (2020). Interdisciplinary Research Collaboration in Rhythm, Time and Motion. Full text in Research Archive
• Tørresen, Jim (2020). Intelligent Robots and Systems for Foundational and Applied Healthcare Research. Full text in Research Archive
• Noori, Farzan Majeed; Tørresen, Jim; Uddin, Md Zia & Riegler, Michael A. (2023). Multimodal Deep Learning Approaches for Human Activity Recognition. Universitetet i Oslo. Full text in Research Archive Show summary

  Smart homes may be beneficial for people of all ages, but this is especially true for those with care needs, such as the elderly. To assist, monitor for emergencies, and provide companionship for the elderly, a substantial amount of research on human activity recognition systems has been conducted. Several algorithms for activity recognition and prediction of future events have been reported in the scientific literature. However, the majority of published research does not address privacy concerns or employ a variety of ambient sensors. The objective of this thesis is to contribute to the progress in research relevant to activity recognition systems that use sensors that collect less privacy-related information. The following tasks are included in the work: assessment of sensors while keeping privacy concerns in mind, selection of cutting-edge classification methods, and how to fuse the data from multiple sensors. This thesis contributes to making progress on systems for analyzing human activity and state—or vital signs—for application in a mobile robot. This dissertation examines two topics. First, it examines the privacy concerns associated with having a robot in the home. On a robot, an ultra-wideband (UWB) radar-based sensor and an RGB camera (for ground truth) were installed. An actigraphy device was also worn by the users for heart rate monitoring. The UWB sensor was selected to maintain privacy while monitoring human activities. Considering different ways to represent data from a single sensor is the second topic under investigation. That is, how data from multiple representations can be combined. For this purpose, we investigate various data representations from a single sensor’s data and analysis using cutting-edge deep learning algorithms. The contributions provide considerations for equipping a mobile home robot with activity recognition abilities while reducing the amount of privacy-sensitive sensor data. The work also concerns examining the potential privacy restrictions that must be established for the analyzing systems. The thesis contains new methods for combining data from multiple information sources. To achieve our objective, convolutional neural networks and recurrent neural networks were applied and validated using conventional methods. The conclusion of the thesis is that we can achieve good accuracy with limited sensors while maintaining privacy. It is, however, likely adequate for assisting healthcare personnel and caregivers in their work by indicating current activity status and measuring activity levels, providing alerts about abnormal activities. The results can hopefully contribute to older people being able to live alone in their homes with a larger chance of any unwanted events being quickly detected and notified to the caregivers and providers.

• Soma, Rebekka; Saplacan, Diana & Sikora, Magdalena Claudia (2022). Exploring personality and behavior with a robot table - a qualitative case study. Universitetet i Oslo. Full text in Research Archive
• Wang, Lina; Tørresen, Jim & Noori, Farzan Majeed (2021). Emergency Monitoring of Older People Using Multiple Sensors with Privacy in Consideration. Universitetet i Oslo. Full text in Research Archive
• Orten, Mathias Kramer & Tørresen, Jim (2021). Machine Ethics for Autonomous systems – Learning verifiable rules from the environment. Universitetet i Oslo. Full text in Research Archive
• Saplacan, Diana (2020). Situated Abilities: Understanding Everyday Use of ICTs. Universitetet i Oslo. Full text in Research Archive Show summary

  The thesis is positioned at the cross of Human-Computer Interaction (HRI), Human-Robot Interaction (HRI), and Computer-Supported Cooperative Work (CSCW) fields, with Universal Design (UD) concepts and literature being orthogonal to those. The main thesis’ contribution is the concept of situated ability that emerged from the findings of two different cases included in the thesis. The situated abilities concept is defined, framed, explained, and exemplified with concrete examples. Its anatomy is presented together with the situated ability continuum that includes low- and high-end abilities. The concept is theoretically anchored in Heidegger’s concept of Befindlichkeit (= situatedness). Other smaller contributions consist of a salutogenic approach to design, concept development, introducing qualitative data analysis methods well established in the medical field to the design fields, and introducing a workshop method of both data collection and analysis to the HCI community. The work is relevant for an audience interested in theoretical and philosophical explorations in design research fields, but also practitioners and activists within Universal Design, or those who wish to create ethical or legal frameworks, guidelines, or recommendations addressing the use of modern technology, such as robots, in the public sector (e.g., homecare, healthcare or education).

• Schulz, Trenton Wade (2020). Exploration of Moving Things in the Home. Universitetet i Oslo. Full text in Research Archive Show summary

  With more and more people getting older and needing extra medical attention, robots may be a way of helping them live independently at home longer. Robots in the home, however, can gather lots of personal information about people living in the home and a robot may be confusing to understand. It is therefore important that these robots respect the residents' privacy and can be seen as friendly and welcome in the home. Future designers of robots can address privacy concerns by examining what data the robot collects, what dilemmas can come from the data collection, and design to address the dilemmas. In addition, these designers can use techniques from animation, such as the twelve principles of animation, to add expressiveness to a robot and make it easier to understand. The dissertation presents a technique for moving a robot with the animation principle of slow in and slow out and investigate how people experience a robot moving in the home using this technique.

View all works in NVA