Background:
Echocardiography is a non-invasive diagnostic tool that utilizes ultrasound for visualizing the heart. The information about cardiac structures and function obtained from these images can provide doctors with incremental information for patient management in a variety of settings ranging from evaluation of cardiac murmurs to suspected heart attacks (1). A crucial premise for obtaining useful information are high-quality images obtained in a standardized manner for which methods of interpretation have been validated (2). Today, achieving images suitable for diagnostic purposes necessitates a skilled human operator who has undergone extensive training, with the time required for achieving the most comprehensive level of competence often spanning several years (3, 4).
While portable echocardiography machines are becoming more accessible, allowing for wider use of echocardiography in different settings, the specialized skills necessary for acquiring and analyzing images are not as widespread. This limitation could restrict further adoption and application. However, with robotics advancements, the concept of teleoperated echocardiography, where an operator remotely controls a robotic arm to capture echocardiographic images, has been proposed and shown to be feasible(5, 6).
Concurrently, the escalating elderly population is anticipated to further extend the stress placed on the healthcare system (7, 8). As the proportion working-age individuals compared to patients declines, optimizing and utilizing the available workforce efficiently becomes necessary. Automation, an effective method for performing tasks accurately and without fatigue, holds promise in improving efficiency and effectiveness. Driven by these considerations, the proposed project aims to develop an automated system that can capture high-quality echocardiographic images. This system will be designed to function without the presence of skilled human operators, either on-site or remotely. The primary goal is to reduce the workload on medical staff and expand the accessibility of this health screening to a broader spectrum of situations.
Aim: Developing an automated system for acquiring high-quality echocardiographic images without the need for skilled human operators on site or remotely.
Method: To the best of our knowledge, no existing system facilitates the automatic acquisition of high-quality echocardiography data. However, a recent publication by Soemantoro et al. (9) showcased a promising simulation-based proof of concept for such an endeavor. Drawing insights from their innovative approach, we propose an automation architecture that divides the system into three primary modules. One of the main innovations of our proposed work is the incorporation of safe reinforcement learning (10), offering a more robust and generalizable method for ultrasound probe manipulation:
- Image Quality Assessment Model: Serving as the primary reward signal, this module will take the signal recorded from the echocardiogram data as input and output a quality assessment of the recorded image. By evaluating the current image quality, it can guide the subsequent movement required from the probe to achieve a higher image quality. This module is based on a pre-trained deep learning model that will undergo further refinement using data gathered during the proposed project.
- Echocardiography-Window Classification Model: This component autonomously identifies the specific type of image captured by the probe. It differentiates between various echocardiographic viewpoints, for example the apical 4-chamber and 2-chamber views. Its main function is to direct the probe through the range of views needed during an examination. Like the previous module, it is also founded on a pre-trained deep learning model, set for further fine-tuning with project-specific data.
- Robotic Arm Control Module: Responsible for controlling the robotic arm, this module adjusts the position of the probe based on feedback from the first two modules and a camera. The data from the camera in conjunction with the robotic arm’s current actuator state will be used to determine the relative position of the probe to the patient. The module employs the principles of safe reinforcement learning for precise control of the end-effector holding the ultrasound probe.
Collectively, these modules will enable the automatic acquisition of diagnostically relevant echocardiographic data.
References
1. Bennett S, Stout M, Ingram TE, Pearce K, Griffiths T, Duckett S, et al. Clinical indications and triaging for adult transthoracic echocardiography: a consensus statement by the British Society of Echocardiography. Echo Res Pract. 2022;9(1):5.
2. Mitchell C, Rahko PS, Blauwet LA, Canaday B, Finstuen JA, Foster MC, et al. Guidelines for Performing a Comprehensive Transthoracic Echocardiographic Examination in Adults: Recommendations from the American Society of Echocardiography. J Am Soc Echocardiogr. 2019;32(1):1-64.
3. Kirkpatrick JN, Grimm R, Johri AM, Kimura BJ, Kort S, Labovitz AJ, et al. Recommendations for Echocardiography Laboratories Participating in Cardiac Point of Care Cardiac Ultrasound (POCUS) and Critical Care Echocardiography Training: Report from the American Society of Echocardiography. J Am Soc Echocardiogr. 2020;33(4):409-22 e4.
4. Popescu BA, Andrade MJ, Badano LP, Fox KF, Flachskampf FA, Lancellotti P, et al. European Association of Echocardiography recommendations for training, competence, and quality improvement in echocardiography. Eur J Echocardiogr. 2009;10(8):893-905.
5. Solvin H, Sajadi S, Lippert M, Massey R, Holmstrøm H, Elle O, et al. Feasibility of teleoperated robotic echocardiography – A pilot study. WFUMB Ultrasound Open. 2023;1(2).
6. Mathiassen K, Fjellin JE, Glette K, Hol PK, Elle OJ. An Ultrasound Robotic System Using the Commercial Robot UR5. Frontiers in Robotics and AI. 2016;3.
7. Gassen NSn, Timothy H. THE NORDIC POPULATION IN 2040 – Analysis of past and future demographic trends. 2019.
8. Haugstveit FV, Otnes B. Vesentlig mer bruk av omsorgstjenester ved passerte 85 år: Statistisk Sentralbyrå, Sratistics Norway; 2019 [Available from: https://www.ssb.no/helse/artikler-og-publikasjoner/vesentlig-mer-bruk-av-omsorgstjenester-ved-passerte-85-ar.
9. Soemantoro, R., Kardos, A., Tang, G., & Zhao, Y. (2023). An AI-powered navigation framework to achieve an automated acquisition of cardiac ultrasound images. Scientific Reports, 13(1), 15008.
10. Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022). Safe learning in robotics: From learning-based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 5, 411-444.