Augmented therapeutic tutoring in diligent image-assisted robotic interventions

1.   Introduction

Recently, medical interventions have moved from completely intrusive interventions, through the medical team's tactile skills and direct visual observations, to adjusted, image-guided, closed-loop robotic processes. These precise and minimally intrusive procedures primarily involve complex surgical procedures and drug delivery, which can be supervised by medical staff. In these two intervention categories, local position accuracy is crucial for strictly covering the affected area without affecting the surroundings. Thus, patient well-being (comfort and safety) and staff performance (accuracy and efficiency) shape a successful medical outcome. This advancement is not only beneficial for humans but follows the “One Health” concept regarding humans, animals, and ecosystems [1]–[3]. This concept was established to describe the awareness that human health is convolutedly associated with other animals and their environments. This is normally linked to vital threats but also to improvements.

In the image-assisted robotic process mentioned above (replacing human fingers and vision), image assistance could be achieved using predetermined digitized images of the relevant body area. A more effective and instantly controlled method is to directly use an imaging scanner. This solution, even more expensive, avoids doubts and possible interruptions of the procedure. In this case, the robotic process is assisted by an instantaneous image. In theory, all types of scanners could be used in these image-guided robotic interventions; different categories of scanners are normally used, each tailored to a specific target. An important selection factor is the usage interval: For treatments at long intervals, it is preferable to avoid scanners reflecting ionizing radiation. Thus, the relatively long exposure interval of image-guided interventions suggests the use of non-ionizing scanners, namely magnetic resonance imaging (MRI) and ultrasound scanners (see e.g., [4]–[7]). The first is more complex and expensive but works with all tissues of the body and is more precise. Caution is advised when using MRI due to its sensitivity to electromagnetic noise. The ultrasound scanner is almost portable and adequate for easy daily use but does not work in bones or air cavities. In general, image-assisted robotic interventions are minimally invasive and non-ionizing. Different dedicated scanners have been built to allow the introduction of different body parts inside the imaging scaffold. In addition, in the case of MRI, the robot and medical tools are made from materials not sensitive to electromagnetic fields, called MRI-compatible. Several investigations and clinical applications could be found in the literature: see e.g., [8]–[10].

The closed-loop, quasi-autonomous image-guided robotic procedure, in addition to having a beneficial impact on patient well-being, can enable medical staff to perform useful tasks in the absence of the patient. These tasks are primarily related to the exploration of new therapies and treatments as well as training exercises and testing. This could be done using “copies” of the patient's body: a physical phantom constructed from materials matching the biological properties of real body tissues. Regarding training, exercises performed by medical staff on physical phantoms help to reveal the most suitable treatment for the real patient. Another type of body copy is a digital replica matching real tissue properties; it would be a digital phantom. These physical and digital phantoms make it possible to investigate different scenarios without risk for the patient, but the construction of such physical and digital phantoms is not without problems. Physical phantoms have been used in conventional medical testing for a while and can be easily found on the market. Much research relating to digital constructs can be found in the literature (e.g., [11]–[15]).

The purpose of this commentary is to analyze and illustrate the possibilities of increasing the capabilities of medical personnel to manage training exercises for image-assisted robotic diligent interventions. The contribution will focus on the use of the physical-virtual digital twin concept to supervise image-assisted robotic interventions, thus reducing the complexity and uncertainties involved in such a procedure.

Section 2 illustrates the image-assisted robotic intervention procedure and analyzes the characteristics of physical and digital tissue phantoms. The representation of the dynamic behavior of tissues will be highlighted via the concept of complexity. Section 3 presents the characters of the physical-virtual pairs and their roles in reducing the complexity issue. Section 4 concerns the use of the physical-virtual digital twin concept in healthcare, focusing on its use in image-assisted robotic interventions and its extension to human-in-the-loop DT. Section 5 concerns the therapeutic tutoring of medical staff; disease monitoring and diagnosis, general training, and personalized planning training are also analyzed.

2.   Image-assisted robotic interventions

As mentioned previously, image-assisted robotic interventions enable minimally invasive and generally non-ionizing procedures, allowing for high-precision localization. Thus, these interventions are mainly used for complex surgical procedures and restricted drug delivery. Figure 1 illustrates an example of an autonomous, collaborative, autonomous, image-assisted restricted drug delivery system.

Figure 1.  Schematic of autonomous image-assisted robotic intervention of drug-restricted delivery.

DownLoad: Full-Size Img PowerPoint

2.1.   Physical and digital tissue phantoms

A fully image-assisted robotic intervention can be performed with a physical phantom. Additionally, a digital phantom could be used in a digitally simulated intervention. The main difficulty of these two options lies in the representation of the tissues. Accounting for biological assets allows the construction of these phantom representations with static behavior. Such a static picture could be appropriate for specific living tissues and applications. However, as mentioned earlier, physical and digital phantoms should approach real living tissues through realistic biological properties. In general, the deformation and movement (mechanical behavior) of living soft tissues, irrigated by the fluids that enable their functioning, correspond to a complex dynamic behavior model [16]–[20]. This complex behavior of living tissues requires real, estimated constitutive laws, adjusted computational methods, or both to solve this open research problem. We can only approach this type of model or representation in an approximate manner.

2.2.   Complexity

The notion of complexity is implicated in many natural and artificial phenomena [21] as well as societal concerns. Thus, complexity could be found in various areas, e.g., neuroscience [22], global climate [23], space shuttles [24], computer image processing [25], fair division economics [26], strategic interactions in game theory [27], politics and governments [28], and history sciences [29],[30]. These composite activities interact in complex ways, exhibiting nonlinear, codependent spatiotemporal behaviors.

3.   Physical-virtual matched pairs

A suitable solution to the complexity of dynamic tissue behavior is the use of physical representation and digital replication in a tailored procedure. Such a coupling process is well suited to uncertain (approximate) physical and numerical aspects. Matching iterations allow each side of the physical-digital pair to correct the other, thereby reducing physical and digital uncertainties [31]. This matching twin corresponds to the concept of digital twin (DT) introduced by Grieves in 2001 [32]. DTs are used in many applications (e.g., [33]–[35]) in different fields such as industrial manufacturing and production, mobility, aeronautics, space, or health. This strategy makes it possible to supervise complex dynamic procedures in real-time, reducing uncertainties and controlling all unforeseen events and undesirable and dangerous behaviors. Another particularity of DT is that the virtual model is supposed to be precise and very close to the real procedure. For a complex procedure, its model would also be complex, therefore with a high calculation time leading to slow and degraded matching. For fast matching, we need to reduce the computation time by using strategies to preserve the accuracy of key attributes. Thus, reduced order or surrogate models can be used for this purpose; see e.g., [36],[37].

4.   DT and healthcare

Many applications in the health field use virtual replicas of physical entities, which go further than a static image, integrating the dynamic behavior of a real living character [38]–[44].

4.1.   Application of image-assisted robotic interventions

In the present study, the concept of DT could be used to monitor the entire image-assisted robotic intervention system [45], including a physical phantom of the concerned part of human tissue; this represents the true physical side of the twin. A virtual model involving the digital phantom replica of the real side will represent the virtual side of the twin. Between the two sides is a link involving the matching and processing of information (Figure 2).

Figure 2.  Summarized illustration of a matched monitoring of an image-assisted robotic intervention with its virtual model. Physical and digital tissue phantoms reflect dynamic behaviors.

DownLoad: Full-Size Img PowerPoint

4.2.   Human-in-the-loop DT

In principle, the concept of DT can be used with real patient tissues, which can enable precise supervision of the intervention. In such a case, we can use a modified DT by including a human-in-the-loop. This corresponds to a human–robot interaction associated with DT, called HDT [46]–[48], which allows a predominantly human manipulation in the procedure, thus minimizing the risks for the patient. HDT requires access to numerous health data sources, including genetic and historical records, Internet of Things (IoT), and other appropriate providers. Data is processed through advanced analytics and artificial intelligence (AI) methods, including deep learning models that can improve the intelligence of HDT simulations, enabling predictive models and confirmation of decision-making. Indeed, the involvement of AI practices in image-assisted robotic intervention contributes to reducing the complexity of information gaining and post-processing in imaging through the practice of acceleration approach, with quicker analysis times and simpler image processing [49],[50]. AI can also be used to perform planned repeated training jobs in image-assisted robotics.

5.   Therapeutic tutoring

DT has the promise to transform patient care by delivering progressively tailored, data-driven medical treatment. Its application is a part of digital medicine and is generally used in personalized medical treatments, which can be the observation and diagnosis of diseases, tutoring, or interventions. DTs are often associated with AI and augmented reality (AR) tools, as well as virtual reality (VR) and chatbots. Increased interaction between the human and the robot increases the system's performance through AR-assisted robotic operations. Also, VR training improves the ability to mimic real-world exercise scenarios with the skill to correctly quantify performance, while chatbots allow training via the intelligent creation of tutoring paths. Tutoring primarily involves planning a personalized case or the exploration of new therapies through general training.

5.1.   Disorders monitoring and diagnostics

DTs allow medical staff to supervise the evolution of disorders and adjust care strategies, choosing the most appropriate therapy, thus enabling the development of personalized and early-diagnosis procedures and research of new medications or interventions [51],[52].

5.2.   General training

The elementary DT training of younger staff is related to the creation of virtual anatomy and bodily processes models to allow practicing, enhance dexterity, and refine their understanding of the human body tissues. DT general staff training includes acting out healthcare scenarios, getting vital marks, dispensing drugs, and facing urgent situations. This can be related to patient concerns, e.g., cardiac arrests, or environmental conditions, e.g., fires. In addition, DT allows training on the functioning and safeguarding of medical tools and infection-handling routines [38],[39].

5.3.   Personalized planning training

Personal DT patient planning allows medical staff to predict the disruptions that patients may experience during procedures. This could be related to tools, drugs, or implementations. Furthermore, DT allows staff, in a safe and patient-free situation, to practice procedures, make judgments, and understand possible errors of interpretation [45],[53]–[56].

6.   Conclusions

This commentary focused on the analysis of image-assisted robotics, complex surgery, and restricted drug delivery, which require high-precision positioning for minimally invasive and safe procedures. The uncertainty in the representation of the dynamic behavior of living tissues is reduced by a matched physical-virtual pair applied to interventional procedures. These developments enable effective training of staff on tissue copies in the case of general information as well as planning personalized interventions. The various subjects concerned were supported by a review of the literature. It should be noted that, although matched pairs could reduce the uncertainty of the tissue dynamics model, this model is still difficult to determine and corresponds to an unresolved open challenge.

Future prospects include improving robotic assistance by the scanner using AR and AI tools, achieving complete automation, ensuring patient safety, and utilizing DT in real-time. This allows for more precise, minimally intrusive restricted actions with the option of firm human control. One of the most challenging prospects involves the real-time virtual representation of tissues (dynamic model) within DT, as mentioned before. The main challenge is the nonlinear tissue behavior, which exhibits complex responses to deformation and displacement of elastic tissues.

Use of AI tools declaration

The author declares he has not used Artificial Intelligence (AI) tools in the creation of this article.