A prostate seed implantation robot system based on human-computer interactions: Augmented reality and voice control

1.   Introduction

In the traditional prostate seed implantation surgery, doctors use transrectal ultrasound (TRUS) to obtain medical images to judge the spatial relationship between prostate tissue and the needle ^[1]. The medical images obtained by TRUS have limited vision space, insufficient image quality and lack of depth information, which makes the surgical procedure extremely dependent on the professional knowledge and experience of doctors ^[2,3]. With the development of science and technological innovation, robot-assisted seed implantation has been widely used. However, the visualization problem of prostate seed implantation exists, which makes the process of robot-assisted prostate seed implantation more unstable, and the curative effect is severely limited ^[4].

The development of augmented reality technology has become a reality to provide doctors with high-quality visualization functions in the surgical field. Augmented reality technology can fuse real scenes and virtual scenes in a more natural way, which ensures the intuitiveness and real-time of visual information, and plays a role in intraoperative navigation ^[5,6,7]. The three-dimensional virtual model reconstructed by two-dimensional tomography can make up for the shortcomings of traditional medical images. Pre-operative planning in the three-dimensional model can help doctors obtain a more accurate surgical path and a more intuitive spatial relationship.

Robot-assisted prostate seed implantation can meet the technical and application requirements of the surgery ^[8,9]. In the known medical robots, the control methods are usually traditional methods such as mouse and keyboard. These control methods are all contact control, which easily leads to two-way pollution between doctors and instruments. Voice control technology is a new non-contact control method. Combined with the set voice control instruction, the surgical visualization system can be organically combined with the robot motion control system, so as to ensure the corresponding relationship between the visualization information and the robot motion information ^[10]. Li et al. ^[11] designed a pneumatic robot system for prostate seed implantation which can work in MRI environment.

In terms of prostate seed implantation robot, Fichtinger et al. ^[12] developed a robot-assisted prostate brachytherapy system, which successfully completed clinical feasibility and safety trials. Djohossou et al. ^[13] studied a compact and lightweight delta robot to improve the accuracy of prostate brachytherapy. Halima et al. ^[14] developed a 6-DOF compact and lightweight cooperative robot for prostate brachytherapy. Wang et al. ^[15] improved the structure of the RRT cantilever articulated seed implantation robot and designed a multi-needle soft tissue stable insertion device. The device can fix the prostate so that the needle can be inserted accurately. Chen et al. ^[16] proposed a robot system for prostate brachytherapy guided by TRUS. The system adopts a new automatic needle release mechanism to realize multi-needle insertion into prostate according to real-time dose measurement. Jiang et al. ^[17] proposed a new transperineal prostate brachytherapy robot system. The puncture positioning error of the robot can be limited within 0.7 mm. Dai et al. ^[18] designed an 11-DOF transrectal prostate seed implantation robot. The attitude adjustment module based on double parallelogram mechanism is designed, so that the ultrasonic probe can do centering motion. Bakouri et al. ^[19] designed a robot wheelchair based on convolutional neural network voice control. Tran et al. ^[20] developed a soft robot exoskeleton controlled by voice to assist hand movement, and evaluated the system.

Augmented reality technology provides visual navigation function for seed implantation surgery, which can make up for the limitations of two-dimensional imaging. Watanabe et al. ^[21] proposed a perspective neuronavigation system. The system uses augmented reality technology and motion capture system to ensure navigation functions with minimum blind spots. Cohen et al. ^[22] developed an AR guidance system for robot-assisted laparoscopic surgery. Yamamoto et al. ^[23] have developed an interactive interface for remote minimally invasive surgical robots, which provides augmented reality visual feedback using three-dimensional graphics overlay. Song et al. ^[24] introduced the HMD-based AR prototype into endodontic surgery for the first time, and the system can be used to dynamically guide dentists to operate surgical instruments for posterior teeth treatment. Gîrbacia et al. ^[25] proposed an AR system using RGB-D camera. Lee et al. ^[26] adopted a vision-based augmented reality (AR) tracking system. The system helps doctors locate recurrent laryngeal nerves (RLN) during robotic thyroid surgery. Xu et al. ^[27] developed a surgical navigation system based on space augmented reality to assist the placement of pedicle screws in minimally invasive spinal surgery. Samei et al. ^[28] proposed a robot image guidance system to assist laparoscopic radical prostatectomy. The system has been successfully applied in the operation of 12 patients. Schiavina et al. ^[29] performed radical prostatectomy on 26 patients using a robotic surgical system guided by real-time augmented reality, and evaluated the surgical results.

Voice control technology is a more direct and convenient way of human-computer interaction. According to the actual needs, doctors can control the corresponding equipment by voice. As early as the 1990s, voice control was used in medical robots. Reichenspurner et al. ^[30] controlled the robotic arm to locate the endoscope through voice control technology. Zinchenko et al. ^[31] developed a voice control method of endoscope stand for long-distance movement control. Gundogdu et al. ^[32] developed a voice control system to control the robotic arm. The control system has been applied to RRRR robotic arm. The developed system improves the efficiency of voice recognition. Ruzaij et al. ^[33] proposed a multi-mode voice controller for rehabilitation. Pramanik et al. ^[34] designed a fully voice-controlled robot that can be used as a 24-hour hospital assistant. The voice control system has a training system, which can guide robots in multiple languages. Matarneh et al. ^[35] proposed a set of voice instructions for intracranial surgery. Doctors control the medical robots through voice instructions.

In this paper, a prostate seed implantation robot system based on AR and voice control is proposed, which provides a more friendly visualization system and control system. The MRI medical image of prostate is denoised and segmented, and then the three-dimensional virtual model of prostate is reconstructed by surface rendering technology. Based on the obtained three-dimensional virtual model of prostate, the architecture and scene of augmented reality system are constructed. An improved singular value decomposition (SVD) three-dimensional registration algorithm based on iterative closest point (ICP) is proposed, and the accuracy experiment, real-time experiment, and repetitive experiment of the three-dimensional registration algorithm are used to verify the improved algorithm. The scheme and process of voice control are defined, and the non-real-time problem of voice control system is optimized by three-dimensional registration technology. Finally, a voice denoising fusion algorithm based on spectral subtraction and back propagation neural network (BPNN) is proposed to denoise and optimize the voice signal. The experimental platform of voice control is built, and the reliability of the fusion algorithm is verified by the voice signal-to-noise ratio experiment, denoising delay experiment, voice denoising waveform diagram experiment, and voice recognition experiment.

2.   Materials and methods

2.1.   Augmented reality system of prostate seed implantation

2.1.1.   Three-dimensional model reconstruction

Median filtering has a good effect on eliminating salt and pepper noise and it can avoid the damage to the edge structure of image features. Therefore, median filtering method is used to denoise the medical image of the prostate. The comparison of prostate magnetic resonance imaging (MRI) before and after denoising are shown in Figure 1.

According to the characteristics of prostate MRI sequence images, a region-based threshold segmentation algorithm is used to segment prostate medical images. After selecting an appropriate threshold, the segmentation results of important physiological features in prostate medical images are shown in Figure 2. After image segmentation, the obvious characteristics of organs and tissues mainly include rectum, prostate, bladder, femur and pubic bone.

The surface rendering method of Marching Cubes algorithm is adopted to complete the rendering of 3D virtual model ^[36]. In this paper, 60 MRI sequence images of prostate with 1.5 mm interval are used as the original data of 3D virtual model reconstruction. After the image denoising and image segmentation, it is necessary to render the medical image to reconstruct the 3D virtual model, as shown in Figure 3.

The rendered medical image is imported into the 3DSlicer open-source platform, and the 3D virtual model reconstruction program is written by marching cubes algorithm, so as to reconstruct the 3D virtual model of prostate and its surrounding tissues and organs, as shown in Figure 4.

2.1.2.   Augmented reality scene construction

Augmented reality system is a system that simultaneously meets real-time interaction, virtual-real integration and three-dimensional display technology ^[37]. The architecture of augmented reality system for prostate seed implantation is completed from two aspects: Hardware and software. The main 3D display device of augmented reality system is Microsoft HoloLens 2 head-mounted display. The software system adopts unity, visual studio and Hololens 2 simulator.

The augmented reality scene is built in Unity. According to the actual needs, interactive setting, boundary setting, and spatial perception setting are made. Auxiliary orientation determination of augmented reality scene is shown in Figure 5. In order to provide better visualization and interactive control effect, the boundary and collision volume of the model are set, as shown in Figure 6.

In the augmented reality scene, doctors cannot only adjust the position of the three-dimensional virtual model in the visual field by changing the head and gaze direction, but also directly adjust the position of the model itself in the visual field through the preset interactive control mode, as shown in Figure 7. The interactive control methods such as gestures, gaze and voice can be used in simulators or hardware devices to manipulate the model.

2.2.   Three-dimensional registration technology

A total of 20 photos of checkerboard calibration board were taken by binocular camera at different angles, as shown in Figure 8. The photos obtained by left camera and right camera and the corresponding calibration error histograms were exported, as shown in Figure 9. Then, the camera calibration and distortion correction are realized using these 20 photos and the camera calibration toolbox of Matlab, and the internal parameters of the camera are solved.

A three-dimensional registration conversion relationship based on point-to-point data is established between the virtual world coordinate system and the real-world coordinate system. $\left\{{v}_{i}\right\}$ and $\left\{{r}_{i}\right\}$ are the coordinate point sets of the patient's body surface markers in the virtual world coordinate system and the real-world coordinate system. i is the number of markers. Four-dimensional homogeneous matrix ${T}_{v}^{r}$ is the transformation matrix between the two coordinate point sets. The conversion relationship between two coordinate point sets can be expressed by Eq (1).

where ${R}^{'}$ is the rotation matrix and ${T}^{'}$ is the translation matrix. The solved ${R}^{'}$ and ${T}^{'}$ should satisfy the least square Eq (2).

It is necessary to solve the optimal solution of ${R}^{'}$ and ${T}^{'}$ satisfying the least square Eq (2). An improved SVD algorithm based on ICP is proposed to optimize the 3D registration technology. The solution process is as follows.

1). Solve centroids ${\bar v}$ and ${\bar r}$ of point sets $\left\{{v}_{i}\right\}$ and $\left\{{r}_{i}\right\}$:

2). Establish new point sets $\left\{{v}_{ci}\right\}$ and $\left\{{r}_{ci}\right\}$ with centroid as coordinate origin:

3). Substitute new point sets $\left\{{v}_{ci}\right\}$ and $\left\{{r}_{ci}\right\}$ to optimize the least square Eq (2):

4). Solve the rotation matrix ${R}^{'}$:

According to the Eq (7), when the value of the last item $2{{r}_{ci}}^{T}{R}^{'}{v}_{ci}$ is larger, the value of E will be smaller, that is, the condition $\mathit{max}\left({R}^{'}H\right)$ needs to be met. H can be expressed by Eq (8).

By singular value decomposition of H, unitary matrix U, semi-positive definite diagonal matrix $\varLambda$ and conjugate transposed unitary matrix ${V}^{T}$ can be obtained.

The rotation matrix ${R}^{'}$ satisfying condition $\mathit{max}\left({R}^{'}H\right)$ can be expressed by Eq (10).

5). Solve the translation matrix ${T}^{'}$:

After calculating with SVD algorithm, the rotation matrix ${R}^{'}$ and translation matrix ${T}^{'}$ satisfying the least square Eq (2) can be obtained. Because there is no iterative optimization, the accuracy of the results cannot be guaranteed. Therefore, it is necessary to make accuracy judgment and iterative optimization on the results calculated by SVD algorithm, so as to obtain the optimal solution. The iteration conditions are set as iteration times ${N}_{max}$ and iteration accuracy $\varepsilon$.

6). The rotation matrix ${R}^{'}$ and translation matrix ${T}^{'}$ obtained by SVD algorithm are substituted into Eq (1), and the calculation result is set as a new virtual coordinate point set $\left\{{v}_{i}\right\}$.

7). The maximum distance between the real coordinate point set $\left\{{r}_{i}\right\}$ and the new virtual coordinate point set $\left\{{v}_{i}\right\}$ is solved, as shown in Eq (12).

8). If the calculation results do not meet the iterative conditions, the SVD algorithm needs to be used for re-calculation, and the optimal solution will be obtained according to the iterative process. If the calculation results meet the iterative conditions, the calculated rotation matrix ${R}^{'}$ and translation matrix ${T}^{'}$ are the optimal solutions.

2.3.   Voice control, optimization and denoising

2.3.1.   Control

Prostate seed implantation surgery usually includes multiple implantation paths and multiple implantation targets. In order to observe more intuitively and reduce interference information, the displayed augmented reality information is set as a single implantation target in a single implantation path. Doctors can switch the display information of the implantation path and the implantation target in real time through voice control instructions.

Microsoft HoloLens 2 is equipped with a microphone to receive voice control instructions, and the MRTK toolkit provides the software development kit (SDK) for voice control. The "MRTK Project Configurator" is set, the microphone function is enabled, and then voice control instructions are added to the keyword list in the inspector window, as shown in Figure 10(a).

Seed implantation path and seed implantation target components are added to the 3D virtual model. Blank items are created in the hierarchical structure window, and the voice input handler component and keyword elements are added to the blank items, as shown in Figure 10(b). By issuing the voice control instruction, the specific implantation path and target information can be displayed or hidden in the three-dimensional virtual model of the prostate. Voice control instructions contain specific meanings for doctors, not simple sequential voice control instructions.

The voice control instruction contains the information of seed implantation path and seed implantation target. In the operation of prostate seed implantation, the doctor will use the guide plate to assist the needle insertion. By numbering the pinholes distributed in the guide plate, the two-dimensional spatial positions of the prostate are described, as shown in Figure 11(a). The first instruction of voice control instruction represents the horizontal coordinate of prostate, and the second instruction represents the vertical coordinate of prostate. The depth information is added to describe the distribution position of seed targets. Targets distributed on the seed implantation path can be referred to by numbers 1, 2, and 3, as shown in Figure 11(b). The voice control instructions of the seed implantation can be set by instructions such as AA2, BC7, etc.

2.3.2.   Optimization of voice control instruction based on 3D registration algorithm

The robot motion control instructions planned before operation can only guarantee the position relationship between the implantation path and the implantation target, but cannot guarantee the position relationship between the patient and the robot. Therefore, the set positional relationship needs a series of conversion operations to make the actual motion of the robot under voice control meet the intraoperative requirements.

As shown in Figure 12, the doctor can preliminarily set the distribution information of the implantation path and the implantation target in the robot base coordinate system before operation, and the coordinate relationship between the path and the target can be represented by the virtual world coordinate $\left\{{v}_{i}\right\}$. During the operation, the doctor controls the robot in teaching mode so that the end of the implantation needle touches the markers attached to the abdominal surface of the patient. According to the information feedback result, the computer can obtain the coordinates of real markers in the robot base coordinate system, and use the real-world coordinates $\left\{{r}_{i}\right\}$ to represent them.

The transformation matrix ${T}_{v}^{r}$ from $\left\{{r}_{i}\right\}$ to $\left\{{v}_{i}\right\}$ is solved based on $\left\{{v}_{i}\right\}$. The motion control parameters of the robot are updated by this matrix, which can ensure that the information from preoperative planning to actual control during surgery is consistent. The process of solving the transformation matrix is actually a three-dimensional registration process.

During the operation, the patient's breathing and slight trembling will reduce the accuracy of the operation. These influencing factors are microscopic changes, and mainly reflected in the change of the positional relationship between the patient and the robot in the translation direction. The augmented reality visualization device will update the transformation matrix ${T}_{v}^{r}$ between the virtual world coordinate system and the real-world coordinate system in real time through sensors. The translation parameters in the transformation matrix ${T}_{v}^{r}$ can be extracted in real time during the operation, and the positional relationship between the patient and the robot in the translation direction and the motion control parameters can be continuously corrected.

The motion accuracy of UR robot can reach 0.1 mm, while the registration accuracy of 3D registration algorithm is lower than 3 mm. Therefore, through the 3D registration technology, the corrected theoretical accuracy error is within an acceptable range, and the non-real-time problems caused by voice control can be initially solved.

2.3.3.   Voice control instruction denoising

In the voice control system of prostate seed implantation, the collected voice signals are complex and contain various noises. Spectral subtraction is often used to eliminate broadband noise, which cannot meet actual needs. If the BP neural network is used to denoise complex time-varying noisy voice, the denoise result cannot guarantee convergence or fall into local optimum due to the imperfection of the training model. Based on the above problems and the actual demand of voice control system, a fusion algorithm based on spectral subtraction and BP neural network is proposed.

After the signal preprocessing and spectral subtraction denoising process are completed, the time domain signal is a noisy signal because it contains the music noise generated by spectral subtraction denoising. The BP neural network is used for secondary processing to remove music noise. For periodic noise in the system, the low-pass filter is used for hardware denoising in anti-aliasing filtering during preprocessing. In the subsequent denoising process, the periodic noise is further removed by spectral subtraction and the BP neural network, to obtain the final denoised signal. The denoising process is as follows.

1). The preprocessed voice signal is processed by three operations: Noise estimation, amplitude quadratic solution, phase angle calculation, and retention. The average energy estimation of the leading voiceless segment (noisy segment) of the voice signal is shown in Eq (13).

where $E\left(k\right)$ is the average energy, $fnT$ is the number of frames corresponding to the preamble voiceless segment, ${y}_{i}\left(k\right)$ is the preprocessed voice signal, and ${\left|{y}_{i}\left(k\right)\right|}^{2}$ is the amplitude quadratic solution of the voice signal. The phase angle of voice signal is calculated as shown in Eq (14).

where ${y}_{i}^{a}\left(k\right)$ is the phase angle of the voice signal. $Im{y}_{i}\left(k\right)$ is the imaginary part of complex number of ${y}_{i}\left(k\right)$, and $Re{y}_{i}\left(k\right)$ is the real part of complex number of ${y}_{i}\left(k\right)$.

2). As shown in Eq (15), the average energy estimated value $E\left(k\right)$ and the amplitude quadratic solution ${\left|{y}_{i}\left(k\right)\right|}^{2}$ are processed by spectral subtraction, so as to obtain the amplitude quadratic solution of the denoised voice signal.

where ${\left|{y}_{i}^{'}\left(k\right)\right|}^{2}$ represents the amplitude quadratic value of the denoised signal. Over-reduction factor a and gain compensation factor b can be used to reduce the amplitude of broadband peak, and remove part of broadband noise. The setting range of over-reduction factor a should satisfy $a\ge 1$, and the setting range of gain compensation factor b should satisfy $0 < b\le 1$.

3). Finally, combining the result of Eq (16) with the inverse fast Fourier transform (IFFT) technique, the voice signal in time domain after spectral subtraction denoising is obtained.

In denoising, the error between the denoised voice and the learning model is calculated. The optimal voice signal denoising is completed by combining the iterative adjustment of weight parameters and offset values. Based on the characteristics and requirements of voice denoising, the structure of the BP neural network model is shown in Figure 13.

The BP neural network model structure shown in Figure 13 includes input layer, hidden layer and output layer. The hidden layer consists of hidden layer neurons, external offset value, summation processing and training function. There are ten neurons in the hidden layer from H1 to H10, and the external offset value b is 1 by default. The trainglm function is used as the training function f. The maximum training number is 600, the error accuracy is 0.001, the momentum parameter is 0.8, and the learning rate is 0.002.

According to the above BP neural network model structure and parameter settings, the voice signal denoising program is written. The noisy voice signal is input into the BP neural network algorithm program. According to the preset parameters of the program and the learned model, the voice signal is denoised to obtain the denoised digital signal in the time domain.

Structure of the BP neural network model is shown in Figure 13.

3.   Results

3.1.   Three-dimensional registration experiment

3.1.1.   Accuracy experiment of three-dimensional registration algorithm

In the augmented reality system, the spatial coordinate error of the point cloud data is within $\text{30}\times \text{30}\times \text{30 m}{\text{m}}^{3}$, considering the displacement of marker points caused by a patients' breathing. A three-dimensional point cloud set P containing fifty data points is randomly generated, and a homogeneous matrix T for spatial transformation is set. Combining P and T, the new three-dimensional point cloud set P_i is obtained. The data of three-dimensional point cloud set P will fluctuate randomly within $\text{30}\times \text{30}\times \text{30 m}{\text{m}}^{3}$, and the SVD algorithm and improved algorithm are used to solve the three-dimensional registration matrices T₁ and T₂ of P and P_i, respectively. Using three-dimensional registration matrices T₁, T₂, and P_i, new three-dimensional point cloud sets Q₁ and Q₂ are obtained, and the Euclidean distances between Q₁ and P and between Q₂ and P are calculated. The experimental results are shown in Figure 14(a)–(c).

3.1.2.   Real-time experiment of three-dimensional registration algorithm

After improving the 3D registration algorithm, the 3D registration process has changed, so the 3D registration delay time is different under different algorithms. The real-time experiments of SVD algorithm and improved algorithm are carried out, and ten groups of time data under each algorithm are collected. Among them, the iterative condition of the improved algorithm is set as the minimum value to ensure its convergence, and the numerical value is accurate to one decimal place. Therefore, the set iteration condition is that the maximum distance between the corresponding points is less than 0.5 mm. The experimental results are shown in Figure 14(d).

3.1.3.   Repetitive experiment of three-dimensional registration algorithm

As shown in Figure 15, the markers are pasted on the surface of the mannequin, and then three groups of three-dimensional registration experiments are conducted for the SVD algorithm and improved algorithm. In the experiment, the z-axis data in the depth direction are all replaced by zero.

The position data of the markers need to be divided into ideal data and actual data. The ideal data are equivalent to the position of the three-dimensional virtual model in the virtual world coordinate system. The actual data are determined by obtaining the pixel information of the markers in the image. There will be certain errors between the actual data and the ideal data due to the actual pasting position, shooting distortion and coordinate information extraction method. To reduce the influence of errors on the experimental results, aiming at the same group of ideal data, five groups of actual data were collected for repeated experiments. Then, the SVD algorithm and improved algorithm are used to calculate the average error of each group of actual data. There are three groups of ideal data corresponding to six, nine and eleven markers. The experimental results are shown in Table 1.

3.2.   Voice experiments

3.2.1.   Voice signal-to-noise ratio experiment

The pure voice signal was recorded in a quiet environment. Ten segments of voice signals were recorded as noisy voice signals in the actual environment of robot movement. The noisy voice signals are denoised by spectral subtraction, BP neural network and fusion algorithm, so as to obtain ten groups of denoised voice signals. The signal-to-noise ratio of noisy voice signals and denoised voice signals is shown in Figure 16(a).

3.2.2.   Denoising delay experiment

The delay experiment of the recorded noisy voice signal is carried out. Quantitative analysis of the voice denoising delay using different denoising algorithms is made, as shown in Figure 16(b). Figure 16(b) shows that the denoising delay of spectral subtraction is the lowest, and that of BP neural network is the highest. The average delay of spectral subtraction is 0.814 s, that of the BP neural network is 2.368 s, and that of the fusion algorithm is 1.314 s.

3.2.3.   Voice denoising waveform diagram experiment

The voice signal waveform diagram experiment was carried out to observe the distribution and distortion of the waveform. Figure 17 shows the waveform comparison diagrams. Figure 18 shows the voice denoising waveform diagram based on fusion algorithm.

3.2.4.   Voice recognition experiment

To evaluate the fusion algorithm in voice denoising more comprehensively, the voice recognition experiment was carried out based on the experimental platform. The experimental platform is shown in Figure 19. The computer is connected to the robot control cabinet. The seed implantation device is installed at the end of the robot, and keeps communication with the computer through the motion control card. The agar is used to imitate human tissues. According to the preset control scheme, five groups of voice control instructions are selected, and the robot motion control program corresponding to the instructions is written. In the experiment, the robot voice control is carried out under the conditions of original voice, spectral subtraction denoising, BP neural network denoising and fusion algorithm denoising. Each group of voice control instructions should be subjected to 40 groups of experiments under different voice processing conditions, and then the voice recognition degree under different processing conditions can be obtained by calculating the proportion that the robot can complete the corresponding actions according to the voice control instructions, as shown in Table 2.

3.2.5.   Response speed experiment of integrated system

The augmented reality application saved on the computer is opened by the HoloLens 2 simulator to complete the response speed experiment of the integrated system. Five different voice control instructions were sent in turn. The time from voice control instruction to Hololens 2 simulator and UR5e robot is recorded, and five pairs of time data are obtained, as shown in Table 3.

4.   Discussion

To verify the effectiveness of the improved 3D registration algorithm, accuracy experiments, real-time experiments, and repetitive experiments are carried out. During the simulation process, due to the error accumulation effect of the algorithm, the calculation result of one point cloud data among the 50 registered point cloud data has a big deviation. Figure 14(a), (b) shows that the deviation does not affect the overall three-dimensional registration effect. Therefore, the data can be removed as a singular value in the subsequent Euclidean distance calculation. After removing singular values, the results in Figure 14(c) shows that the overall accuracy of the improved algorithm is better than that of the SVD algorithm, and the data fluctuation of the improved algorithm is more stable than that of the SVD algorithm. Figure 14(d) shows that the real-time performance of the SVD algorithm is lower than that of the improved algorithm when processing actual data. Therefore, in practical applications, the real-time performance of the improved algorithm in the system is better than that of the SVD algorithm. The data in Table 1 show that the average error of the improved algorithm is smaller than that of the SVD algorithm, as shown in Figure 20. Compared with the accuracy experiment, the average error has increased, which is mainly due to the introduction of errors. However, the improved algorithm is better than the SVD algorithm in 3D registration.

As shown in Figure 21, the more labeled points are pasted, the smaller the average error of data is. The comparison diagram before and after three-dimensional registration based on improved algorithm for 11 markers is shown in Figure 15(b). Experimental results show that the improved algorithm can meet the requirements of augmented reality system for visualization accuracy. Therefore, considering the above factors and clinical needs comprehensively, the improved algorithm can meet the visualization accuracy requirements of augmented reality system for prostate seed implantation. Three-dimensional registration technology is used to optimize the non-real-time problems in the voice control system to ensure the accuracy of prostate seed implantation. The signal-to-noise ratio curves shown in Figure 16(a) indicate that the denoising effect of spectral subtraction fluctuates greatly. The denoising effect of the BP neural network and fusion algorithm is stable. Considering the difference in the signal-to-noise ratio between the data, the denoising effect of the fusion algorithm is approximately twice that of the spectral subtraction and three times that of the BP neural network. Figure 16(b) shows that the average delay of spectral subtraction is 0.814 s, that of the BP neural network is 2.368 s, and that of the fusion algorithm is 1.314 s. To ensure the denoising effect of the BP neural network, it is necessary to set more iterations and higher learning accuracy; thus, it takes more time to learn and approximate the model. The fusion algorithm combines the advantages of spectral subtraction and BP neural network. The fusion algorithm is less affected by the number of iterations and learning accuracy, so the delay of the fusion algorithm is lower than that of the BP neural network. However, the denoising process of fusion algorithm is more complicated than that of spectral subtraction, so the delay of the fusion algorithm is greater than that of spectral subtraction. By comparing the voice waveforms before and after denoising in Figures 17 and 18, it can be seen that the denoising effect of spectral subtraction is better than that of the BP neural network. However, compared with the pure voice waveform, there is music noise, and the signal distortion is large. Compared with fusion algorithm, the music noise generated by the fusion algorithm is smoother, the noise amplitude is reduced, and the distortion of voice signals is smaller. By analyzing the experimental results of the voice recognition degree shown in Table 2, we can know that the voice recognition degree of the system is the highest under the voice denoising of the fusion algorithm, as shown in Figure 22. Therefore, the experiment also proves that the use of the fusion algorithm optimizes and enhances the voice denoising function of the integrated system. By analyzing the data in Table 3, the response time of the augmented reality system is approximately 0.25 s, and that of robot control system is approximately 1.5 s. We mainly consider the response time of the robot system. Therefore, the overall response time of the integrated system is approximately 1.5 s. The experimental results show that the fusion algorithm can effectively improve the reliability of the voice control system and ensure the control effect during the process of system operation. The response speed of the system can meet the requirements of prostate seed implantation surgery.

5.   Conclusions

In this paper, we propose a prostate seed implantation robot system based on AR and voice control. The median filtering algorithm is used to denoise prostate MR images, the local threshold segmentation method is used to segment the features of medical images, and the surface rendering technology is used to reconstruct the three-dimensional virtual model of the prostate and its surrounding organs and tissues. Combined with holographic application, the augmented reality system and scene display of prostate seed implantation are built. According to the three-dimensional registration process, an improved SVD algorithm based on ICP is proposed, and the effectiveness of the improved algorithm is verified by accuracy experiment, real-time experiment and repetitive experiment. The voice control system is built for the prostate seed implantation robot. The non-real-time problem in voice control is analyzed, and the control process is optimized in combination with three-dimensional registration technology. A fusion algorithm based on spectral subtraction and BP neural network is proposed for voice denoising. The reliability of the fusion algorithm is verified through voice signal-to-noise ratio experiment, denoising delay experiment, voice denoising waveform diagram experiment, and voice recognition experiment. Finally, the response speed experiment of the integrated system shows that the overall response time of the system is approximately 1.5 s. The experiments prove that the responsiveness of the voice control system of the prostate seed implantation robot in the augmented reality environment can meet the surgical requirements.

According to the needs of surgery, augmented reality and voice control technology are used to provide a more friendly visualization system and control system for prostate seed implantation surgery. The system not only ensures the robot motion control based on voice commands, but also provides the corresponding augmented reality visualization information for doctors. The combination of augmented reality and voice control provides more convenient and natural non-contact control for the surgical process, and greatly improves the intelligence, visualization and information of the surgical process. The limitation of this paper is that the system and method have only carried out relevant experiments and tests, and have not really been used in clinic. The next step is to further improve the system functions and conduct clinical trials. In the future, we will continue to conduct in-depth research on the following aspects.

1). The reconstruction method of the 3D virtual model should be further optimized to improve the reconstruction accuracy. Appropriate image processing methods are adopted for human soft tissues to solve the problem of unclear feature boundary transition and improve the accuracy of feature extraction.

2). The real-time registration speed of the 3D registration algorithm should be further optimized. The rigid registration process of the 3D registration algorithm should be improved to solve the problem of soft tissue deformation and improve the accuracy and real-time performance of the 3D registration algorithm.

3). More voice commands and visualization functions will be developed, so that the prostate seed implantation robot system can have better human-computer interaction ability, and is more intelligent and more multimodal.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Author contributions

Xinran Zhang: Formal analysis, Software, Validation, Writing–original draft, Writing–review & editing; Yongde Zhang: Conceptualization, Method-ology; Jianzhi Yang: Formal analysis, Software; Haiyan Du: Writing – review & editing.

Acknowledgments

This research was funded by the National Natural Science Foundation of China, grant number 52275015, the Reserve Leader Funding Project of Leading Talent Echelon of Heilongjiang Province of China, grant number 2501050628, and the Science and Technology Innovation Team Project of Foshan City of China, grant number 2018IT100302.

Conflict of interest

The authors declare that there are no conflicts of interest.