In recent years, minimally invasive surgery has developed rapidly in the clinical practice of surgery and has gradually become one of the critical surgical techniques. Compared with traditional surgery, the advantages of minimally invasive surgery include small incisions and less pain during the operation, and the patients recover faster after surgery. With the expansion of minimally invasive surgery in several medical fields, traditional minimally invasive techniques have bottlenecks in clinical practice, such as the inability of the endoscope to determine the depth information of the lesion area from the two-dimensional images obtained, the difficulty in locating the endoscopic position information and the inability to get a complete view of the overall situation in the cavity. This paper uses a visual simultaneous localization and mapping (SLAM) approach to achieve endoscope localization and reconstruction of the surgical region in a minimally invasive surgical environment. Firstly, the K-Means algorithm combined with the Super point algorithm is used to extract the feature information of the image in the lumen environment. Compared with Super points, the logarithm of successful matching points increased by 32.69%, the proportion of effective points increased by 25.28%, the error matching rate decreased by 0.64%, and the extraction time decreased by 1.98%. Then the iterative closest point method is used to estimate the position and attitude information of the endoscope. Finally, the disparity map is obtained by the stereo matching method, and the point cloud image of the surgical area is finally recovered.
Citation: Liwei Deng, Zhen Liu, Tao Zhang, Zhe Yan. Study of visual SLAM methods in minimally invasive surgery[J]. Mathematical Biosciences and Engineering, 2023, 20(3): 4388-4402. doi: 10.3934/mbe.2023203
In recent years, minimally invasive surgery has developed rapidly in the clinical practice of surgery and has gradually become one of the critical surgical techniques. Compared with traditional surgery, the advantages of minimally invasive surgery include small incisions and less pain during the operation, and the patients recover faster after surgery. With the expansion of minimally invasive surgery in several medical fields, traditional minimally invasive techniques have bottlenecks in clinical practice, such as the inability of the endoscope to determine the depth information of the lesion area from the two-dimensional images obtained, the difficulty in locating the endoscopic position information and the inability to get a complete view of the overall situation in the cavity. This paper uses a visual simultaneous localization and mapping (SLAM) approach to achieve endoscope localization and reconstruction of the surgical region in a minimally invasive surgical environment. Firstly, the K-Means algorithm combined with the Super point algorithm is used to extract the feature information of the image in the lumen environment. Compared with Super points, the logarithm of successful matching points increased by 32.69%, the proportion of effective points increased by 25.28%, the error matching rate decreased by 0.64%, and the extraction time decreased by 1.98%. Then the iterative closest point method is used to estimate the position and attitude information of the endoscope. Finally, the disparity map is obtained by the stereo matching method, and the point cloud image of the surgical area is finally recovered.
| [1] |
T. N. Robinson, G. V. Stiegmann, Minimally invasive surgery, Endoscopy, 36 (2004), 48–51. https://doi.org/10.1055/s-2004-814113 doi: 10.1055/s-2004-814113
|
| [2] | J. Perissat, Laparoscopic cholecystectomy, a treatment for gallstones: From idea to reality, World J. Surg., 23 (1999), 328–331. |
| [3] | J. Chen, Y. Guo, Observation and study on the therapeutic effect of endoscopy combined with minimally invasive abdominal surgery robot in the treatment of gallstones, in Proceedings of 2019 International Conference on Biology, Chemistry and Medical Engineering Francis, (2019), 79–84. |
| [4] | J. Xiao, Q. Wu, D. Sun, C. He, Y. Chen, Classifications and functions of vitreoretinal surgery assisted robots-a review of the state of the Art, in 2019 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS) IEEE, (2019), 474–484. https://doi.org/10.1109/ICITBS.2019.00122 |
| [5] |
C. Siristatidis, C. Chrelias, Feasibility of office hysteroscopy through the "see and treat technique" in private practice: A prospective observational study, Arch. Gynecol. Obstet., 283 (2011), 819–823. https://doi.org/10.1007/s00404-010-1431-3 doi: 10.1007/s00404-010-1431-3
|
| [6] | P. Cheeseman, R. Smith, M. Self, A stochastic map for uncertain spatial relationships, in 4th International Symposium on Robotic Research, (1987), 467–474. |
| [7] | P. Mountney, D. Stoyanov, A. Davison, G. Yang, Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery, in International Conference on Medical Image Computing and Computer-Assisted Intervention Springer, Berlin, Heidelberg, (2006), 347–354. https://doi.org/10.1007/11866565_43 |
| [8] |
G. Mattioli, V. Rossi, F. Palo, M. C. Y. Wong, P. Gandullia, S. Arrigo, et al., Minimal invasive approach to paediatric colorectal surgery, J. Ped. Endosc. Surg., 3 (2021), 129–139. https://doi.org/10.1007/s42804-020-00090-6 doi: 10.1007/s42804-020-00090-6
|
| [9] | P. Mountney, G. Z. Yang, Dynamic view expansion for minimally invasive surgery using simultaneous localization and mapping, in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE, (2009), 1184–1187. https://doi.org/10.1109/IEMBS.2009.5333939 |
| [10] | B. Lin, Visual SLAM and Surface Reconstruction for Abdominal Minimally Invasive Surgery, University of South Florida, 2015. |
| [11] | L. Chen, W.Tang, N. W. John, T. R. Wan, J. J. Zhang, Augmented reality for depth cues in monocular minimally invasive surgery, preprint, arXiv: 1703.01243. |
| [12] | A. Marmol, P. Corke, T. Pinot, ArthroSLAM: Multi-sensor robust visual localization for minimally invasive orthopedic surgery, in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IEEE, (2018), 3882–3889. https://doi.org/10.1109/IROS.2018.8593501 |
| [13] | C. Girerd, A. V. Kudryavtsev, P. Rougeot, P. Renaud, K. Rabenorosoa, B. Tamadazte, Automatic tip-steering of concentric tube robots in the trachea based on visual slam, in IEEE Transactions on Medical Robotics and Bionics, 2 (2020), 582–585. https://doi.org/10.1109/TMRB.2020.3034720 |
| [14] | I. Font, S. Weiland, M. Franken, M. Steinbuch, L. Rovers, Haptic feedback designs in teleoperation systems for minimal invasive surgery, in 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583), (2004), 2513–2518. https://doi.org/10.1109/ICSMC.2004.1400707 |
| [15] | S. Seung, B. Kang, H. Je, J. Park, K. Kim, S. Park, Tele-operation master-slave system for minimal invasive brain surgery, in 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO), (2009), 177–182. https://doi.org/10.1109/ROBIO.2009.5420619 |
| [16] | D. DeTone, T. Malisiewicz, A. Rabinovich, Superpoint: Self-supervised interest point detection and description, in Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition Workshops, (2018), 224–236. https://doi.org/10.1109/CVPRW.2018.00060 |
| [17] | E. Mühe, Laparoscopic cholecystectomy—late results, in Die Chirurgie und ihre Spezialgebiete Eine Symbiose, (1991), 416–423. https://doi.org/10.1007/978-3-642-95662-1_189 |
| [18] |
J. Gimenez, A. Amicarelli, J. M. Toibero, F. di Sciascio, R. Carelli, Iterated conditional modes to solve simultaneous localization and mapping in markov random fields context, Int. J. Autom. Comput., 15 (2018), 310–324. https://doi.org/10.1007/s11633-017-1109-4 doi: 10.1007/s11633-017-1109-4
|
| [19] | N. Ketkar, J. Moolayil, Convolutional neural networks, in Deep Learning with Python, Springer International Publishing, (2017), 197–242. https://doi.org/10.1007/978-1-4842-5364-9_6 |
| [20] | Z. Zhang, Flexible camera calibration by viewing a plane from unknown orientations, in Proceedings of the Seventh Ieee International Conference on Computer Vision, (1999), 666–673. |
| [21] |
D. G. Lowe, Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vision, 60 (2004), 91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94 doi: 10.1023/B:VISI.0000029664.99615.94
|
| [22] | T. Qin, P. Li, S. Shen, Vins-mono: a robust and versatile monocular visual-inertial state estimator, in IEEE Transactions on Robotics, 34 (2018), 1004–1020. https://doi.org/10.1109/TRO.2018.2853729 |
| [23] | O. Garcıa, J. Civera, A. Gueme, V. Munoz, J. M. M. Montiel, Real-time 3d modeling from endoscope image sequences, Adv. Sens. Sensor Integr. Med. Robot., (2009), 1–3. |
| [24] |
M. A. Fischler, R. C. Bolles, Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography, Commun. ACM, 24 (1981), 381–395. https://doi.org/10.1145/358669.358692 doi: 10.1145/358669.358692
|
| [25] | F. Wolfgang, B. P. Wrobel, Bundle adjustment, in Photogrammetric Computer Vision, (2016), 643–725. https://doi.org/10.1007/978-3-319-11550-4_15 |
Figures(9) / Tables(1)
Liwei Deng, Zhen Liu, Tao Zhang, Zhe Yan. Study of visual SLAM methods in minimally invasive surgery[J]. Mathematical Biosciences and Engineering, 2023, 20(3): 4388-4402. doi: 10.3934/mbe.2023203
DownLoad: