- Research
- Open access
- Published:
Multi-mode information fusion navigation system for robot-assisted vascular interventional surgery
BMC Surgery volume 23, Article number: 51 (2023)
Abstract
Background
Minimally invasive vascular intervention (MIVI) is a powerful technique for the treatment of cardiovascular diseases, such as abdominal aortic aneurysm (AAA), thoracic aortic aneurysm (TAA) and aortic dissection (AD). Navigation of traditional MIVI surgery mainly relies only on 2D digital subtraction angiography (DSA) images, which is hard to observe the 3D morphology of blood vessels and position the interventional instruments. The multi-mode information fusion navigation system (MIFNS) proposed in this paper combines preoperative CT images and intraoperative DSA images together to increase the visualization information during operations.
Results
The main functions of MIFNS were evaluated by real clinical data and a vascular model. The registration accuracy of preoperative CTA images and intraoperative DSA images were less than 1 mm. The positioning accuracy of surgical instruments was quantitatively assessed using a vascular model and was also less than 1 mm. Real clinical data used to assess the navigation results of MIFNS on AAA, TAA and AD.
Conclusions
A comprehensive and effective navigation system was developed to facilitate the operation of surgeon during MIVI. The registration accuracy and positioning accuracy of the proposed navigation system were both less than 1 mm, which met the accuracy requirements of robot assisted MIVI.
Background
Cardiovascular diseases are the severe causes of death globally and the number of cases suffering from these diseases is on the rise. Minimally invasive vascular intervention (MIVI), which is safe and easy for recovery, has been the state-of-the-art therapy for cardiovascular and cerebrovascular diseases. In recent years, the number of robots used in clinical surgery has increased dramatically [1, 2]. Vascular interventional surgery robots are also being developed by many institutions [3, 4].
At present, most of the navigation systems used in normal or robotic assisted MIVI are 2D navigation systems, which lack positioning informatics of the intervention instruments and 3D structure of vessels. Vascular interventional surgical robots, such as CorPath, Amigo and Niobe, all employ preoperative CTA for diagnosis and surgical planning, and use DSA images for intraoperative navigation [5,6,7]. Enhancing interventional visualization by integrating multi-mode intraoperative and preoperative information into a comprehensive system is of great importance of MIVI navigation [8]. Traditional vascular interventional surgery is usually guided only by DSA images [9,10,11], which can display the two-dimensional structure and morphology of blood vessels in real time [12]. However, patients and doctors need to be constantly exposed to the X-rays during the DSA process. Besides, the vascular structure is partially displayed in each frame with blurred boundaries and inhomogeneous distribution of contrast agent [13].
3D reconstruction of preoperative computed tomography angiograph (CTA) slices offers surgeon 3D vascular structures, while segmentation of intraoperative DSA sequences integrates vascular information in multi-frame images to provide doctors with more 2D vascular information [14,15,16]. 2D-3D vascular image registration combines the advantages of both preoperative CTA slices and intraoperative DSA sequences to increase visual information and reduce the use of X-rays during surgery [17,18,19,20,21,22,23]. Electromagnetic system (EM) as a new accurate and radiation-free positioning method has also been gradually used in the positioning of the end of interventional devices [24,25,26,27].
This study presented a navigation system combining both preoperative and intraoperative multi-mode information of MIVI. The accuracy of the navigation system has also been evaluated using clinical CTA and DSA images or a vascular model.
Method
Functions of the robot assisted MIVI navigation system presented in this study, as shown in Figs. 1 and 2, include three main parts: preoperative 3D vascular reconstruction of CT slices, registration of preoperative 3D vascular model and intraoperative 2D DSA images, and intraoperative interventional instrument positioning.
3D vascular model reconstruction can be divided into three main steps: vascular segmentation from CT slices, erasing mis-segmented tissue and 3D vascular reconstruction [28]. Since the diameter of human blood vessels is difficult to quantify, the accuracy of 3D reconstruction is verified by a blood vascular model. 2D-3D registration of preoperative 3D reconstructed model and 2D intraoperative DSA images aims to find the optimal spatial transformation from a 3D model to 2D DSA images [22, 23]. The quantitative analysis of 2D-3D registration was quantified by the vascular model, while the qualitative analysis was evaluated by clinical CT and DSA images. Three evaluation criteria, mean target registration error (mTRE), mean absolute error (MAE) and dice coefficient are used in this paper to evaluate the registration result. mTRE was the mean error between registration points and target points. MAE represented the mean error between the registered and the groudtruth of the transformation parameters, mDice measured the fit between two registered images.
EM was used to measure the position of surgical instruments, and its positioning accuracy was evaluated by measuring the root mean square error (RMS) between the spatial coordinates and the EM coordinates of twenty-six catheter end positions selected on each path.
Result
3D vascular model reconstruction
Figure 3a shows the process and results of 3D vascular reconstruction. Since the vessel diameter contained in CTA is difficult to measure due to the influence of contrast agent and blood flow, the vessel model as shown in Fig. 3b with known shape parameters is used to facilitate quantification of 3D vascular reconstruction.
The mean absolute errors (MAE) between diameters of the reconstructed model and the real model were reliable indicators to access the reconstruction accuracy. MAEs of branches A ~ F with varied diameters as shown in Fig. 3b were calculated separately and the results were shown in the following Table 1. D in Table 1 means the diameters of these branches. The MAEs demonstrated that the average reconstruction errors were all less than 1 mm, which meets the requirements for robot assisted MIVI.
2D-3D image registration
Figure 4 demonstrates the process of 2D-3D registration. The registration results directly affect the accuracy of surgical map construction. The accuracy of 2D-3D vascular image registration was evaluated by both the vascular model and clinical images. Vascular model was used to quantify the registration effect, and clinical data were used to verify the clinical application prospect of the proposed navigation system.
As shown in Table 2, the MAEs of three transformation parameters (T) were all less than 1 mm, and the MAEs of three rotation parameters (R) were smaller than or approach 0.5°. The mTREs were within the range of (0.1,0.2) that satisfied the requirement of clinical experiments. There were nine patients covered in the clinical experiments including three each with abdominal aortic aneurysm (AAA), thoracic aortic aneurysm (TAA) and aortic dissection (AD). Clinical data lacked ground truth registration result values, so mDice values as shown in Table 3 were used to measure the registration results. As shown in Table 3, mDice values of the 9 patients were all around 0.5, and there was no significant difference between the registration results of the three kinds of diseases. The main reason for the low mDice value in the registration results of clinical CTA image and DSA image is that the vascular region segmented in DSA image is affected by uneven flow of contrast agent, resulting in partial loss of blood vessels. The difference in the range of vessels covered by the two mode images leads to the low accuracy of the final registration results.
Surgical path planning and navigation accuracy
The movement paths of the catheter from the starting point to positions 1, 2, 3 and 4 were conducted by the MIVI robot as shown in Figs. 5 and 6. The positioning accuracy of EM for the interventional device was evaluated by measuring the root mean square error (RMS) between the spatial coordinates and the EM coordinates of twenty-six catheter end positions selected on each path.
As shown in Fig. 7, the mean values of RMS were all less than 1 mm, and the maximum values were all smaller than 2 mm. The experiment results verified that the positioning accuracy of EM satisfied the requirements of clinical operations.
Discussion
The navigation of robot assisted MIVI were successfully completed on the vascular model with high accuracy and less contrast injection. The 2D-3D image registration combining both the advantages of CTA and DSA, makes it possible to simultaneously visualize the preoperative and intraoperative images. Accurate 3D reconstruction of blood vessels and positioning of surgical instruments ensure the operation accuracy of robot-assisted surgery and improve the safety of surgery.
First of all, 3D vascular models containing sufficient vascular shape and structure information were reconstructed from preoperative CTA slices, which were the foundation of three-dimensional navigation. The accuracy of vascular 3D reconstruction was quantified by vascular model to verify that it satisfied the clinical requirements. Meanwhile, 2D-3D vascular image registration combined the advantages of preoperative 3D vascular model and intraoperative DSA images to increase the real-time performance of the navigation system. The registration result offers surgeon a global vascular map which only needs a clear DSA sequence during operation. The reduction of the demand for DSA sequence decreases the use of contrast agent during the operation. Although the performance of 2D-3D registration on clinical data was not as satisfied as that on vascular models, it provided doctors with a static but intuitive map of vessels. Moreover, surgical path planning helped doctors or robotics select the optimal surgical route to shorten the surgical time and improve the surgical safety. Finally, the EM system was used to track and locate the surgical instruments by integrating the multi-mode information of the navigation system. The average positioning accuracy was less than 1 mm, which evaluated the accuracy and clinical application prospect of the navigation system.
Conclusion
The multi-mode information fusion navigation system proposed in this paper for TIVI combined multi-modal medical information during both preoperative and intraoperative operation to provide comprehensive and intuitive visual navigation for doctors or robots. The main functions of the navigation system were evaluated by sufficient experiments on clinical data and vascular models to verify that they satisfied the clinical applications. The navigation system also needs further improvements to provide more accurate navigation information for doctors and robots. For example, the clinical application of the navigation system is only partially verified in this paper. In future studies, more clinical animal and human experiments will be covered in our research to verify the function and accuracy of the navigation system.
Availability of data and materials
The datasets analyzed during the current study are not publicly available due to respect the participants’ rights to privacy and to protect their identity, but are available from Shaoya Guan on reasonable request.
References
Nakadate R, Iwasa T, Onogi S, Arata J, Oguri S, Okamoto Y, Akahoshi T, Eto M, Hashizume M. Surgical robot for intraluminal access: an ex vivo feasibility study. Cyborg Bionic Syst. 2020. https://doi.org/10.34133/2020/8378025.
Kaukemueller L, Klapdor R, Roettger M, et al. Establishment of the DaVinci surgical robot system in the surgical Gynecology. Geburtshilfe Frauenheilkd. 2020;10:80.
Lo N, Gutierrez JA, Swaminathan RV. Robotic-assisted percutaneous coronary intervention. Curr Treat Options Cardiovasc Med. 2018;20(2):14.
Maor E, Eleid MF, Gulati R, et al. Current and future use of robotic devices to perform percutaneous coronary interventions: a review. J Am Heart Assoc. 2017;6(7):8.
Granada J, Delgado J, Uribe M, et al. First-in-human evaluation of a novel robotic-assisted coronary angioplasty system. JACC Cardiovasc Interven. 2011;4(4):460–5.
Datino T, Arenal A, Pelliza M, et al. Comparison of the safety and feasibility of arrhythmia ablation using the amigo robotic remote catheter system versus manual ablation. Am J Cardiol. 2014;113(5):827–31.
Antoniou G, Riga C, Mayer E, et al. Clinical applications of robotic technology in vascular and endovascular surgery. J Vasc Surg. 2011;53(2):493–9.
Cleary K, Peters TM. Image-guided interventions: technology review and clinical applications. Annu Rev Biomed Eng. 2010;12:119–42.
Auffermann WF, Krupinski EA, Tridandapani S. Search pattern training for evaluation of central venous catheter positioning on chest radiographs. J Med Imaging. 2018;5(3): 031407.
Khan EM, Frumkin W, Ng GA, et al. First experience with a novel robotic remote catheter system: Amigo™ mapping trial. J Interv Card Electrophysiol. 2013;37(2):121–9.
Yi X, Adams S, Babyn P, et al. Automatic catheter and tube detection in pediatric x-ray images using a scale-recurrent network and synthetic data. J Digit Imaging. 2020;33(1):181–90.
Markelj P, Tomaevi D, Likar B, et al. A review of 3D/2D registration methods for image-guided interventions. Med Image Anal. 2012;16(3):642–61.
Sun W, He Y, Ge R, et al. Projection network with Spatio-temporal information: 2D+ time DSA to 2D aorta segmentation. Multimed Tools Appl. 2022; 1–15.
Meng C, Sun K, Guan S, et al. Multiscale dense convolutional neural network for DSA cerebrovascular segmentation. Neurocomputing. 2020;373:123–34.
Sulayman N, Al-Mawaldi M, Kanafani Q. Semi-automatic detection and segmentation algorithm of saccular aneurysms in 2D cerebral DSA images. Egypt J Radiol Nucl Med. 2016;47(3):859–65.
Meng C, Xu Y, Li N, et al. Incremental robust PCA for vessel segmentation in DSA sequences. Biomed Phys Eng Express. 2022;8(4): 045002.
Fan J, Cao X, Wang Q, et al. Adversarial learning for mono-or multi-modal registration. Med Image Anal. 2019;58: 101545.
Fechter T, Baltas D. One-shot learning for deformable medical image registration and periodic motion tracking. IEEE Trans Med Imaging. 2020;39(7):2506–17.
Cao X, Yang J, Zhang J, et al. Deformable image registration using a cue-aware deep regression network. IEEE Trans Biomed Eng. 2018;65(9):1900–11.
Miao S, Wang ZJ, Liao R. A CNN regression approach for real-time 2D/3D registration. IEEE Trans Med Imaging. 2016;35(5):1352–63.
Zheng J, Miao S, Wang ZJ, et al. Pairwise domain adaptation module for CNN-based 2-D/3-D registration. J Med Imaging. 2018;5(2): 021204.
Guan S, Wang T, Sun K, et al. Transfer learning for nonrigid 2D/3D cardiovascular images registration. IEEE J Biomed Health Inform. 2020;2020(29):3300–9.
Guan S, Meng C, Wang T, et al. Deformable cardiovascular image registration via multi-channel convolutional neural network. IEEE Access. 2019;7:17524–34.
Gullo G, Colin A, Frossard P, et al. Appropriateness of replacing fluoroscopic guidance with ECG-electromagnetic guidance for PICC insertion: a randomized controlled trial. Am J Roentgenol. 2021;216(4):981–8.
Maalouf J, Whiteside HL, Pillai A, et al. Reduction of radiation and contrast agent exposure in a cryoballoon ablation procedure with integration of electromagnetic mapping and intracardiac echocardiography: a single center experience. J Interv Card Electrophysiol. 2020;59(3):545–50.
Cavaliere M, McVeigh O, Jaeger HA, et al. Inductive sensor design for electromagnetic tracking in image guided interventions. IEEE Sens J. 2020;20(15):8623–30.
Schwein A, Kramer B, Chinnadurai P, et al. Electromagnetic tracking of flexible robotic catheters enables “assisted navigation” and brings automation to endovascular navigation in an in vitro study. J Vasc Surg. 2018;67(4):1274–81.
Lorensen WE, Cline HE. Marching cubes: a high resolution 3D surface construction algorithm. ACM Siggraph Comput Graph. 1987;21(4):163–9.
Acknowledgements
Thanks to Yi Ning and Xiaoyong Huang, Beijing Anzhen Hospital, Capital Medical University, for providing the clinical data.
Funding
This work was supported by the Cross-Disciplinary Science Foundation from Beijing Institute of Petrochemical Technology [Grant number BIPTCSF-016], National Innovation and Entrepreneurship Program for College Students of China [Grant numbers 2022J00037, 2022J00100], National Major Research Program of China [Grant number 2019YFB1311703] and the National Natural Science Foundation of China [Grant numbers 61533016, 61873010].
Author information
Authors and Affiliations
Contributions
SG wrote the main manuscript text. TL did the experiments in this paper with the help of SG and LM, CM and LM contributed significantly to analysis and manuscript preparation. All authors reviewed the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The study was approved by the Medical ethics committee of Beijing Anzhen Hospital Affiliated to Capital Medical University. All experiments were performed in accordance with the guidelines and regulations of “Medical ethics committee of Beijing Anzhen Hospital Affiliated to Capital Medical University”. The informed consent about the usage of medical images covered in the experiments of this paper was obtained of all subjects or their legal guardians.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Guan, S., Li, T., Meng, C. et al. Multi-mode information fusion navigation system for robot-assisted vascular interventional surgery. BMC Surg 23, 51 (2023). https://doi.org/10.1186/s12893-023-01944-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12893-023-01944-5