Skip to main content
Download PDF

Multi-mode information fusion navigation system for robot-assisted vascular interventional surgery

BMC Surgery volume 23, Article number: 51 (2023) Cite this article

Abstract

Background

Minimally invasive vascular intervention (MIVI) is a powerful technique for the treatment of cardiovascular diseases, such as abdominal aortic aneurysm (AAA), thoracic aortic aneurysm (TAA) and aortic dissection (AD). Navigation of traditional MIVI surgery mainly relies only on 2D digital subtraction angiography (DSA) images, which is hard to observe the 3D morphology of blood vessels and position the interventional instruments. The multi-mode information fusion navigation system (MIFNS) proposed in this paper combines preoperative CT images and intraoperative DSA images together to increase the visualization information during operations.

Results

The main functions of MIFNS were evaluated by real clinical data and a vascular model. The registration accuracy of preoperative CTA images and intraoperative DSA images were less than 1 mm. The positioning accuracy of surgical instruments was quantitatively assessed using a vascular model and was also less than 1 mm. Real clinical data used to assess the navigation results of MIFNS on AAA, TAA and AD.

Conclusions

A comprehensive and effective navigation system was developed to facilitate the operation of surgeon during MIVI. The registration accuracy and positioning accuracy of the proposed navigation system were both less than 1 mm, which met the accuracy requirements of robot assisted MIVI.

Peer Review reports

Background

Cardiovascular diseases are the severe causes of death globally and the number of cases suffering from these diseases is on the rise. Minimally invasive vascular intervention (MIVI), which is safe and easy for recovery, has been the state-of-the-art therapy for cardiovascular and cerebrovascular diseases. In recent years, the number of robots used in clinical surgery has increased dramatically [1, 2]. Vascular interventional surgery robots are also being developed by many institutions [3, 4].

At present, most of the navigation systems used in normal or robotic assisted MIVI are 2D navigation systems, which lack positioning informatics of the intervention instruments and 3D structure of vessels. Vascular interventional surgical robots, such as CorPath, Amigo and Niobe, all employ preoperative CTA for diagnosis and surgical planning, and use DSA images for intraoperative navigation [5,6,7]. Enhancing interventional visualization by integrating multi-mode intraoperative and preoperative information into a comprehensive system is of great importance of MIVI navigation [8]. Traditional vascular interventional surgery is usually guided only by DSA images [9,10,11], which can display the two-dimensional structure and morphology of blood vessels in real time [12]. However, patients and doctors need to be constantly exposed to the X-rays during the DSA process. Besides, the vascular structure is partially displayed in each frame with blurred boundaries and inhomogeneous distribution of contrast agent [13].

3D reconstruction of preoperative computed tomography angiograph (CTA) slices offers surgeon 3D vascular structures, while segmentation of intraoperative DSA sequences integrates vascular information in multi-frame images to provide doctors with more 2D vascular information [14,15,16]. 2D-3D vascular image registration combines the advantages of both preoperative CTA slices and intraoperative DSA sequences to increase visual information and reduce the use of X-rays during surgery [17,18,19,20,21,22,23]. Electromagnetic system (EM) as a new accurate and radiation-free positioning method has also been gradually used in the positioning of the end of interventional devices [24,25,26,27].

This study presented a navigation system combining both preoperative and intraoperative multi-mode information of MIVI. The accuracy of the navigation system has also been evaluated using clinical CTA and DSA images or a vascular model.

Method

Functions of the robot assisted MIVI navigation system presented in this study, as shown in Figs. 1 and 2, include three main parts: preoperative 3D vascular reconstruction of CT slices, registration of preoperative 3D vascular model and intraoperative 2D DSA images, and intraoperative interventional instrument positioning.

Fig. 1
figure 1

Navigation system for robot assisted MIVI. A was the intraoperative DSA, B demonstrated the internal vascular view, C was intraoperative physiological signal monitoring, D was reconstructed 3D vascular model from CTA, E represented the 2D-3D registration result, F demonstrated positioning of surgical instruments

Full size image
Fig. 2
figure 2

Workflow and main functions of MIFNS. Stage 1 included CTA slices segmentation and 3D vascular model reconstruction, Stage 2 demonstrated intraoperative 2D-3D image registration and fusion, Stage 3 covered surgical planning and intraoperative instrument localization

Full size image

3D vascular model reconstruction can be divided into three main steps: vascular segmentation from CT slices, erasing mis-segmented tissue and 3D vascular reconstruction [28]. Since the diameter of human blood vessels is difficult to quantify, the accuracy of 3D reconstruction is verified by a blood vascular model. 2D-3D registration of preoperative 3D reconstructed model and 2D intraoperative DSA images aims to find the optimal spatial transformation from a 3D model to 2D DSA images [22, 23]. The quantitative analysis of 2D-3D registration was quantified by the vascular model, while the qualitative analysis was evaluated by clinical CT and DSA images. Three evaluation criteria, mean target registration error (mTRE), mean absolute error (MAE) and dice coefficient are used in this paper to evaluate the registration result. mTRE was the mean error between registration points and target points. MAE represented the mean error between the registered and the groudtruth of the transformation parameters, mDice measured the fit between two registered images.

EM was used to measure the position of surgical instruments, and its positioning accuracy was evaluated by measuring the root mean square error (RMS) between the spatial coordinates and the EM coordinates of twenty-six catheter end positions selected on each path.

Result

3D vascular model reconstruction

Figure 3a shows the process and results of 3D vascular reconstruction. Since the vessel diameter contained in CTA is difficult to measure due to the influence of contrast agent and blood flow, the vessel model as shown in Fig. 3b with known shape parameters is used to facilitate quantification of 3D vascular reconstruction.

Fig. 3
figure 3

3D reconstruction of CTA slices and the vascular model

Full size image

The mean absolute errors (MAE) between diameters of the reconstructed model and the real model were reliable indicators to access the reconstruction accuracy. MAEs of branches A ~ F with varied diameters as shown in Fig. 3b were calculated separately and the results were shown in the following Table 1. D in Table 1 means the diameters of these branches. The MAEs demonstrated that the average reconstruction errors were all less than 1 mm, which meets the requirements for robot assisted MIVI.

Table 1 MAEs between diameters of the reconstruct model and the real model
Full size table

2D-3D image registration

Figure 4 demonstrates the process of 2D-3D registration. The registration results directly affect the accuracy of surgical map construction. The accuracy of 2D-3D vascular image registration was evaluated by both the vascular model and clinical images. Vascular model was used to quantify the registration effect, and clinical data were used to verify the clinical application prospect of the proposed navigation system.

Fig. 4
figure 4

2D-3D registration of a preoperative vascular model and an intraoperative DSA

Full size image

As shown in Table 2, the MAEs of three transformation parameters (T) were all less than 1 mm, and the MAEs of three rotation parameters (R) were smaller than or approach 0.5°. The mTREs were within the range of (0.1,0.2) that satisfied the requirement of clinical experiments. There were nine patients covered in the clinical experiments including three each with abdominal aortic aneurysm (AAA), thoracic aortic aneurysm (TAA) and aortic dissection (AD). Clinical data lacked ground truth registration result values, so mDice values as shown in Table 3 were used to measure the registration results. As shown in Table 3, mDice values of the 9 patients were all around 0.5, and there was no significant difference between the registration results of the three kinds of diseases. The main reason for the low mDice value in the registration results of clinical CTA image and DSA image is that the vascular region segmented in DSA image is affected by uneven flow of contrast agent, resulting in partial loss of blood vessels. The difference in the range of vessels covered by the two mode images leads to the low accuracy of the final registration results.

Table 2 Registration result of vascular model
Full size table
Table 3 Registration result of clinical data
Full size table

Surgical path planning and navigation accuracy

The movement paths of the catheter from the starting point to positions 1, 2, 3 and 4 were conducted by the MIVI robot as shown in Figs. 5 and 6. The positioning accuracy of EM for the interventional device was evaluated by measuring the root mean square error (RMS) between the spatial coordinates and the EM coordinates of twenty-six catheter end positions selected on each path.

Fig. 5
figure 5

Structure and operation of the MIVI robot

Full size image
Fig. 6
figure 6

Navigation accuracy of 4 paths on the vascular model

Full size image

As shown in Fig. 7, the mean values of RMS were all less than 1 mm, and the maximum values were all smaller than 2 mm. The experiment results verified that the positioning accuracy of EM satisfied the requirements of clinical operations.

Fig. 7
figure 7

The statistical root mean square (RMS) error of 4 navigation routes

Full size image

Discussion

The navigation of robot assisted MIVI were successfully completed on the vascular model with high accuracy and less contrast injection. The 2D-3D image registration combining both the advantages of CTA and DSA, makes it possible to simultaneously visualize the preoperative and intraoperative images. Accurate 3D reconstruction of blood vessels and positioning of surgical instruments ensure the operation accuracy of robot-assisted surgery and improve the safety of surgery.

First of all, 3D vascular models containing sufficient vascular shape and structure information were reconstructed from preoperative CTA slices, which were the foundation of three-dimensional navigation. The accuracy of vascular 3D reconstruction was quantified by vascular model to verify that it satisfied the clinical requirements. Meanwhile, 2D-3D vascular image registration combined the advantages of preoperative 3D vascular model and intraoperative DSA images to increase the real-time performance of the navigation system. The registration result offers surgeon a global vascular map which only needs a clear DSA sequence during operation. The reduction of the demand for DSA sequence decreases the use of contrast agent during the operation. Although the performance of 2D-3D registration on clinical data was not as satisfied as that on vascular models, it provided doctors with a static but intuitive map of vessels. Moreover, surgical path planning helped doctors or robotics select the optimal surgical route to shorten the surgical time and improve the surgical safety. Finally, the EM system was used to track and locate the surgical instruments by integrating the multi-mode information of the navigation system. The average positioning accuracy was less than 1 mm, which evaluated the accuracy and clinical application prospect of the navigation system.

Conclusion

The multi-mode information fusion navigation system proposed in this paper for TIVI combined multi-modal medical information during both preoperative and intraoperative operation to provide comprehensive and intuitive visual navigation for doctors or robots. The main functions of the navigation system were evaluated by sufficient experiments on clinical data and vascular models to verify that they satisfied the clinical applications. The navigation system also needs further improvements to provide more accurate navigation information for doctors and robots. For example, the clinical application of the navigation system is only partially verified in this paper. In future studies, more clinical animal and human experiments will be covered in our research to verify the function and accuracy of the navigation system.

Availability of data and materials

The datasets analyzed during the current study are not publicly available due to respect the participants’ rights to privacy and to protect their identity, but are available from Shaoya Guan on reasonable request.

References

  1. Nakadate R, Iwasa T, Onogi S, Arata J, Oguri S, Okamoto Y, Akahoshi T, Eto M, Hashizume M. Surgical robot for intraluminal access: an ex vivo feasibility study. Cyborg Bionic Syst. 2020. https://doi.org/10.34133/2020/8378025.

  2. Kaukemueller L, Klapdor R, Roettger M, et al. Establishment of the DaVinci surgical robot system in the surgical Gynecology. Geburtshilfe Frauenheilkd. 2020;10:80.

    Google Scholar 

  3. Lo N, Gutierrez JA, Swaminathan RV. Robotic-assisted percutaneous coronary intervention. Curr Treat Options Cardiovasc Med. 2018;20(2):14.

    Article  PubMed  Google Scholar 

  4. Maor E, Eleid MF, Gulati R, et al. Current and future use of robotic devices to perform percutaneous coronary interventions: a review. J Am Heart Assoc. 2017;6(7):8.

    Article  Google Scholar 

  5. Granada J, Delgado J, Uribe M, et al. First-in-human evaluation of a novel robotic-assisted coronary angioplasty system. JACC Cardiovasc Interven. 2011;4(4):460–5.

    Article  Google Scholar 

  6. Datino T, Arenal A, Pelliza M, et al. Comparison of the safety and feasibility of arrhythmia ablation using the amigo robotic remote catheter system versus manual ablation. Am J Cardiol. 2014;113(5):827–31.

    Article  PubMed  Google Scholar 

  7. Antoniou G, Riga C, Mayer E, et al. Clinical applications of robotic technology in vascular and endovascular surgery. J Vasc Surg. 2011;53(2):493–9.

    Article  PubMed  Google Scholar 

  8. Cleary K, Peters TM. Image-guided interventions: technology review and clinical applications. Annu Rev Biomed Eng. 2010;12:119–42.

    Article  CAS  PubMed  Google Scholar 

  9. Auffermann WF, Krupinski EA, Tridandapani S. Search pattern training for evaluation of central venous catheter positioning on chest radiographs. J Med Imaging. 2018;5(3): 031407.

    Article  Google Scholar 

  10. Khan EM, Frumkin W, Ng GA, et al. First experience with a novel robotic remote catheter system: Amigo™ mapping trial. J Interv Card Electrophysiol. 2013;37(2):121–9.

    Article  PubMed  Google Scholar 

  11. Yi X, Adams S, Babyn P, et al. Automatic catheter and tube detection in pediatric x-ray images using a scale-recurrent network and synthetic data. J Digit Imaging. 2020;33(1):181–90.

    Article  CAS  PubMed  Google Scholar 

  12. Markelj P, Tomaevi D, Likar B, et al. A review of 3D/2D registration methods for image-guided interventions. Med Image Anal. 2012;16(3):642–61.

    Article  CAS  PubMed  Google Scholar 

  13. Sun W, He Y, Ge R, et al. Projection network with Spatio-temporal information: 2D+ time DSA to 2D aorta segmentation. Multimed Tools Appl. 2022; 1–15.

  14. Meng C, Sun K, Guan S, et al. Multiscale dense convolutional neural network for DSA cerebrovascular segmentation. Neurocomputing. 2020;373:123–34.

    Article  Google Scholar 

  15. Sulayman N, Al-Mawaldi M, Kanafani Q. Semi-automatic detection and segmentation algorithm of saccular aneurysms in 2D cerebral DSA images. Egypt J Radiol Nucl Med. 2016;47(3):859–65.

    Article  Google Scholar 

  16. Meng C, Xu Y, Li N, et al. Incremental robust PCA for vessel segmentation in DSA sequences. Biomed Phys Eng Express. 2022;8(4): 045002.

    Article  Google Scholar 

  17. Fan J, Cao X, Wang Q, et al. Adversarial learning for mono-or multi-modal registration. Med Image Anal. 2019;58: 101545.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Fechter T, Baltas D. One-shot learning for deformable medical image registration and periodic motion tracking. IEEE Trans Med Imaging. 2020;39(7):2506–17.

    Article  PubMed  Google Scholar 

  19. Cao X, Yang J, Zhang J, et al. Deformable image registration using a cue-aware deep regression network. IEEE Trans Biomed Eng. 2018;65(9):1900–11.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Miao S, Wang ZJ, Liao R. A CNN regression approach for real-time 2D/3D registration. IEEE Trans Med Imaging. 2016;35(5):1352–63.

    Article  Google Scholar 

  21. Zheng J, Miao S, Wang ZJ, et al. Pairwise domain adaptation module for CNN-based 2-D/3-D registration. J Med Imaging. 2018;5(2): 021204.

    Article  Google Scholar 

  22. Guan S, Wang T, Sun K, et al. Transfer learning for nonrigid 2D/3D cardiovascular images registration. IEEE J Biomed Health Inform. 2020;2020(29):3300–9.

    Google Scholar 

  23. Guan S, Meng C, Wang T, et al. Deformable cardiovascular image registration via multi-channel convolutional neural network. IEEE Access. 2019;7:17524–34.

    Article  Google Scholar 

  24. Gullo G, Colin A, Frossard P, et al. Appropriateness of replacing fluoroscopic guidance with ECG-electromagnetic guidance for PICC insertion: a randomized controlled trial. Am J Roentgenol. 2021;216(4):981–8.

    Article  Google Scholar 

  25. Maalouf J, Whiteside HL, Pillai A, et al. Reduction of radiation and contrast agent exposure in a cryoballoon ablation procedure with integration of electromagnetic mapping and intracardiac echocardiography: a single center experience. J Interv Card Electrophysiol. 2020;59(3):545–50.

    Article  PubMed  Google Scholar 

  26. Cavaliere M, McVeigh O, Jaeger HA, et al. Inductive sensor design for electromagnetic tracking in image guided interventions. IEEE Sens J. 2020;20(15):8623–30.

    Article  Google Scholar 

  27. Schwein A, Kramer B, Chinnadurai P, et al. Electromagnetic tracking of flexible robotic catheters enables “assisted navigation” and brings automation to endovascular navigation in an in vitro study. J Vasc Surg. 2018;67(4):1274–81.

    Article  PubMed  Google Scholar 

  28. Lorensen WE, Cline HE. Marching cubes: a high resolution 3D surface construction algorithm. ACM Siggraph Comput Graph. 1987;21(4):163–9.

    Article  Google Scholar 

Download references

Acknowledgements

Thanks to Yi Ning and Xiaoyong Huang, Beijing Anzhen Hospital, Capital Medical University, for providing the clinical data.

Funding

This work was supported by the Cross-Disciplinary Science Foundation from Beijing Institute of Petrochemical Technology [Grant number BIPTCSF-016], National Innovation and Entrepreneurship Program for College Students of China [Grant numbers 2022J00037, 2022J00100], National Major Research Program of China [Grant number 2019YFB1311703] and the National Natural Science Foundation of China [Grant numbers 61533016, 61873010].

Author information

Authors and Affiliations

  1. School of Engineers, Beijing Institute of Petrochemical Technology, Beijing, China

    Shaoya Guan & Limei Ma

  2. School of Information Engineering, Beijing Institute of Petrochemical Technology, Beijing, China

    Tianqi Li

  3. School of Astronautics, Beihang University, Beijing, China

    Cai Meng

Authors
  1. Shaoya Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cai Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Limei Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SG wrote the main manuscript text. TL did the experiments in this paper with the help of SG and LM, CM and LM contributed significantly to analysis and manuscript preparation. All authors reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Limei Ma.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Medical ethics committee of Beijing Anzhen Hospital Affiliated to Capital Medical University. All experiments were performed in accordance with the guidelines and regulations of “Medical ethics committee of Beijing Anzhen Hospital Affiliated to Capital Medical University”. The informed consent about the usage of medical images covered in the experiments of this paper was obtained of all subjects or their legal guardians.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guan, S., Li, T., Meng, C. et al. Multi-mode information fusion navigation system for robot-assisted vascular interventional surgery. BMC Surg 23, 51 (2023). https://doi.org/10.1186/s12893-023-01944-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12893-023-01944-5

Keywords

BMC Surgery

ISSN: 1471-2482

Contact us