, projecting the coordinates of the annotations to the endoscopic view based on the registration result between the US image and the endoscopic camera
, Endoscopic image augmentation: (1) annotating distal and proximal margin (green) of the simulated tumor on 'rectal wall, US images using crosshairs
, generating resection line (red) 10 mm below the most distal margin
, During the experiment, the surgeon marked the proximal and distal end of the simulated tumor and the distal resection margin on the rectum phantom by using the implemented AR framework, as shown in Figure 4.17 (a). Then, the surgeon made straight cuts on these marks
Calibration for Position Tracking of Swept Motor 3-D Ultrasound, Ultrasound in Medicine and Biology, vol.40, issue.6, pp.1356-1371, 2014. ,
Visual servoing of a robotic endoscope holder based on surgical instrument tracking, IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, pp.13-18, 2014. ,
URL : https://hal.archives-ouvertes.fr/hal-01164895
, Stereotactic navigation for TAMIS-TME. Minimally Invasive Therapy and Allied Technologies, vol.25, pp.271-277, 2016.
A survey of augmented reality, Presence: Teleoperators and Virtual Environments, vol.6, issue.4, pp.355-385, 1997. ,
Robotic surgery for rectal cancer can overcome difficulties associated with pelvic anatomy, Surgical Endoscopy, vol.29, issue.6, pp.1419-1424, 2015. ,
Comparing calibration approaches for 3D ultrasound probes, International Journal of Computer Assisted Radiology and Surgery, vol.4, pp.203-213, 2009. ,
The status of augmented reality in laparoscopic surgery as of 2016, Medical Image Analysis, vol.37, pp.66-90, 2017. ,
Contextual anatomic mimesis hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality, 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp.129-138, 2007. ,
Tracking devices, Image-Guided Interventions, pp.23-45, 2008. ,
Transcervical ultrasonography is feasible to visualize and evaluate base of tongue cancers, PLoS ONE, vol.9, issue.1, p.87565, 2014. ,
Head and neck malignancy: is PET/CT more accurate than PET or CT alone?, Radiology, vol.235, issue.2, pp.580-586, 2005. ,
Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries, CA: A Cancer Journal for Clinicians, vol.68, issue.6, pp.394-424, 2018. ,
Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display, Journal of Biomedical Informatics, vol.55, pp.124-131, 2015. ,
Concurrent photoacoustic markers for direct three-dimensional ultrasound to video registration, SPIE 8943, Photons Plus Ultrasound: Imaging and Sensing, vol.8943, p.89435, 2014. ,
Fused video and ultrasound images for minimally invasive partial nephrectomy: A phantom study, International Conference on Medical Image Computing and Computer-Assisted Intervention, pp.408-415, 2010. ,
An effective visualization technique for depth perception in augmented reality-based surgical navigation, International Journal of Medical Robotics and Computer Assisted Surgery, vol.12, issue.1, pp.62-72, 2016. ,
Intraoperative Ultrasonography during Transoral Robotic Surgery, Rhinology and Laryngology, vol.125, issue.1, pp.37-42, 2016. ,
Intraoperative ultrasound for guidance and tissue shift correction in image-guided neurosurgery, Medical Physics, vol.27, issue.4, pp.787-800, 2000. ,
Visualization of the oropharynx with transcervical ultrasound, American Journal of Roentgenology, vol.205, issue.6, pp.1288-1294, 2015. ,
Hand-eye calibration using dual quaternions, The International Journal of Robotics Research, vol.18, pp.286-298, 1999. ,
The dual quaternion approach to hand-eye calibration, Proceedings -International Conference on Pattern Recognition, vol.1, pp.318-322, 1996. ,
Discrete Mesh Approach in Morphogenesis Modelling: the Example of Gastrulation, Acta Biotheoretica, vol.64, issue.4, pp.427-446, 2016. ,
URL : https://hal.archives-ouvertes.fr/hal-01498621
Transoral robotic surgery using an image guidance system, The Laryngoscope, vol.118, issue.11, pp.2003-2005, 2008. ,
3D ultrasonic image feature localization based on magnetic scanhead tracking: in vitro calibration and validation, Ultrasound in Medicine and Biology, vol.20, issue.9, pp.923-936, 1994. ,
Measures of the amount of ecologic association between species, Ecology, vol.26, issue.3, pp.297-302, 1945. ,
A modified Hausdorff distance for object matching, 12th International Conference on Pattern Recognition, pp.566-568, 1994. ,
Accuracy evaluation of a mitral valve surgery assistance system based on optical tracking, International Journal of Computer Assisted Radiology and Surgery, vol.11, issue.10, pp.1891-1904, 2016. ,
The use of ultrasound in the search for the primary site of unknown primary head and neck squamous cell cancers, Oral Oncology, vol.50, issue.7, pp.640-645, 2014. ,
Augmented fluorescence lifetime imaging for enhanced tumor margin delineation during trans-oral robotic surgery, Proceedings of the 9th International Conference on Head and Neck Cancer, 2016. ,
3D Slicer as an image computing platform for the quantitative imaging network, Magnetic Resonance Imaging, vol.30, issue.9, pp.1323-1341, 2012. ,
Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection, IEEE Transactions on Medical Imaging, vol.27, issue.3, pp.355-369, 2008. ,
Magneto-optical tracking of flexible laparoscopic ultrasound: model-based online detection and correction of magnetic tracking errors, IEEE Transactions on Medical Imaging, vol.28, issue.6, pp.951-967, 2009. ,
Image overlay guidance for needle insertion in CT scanner, IEEE Transactions on Biomedical Engineering, vol.52, issue.8, pp.1415-1424, 2005. ,
URL : https://hal.archives-ouvertes.fr/inria-00590185
Predicting error in rigid-body point-based registration, IEEE Transactions on Medical Imaging, vol.17, issue.5, pp.694-702, 1998. ,
Electromagnetic tracking in medicine -A review of technology, validation, and applications, IEEE Transactions on Medical Imaging, vol.33, issue.8, pp.1702-1725, 2014. ,
Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: Phantom studies, International Journal of Computer Assisted Radiology and Surgery, vol.7, issue.4, pp.547-556, 2012. ,
Fast contour identification through efficient hough transform and simplified interpretation strategy, International Conference on Pattern Recognition, pp.498-500, 1986. ,
Intraoperative ultrasound in colorectal surgery, Journal of Clinical Ultrasound, vol.37, issue.7, pp.375-379, 2009. ,
Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery, IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.199-208, 2013. ,
URL : https://hal.archives-ouvertes.fr/hal-00842855
Accurate fiducial location for freehand 3D ultrasound calibration, Ultrasonic Imaging and Signal Processing, vol.6513, p.651315, 2007. ,
MR to ultrasound registration for image-guided prostate interventions, Medical Image Analysis, vol.16, issue.3, pp.687-703, 2012. ,
Individual variations in mucosa and total wall thickness in the stomach and rectum assessed via endoscopic ultrasound, Physiological measurement, vol.24, issue.4, pp.15-22, 2003. ,
Real-time image guidance in laparoscopic liver surgery: First clinical experience with a guidance system based on intraoperative CT imaging, Surgical Endoscopy, vol.28, issue.3, pp.933-940, 2014. ,
Comparative study of transrectal ultrasonography, pelvic computerized tomography, and magnetic resonance imaging in preoperative staging of rectal cancer, Diseases of the Colon and Rectum, vol.42, issue.6, pp.770-775, 1999. ,
Ultrasound imaging of tongue malignancy, International Journal of Case Reports and Images, vol.8, issue.1, pp.1-6, 2017. ,
Augmented reality navigation system for endoscopic surgery based on three-dimensional ultrasound and computed tomography: Application to 20 clinical cases, International Congress Series, vol.1281, pp.537-542, 2005. ,
Intraoperative US of the liver: techniques and cilnical applications, RadioGraphics, vol.26, issue.4, pp.1067-1084, 2006. ,
Vessel-based non-rigid registration of MR / CT and 3D ultrasound for navigation in liver surgery, Computer Aided Surgery, vol.8, issue.5, pp.228-240, 2003. ,
Augmented reality based navigation for computer assisted hip resurfacing: a proof of concept study, Annals of Biomedical Engineering, vol.46, issue.10, pp.1595-1605, 2018. ,
Toward intraoperative image-guided transoral robotic surgery, Journal of robotic surgery, vol.7, issue.3, pp.217-225, 2013. ,
Augmented reality and cone beam CT guidance for transoral robotic surgery, Journal of Robotic Surgery, vol.9, issue.3, pp.223-233, 2015. ,
Laparoscopic stereoscopic augmented reality: toward a clinically viable electromagnetic tracking solution, Journal of Medical Imaging, vol.3, issue.4, p.45001, 2016. ,
Registration and fusion of 3D head-neck MRI and 3D/4D tongue ultrasound, The Journal of the Acoustical Society of America, vol.144, issue.3, pp.1904-1904, 2018. ,
Robust hand-eye calibration for computer aided medical endoscopy, 2010 IEEE International Conference on Robotics and Automation, pp.5543-5549, 2010. ,
Framework for augmented reality in Minimally Invasive laparoscopic surgery, 17th International Conference on E-Health Networking, Application and Services (HealthCom), pp.22-27, 2015. ,
URL : https://hal.archives-ouvertes.fr/hal-01315574
Augmented reality visualization system for intravascular neurosurgery, Computer Aided Surgery, vol.3, issue.5, pp.239-247, 1998. ,
An augmented reality framework for soft tissue surgery, International Conference on Medical Image Computing and Computer-Assisted Intervention, pp.423-431, 2014. ,
Intraoperative magnetic tracker calibration using a magneto-optic hybrid tracker for 3-D ultrasound-based navigation in laparoscopic surgery, IEEE Transactions on Medical Imaging, vol.27, issue.2, pp.255-270, 2008. ,
Invisible shadow for navigation and planning in minimal invasive surgery, International Conference on Medical Image Computing and Computer-Assisted Intervention, pp.25-32, 2005. ,
Feasibility of transoral roboticassisted high-resolution microendoscopic imaging of oropharyngeal squamous cell carcinoma, Head & Neck, vol.37, issue.8, pp.99-102, 2015. ,
Comparison of calibration methods for spatial tracking of a 3-D ultrasound probe, Ultrasound in Medicine & Biology, vol.31, issue.8, pp.1095-1108, 2005. ,
Three-dimensional registration and fusion of ultrasound and MRI using major vessels as fiducial Markers, IEEE Transactions on Medical Imaging, vol.20, issue.4, pp.354-359, 2001. ,
Rapid calibration for 3-D freehand ultrasound, Ultrasound in Medicine and Biology, vol.24, issue.6, pp.855-869, 1998. ,
Transoral robotic surgery: image guidance and augmented reality, ORL, vol.80, pp.204-212, 2018. ,
An effective visualisation and registration system for image-guided robotic partial nephrectomy, Journal of Robotic Surgery, vol.6, issue.1, pp.23-31, 2012. ,
Automatic detection of the uterus and fallopian tube junctions in laparoscopic images, International Conference on Information Processing in Medical Imaging, pp.552-563, 2015. ,
Toward long-term and accurate augmented-reality for monocular endoscopic videos, IEEE Transactions on Biomedical Engineering, vol.61, issue.10, pp.2609-2620, 2014. ,
Deformable image registration for cone-beam CT guided transoral robotic base of tongue surgery, Physics in Medicine & Biology, vol.58, issue.14, pp.4951-4979, 2013. ,
Augmented reality guidance for needle biopsies: an initial randomized, controlled trial in phantoms, Medical Image Analysis, vol.6, issue.3, pp.313-320, 2002. ,
Confhusius: a robust and fully automatic calibration method for 3D freehand ultrasound, Medical Image Analysis, vol.9, issue.1, pp.25-38, 2005. ,
URL : https://hal.archives-ouvertes.fr/inserm-00136520
Low rectal cancer: classification and standardization of surgery, Diseases of the Colon and Rectum, vol.56, issue.5, pp.560-567, 2013. ,
Ultrasonography: an alternative to x-ray-guided needle localization of nonpalpable breast masses, Surgery, vol.104, issue.5, pp.870-873, 1988. ,
Fast and simple automatic 3D ultrasound probe calibration based on 3D printed phantom and an untracked marker, pp.878-882, 2018. ,
URL : https://hal.archives-ouvertes.fr/lirmm-01828628
, , 1979.
, Intraoperative ultrasonic visualization of biliary calculi, Current surgery, vol.36, issue.3, pp.158-159
Augmented reality visualization during laparoscopic radical prostatectomy, Journal of Endourology, vol.25, issue.12, pp.1841-1845, 2011. ,
Intra-operative ultrasound-based augmented reality guidance for laparoscopic surgery, Healthcare Technology Letters, vol.4, issue.5, pp.204-209, 2017. ,
Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? preliminary in vitro and in vivo results, European Urology, vol.56, issue.2, pp.332-338, 2009. ,
k-Wave: MATLAB toolbox for the simulation and reconstruction of photoacoustic wave fields, Journal of Biomedical Optics, vol.15, issue.2, p.21314, 2010. ,
Oral cavity and oropharyngeal squamous cell cancer: key imaging findings for staging and treatment planning, RadioGraphics, vol.31, issue.2, pp.339-354, 2011. ,
A new technique for fully autonomous and efficient 3D robotics hand/eye calibration, IEEE Transactions on Robotics and Automation, vol.5, issue.3, pp.345-358, 1989. ,
Augmented reality for image-guided surgery in urology, New Technologies in Urology, pp.215-222, 2010. ,
Surgical management of the base of tongue, Operative Techniques in Otolaryngology -Head and Neck Surgery, vol.24, pp.74-85, 2013. ,
Total pelvic exenteration for primary locally advanced and locally recurrent rectal cancer, European Journal of Surgical Oncology, vol.33, issue.4, pp.452-458, 2007. ,
Prostate biopsies assisted by comanipulated probe-holder: first in man, International Journal of Computer Assisted Radiology and Surgery, vol.11, issue.6, pp.1153-1161, 2016. ,
URL : https://hal.archives-ouvertes.fr/hal-01312297
Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: Not only a matter of fashion, Journal of Hepato-Biliary-Pancreatic Sciences, vol.18, issue.4, pp.506-509, 2011. ,
Endoscope field of view measurement, Biomedical Optics Express, vol.8, issue.3, pp.1441-1454, 2017. ,
Fully automatic endoscope calibration for intraoperative use, Bildverarbeitung für die Medizin, pp.419-423, 2006. ,
Robust augmented reality guidance with fluorescent markers in laparoscopic surgery, International Journal of Computer Assisted Radiology and Surgery, vol.11, issue.6, pp.899-907, 2016. ,
Sparse 2-D array design for real time rectilinear volumetric imaging, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control, vol.47, issue.1, pp.93-110, 2000. ,
Real-time surgical tool tracking and pose estimation using a hybrid cylindrical marker, International journal of computer assisted radiology and surgery, vol.12, issue.6, pp.921-930, 2017. ,
A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.22, 2000. ,
Intraoperative ultrasound-based augmented reality guidance, 2019. ,
URL : https://hal.archives-ouvertes.fr/lirmm-02105828
Transrectal ultrasound image-based real-time augmented reality guidance in robot-assisted laparoscopic rectal surgery: a proof-of-concept study, International Journal of Computer Assisted Radiology and Surgery (IJCARS, 2019. ,
URL : https://hal.archives-ouvertes.fr/lirmm-02381958
Intraoperative ultrasonography-based augmented reality for application in image guided robotic surgery. Computer 32nd Assisted Radiology and Surgery (CARS), International Journal of Computer Assisted Radiology and Surgery (IJCARS), vol.13, pp.45-46, 2018. ,
URL : https://hal.archives-ouvertes.fr/lirmm-01828409
Fast and simple automatic 3D ultrasound probe calibration based on 3D printed phantom and an untracked marker, 40th IEEE Conference on Engineering in Medicine and Biology Conference (EMBC), pp.878-882, 2018. ,
URL : https://hal.archives-ouvertes.fr/lirmm-01828628
Intraoperative 3D ultrasound probe calibration, Recherche en Imagerie et Technologies pour la Santée, p.2017, 2017. ,
Augmented perception in transoral robotic surgery for tongue base cancer, 7th Joint Workshop on New Technologies for Computer/Robot Assisted Surgery, p.2017, 2017. ,
Augmented reality visualization based on 3D ultrasonography, 2017. ,
URL : https://hal.archives-ouvertes.fr/lirmm-02105826