Skip to Main content Skip to Navigation

Real-time detection of surgical tools in 2D neurosurgical videos by modelling global shape and local appearance

Abstract : Despite modern-life technological advances and tremendous progress made in surgical techniques including MIS, today's OR is facing many challenges remaining yet to be addressed. The development of CAS systems integrating the SPM methodology was born as a response from the medical community, with the long-term objective to create surgical cockpit systems. Being able to identify surgical tools in use is a key component for systems relying on the SPM methodology. Towards that end, this thesis work has focused on real-time surgical tool detection from microscope 2D images. From the review of the literature, no validation data-sets have been elected as benchmarks by the community. In addition, the neurosurgical context has been addressed only once. As such, the first contribution of this thesis work consisted in the creation of a new surgical tool data-set, made freely available online. Two methods have been proposed to tackle the surgical tool detection challenge. First, the adapted SquaresChnFtrs, evolution of one of the best computer vision state-of-the-art approach for pedestrian detection. Our second contribution, the ShapeDetector, is fully data-driven and performs detection without the use of prior knowledge regarding the number, shape, and position of tools in the image. Compared to previous works, we chose to represent candidate detections with bounding polygons instead of bounding boxes, hence providing more fitting results. For integration into medical systems, we performed different code optimization through CPU and GPU use. Speed gain and accuracy loss from the use of ad-hoc optimization strategies have been thoroughly quantified to find an optimal trade-off between speed and accuracy. Our ShapeDetector is running in-between 5 and 8Hz for 612x480 pixel video sequences.We validated our approaches using a detailed methodology covering the overall tool location, tip position, and orientation. Approaches have been compared and ranked conjointly with a set of competitive baselines. For suction tube detections, we achieved a 15% miss-rate at 0.1 FPPI, compared to a 55% miss-rate for the adapted SquaresChnFtrs. Future works should be directed toward the integration of 3D feature extraction to improve detection performance but also toward the refinement of the semantic labelling step. Coupling the tool detection task to the tool classification in one single framework should be further investigated. Finally, increasing the data-set in diversity, number of tool classes, and detail of annotations is of interest.
Complete list of metadata

Cited literature [29 references]  Display  Hide  Download
Contributor : ABES STAR :  Contact
Submitted on : Thursday, October 15, 2015 - 12:56:06 PM
Last modification on : Wednesday, September 14, 2022 - 10:20:04 AM
Long-term archiving on: : Monday, January 18, 2016 - 5:53:52 AM


Version validated by the jury (STAR)


  • HAL Id : tel-01215961, version 1


David Bouget. Real-time detection of surgical tools in 2D neurosurgical videos by modelling global shape and local appearance. Signal and Image processing. Université Rennes 1, 2015. English. ⟨NNT : 2015REN1B006⟩. ⟨tel-01215961⟩



Record views


Files downloads