A Neurosurgical Instrument Segmentation Approach to Assess Microsurgical Movements
Summary
The paper builds a pipeline that segments and tracks microsurgical instruments in video from a neurosurgical microscope. The masks are a base for later measurement of surgical movements. This paper reports segmentation quality only. It does not score surgeon skill.
Data and method
- CholecSeg8k: public set of 8,080 laparoscopic cholecystectomy frames from 17 clips.
- PSNV: 1,000 frames (1920 x 1080, 25 FPS) from microscope videos at the Burdenko Neurosurgery Center, with four instruments: microsurgical scissors, aspirator, tweezers and spatula. Segment Anything in LabelStudio made the first masks. A neurosurgeon with over 10 years of experience corrected each mask by hand.
- Proposed model: EndoViT fine-tuned on CholecSeg8k, plus the mask tracker from Segment and Track Anything (Cheng et al., 2023). It is tested on PSNV zero-shot, with no neurosurgical training frames.
- Baseline: YOLOv8l-seg fine-tuned on CholecSeg8k, with and without tracking.
- Inference: stabilize the video, segment the first frame, track the mask on the next frames, and segment again when the scene changes a lot.
Results
| Method | Dataset | Mean IoU | Mean Dice |
|---|---|---|---|
| EndoViT + tracking (proposed) | CholecSeg8k | 0.8158 | 0.8657 |
| EndoViT, no tracking | CholecSeg8k | 0.7235 | 0.7328 |
| YOLOv8l-seg + tracking | CholecSeg8k | 0.7762 | 0.8587 |
| YOLOv8l-seg, no tracking | CholecSeg8k | 0.7228 | 0.7731 |
| EndoViT + tracking (proposed) | PSNV | 0.7196 | 0.8202 |
| EndoViT, no tracking | PSNV | 0.6357 | 0.7520 |
| YOLOv8l-seg + tracking | PSNV | 0.4644 | 0.5627 |
| YOLOv8l-seg, no tracking | PSNV | 0.5996 | 0.6099 |
Tracking raised EndoViT on both datasets. On the neurosurgical frames, tracking made YOLOv8l-seg worse (IoU 0.5996 to 0.4644). On CholecSeg8k the authors report mean average precision of 0.87, against 0.82 in earlier work by Kanakatte et al.
Paper:
IOS Press (open access) ·
PubMed 39575805
Cite as
@inproceedings{danilov2024neurosurgical,
title={A Neurosurgical Instrument Segmentation Approach to Assess Microsurgical Movements},
author={Danilov, Gleb and Pilipenko, Oleg and Kostyumov, Vasiliy and Trubetskoy, Sergey and Maloyan, Narek and Nutfullin, Bulat and Ilyushin, Eugeniy and Pitskhelauri, David and Zelenova, Alexandra and Bykanov, Andrey},
booktitle={Collaboration across Disciplines for the Health of People, Animals and Ecosystems (EFMI STC 2024)},
series={Studies in Health Technology and Informatics},
volume={321},
pages={185--189},
publisher={IOS Press},
year={2024},
doi={10.3233/SHTI241089}
}