Simultaneous Depth Estimation and Surgical Tool Segmentation in Laparoscopic Images.

Huang, Baoru ORCID: 0000-0002-4421-652X, Nguyen, Anh ORCID: 0000-0002-1449-211X, Wang, Siyao, Wang, Ziyang, Mayer, Erik, Tuch, David, Vyas, Kunal ORCID: 0000-0002-9670-0233, Giannarou, Stamatia ORCID: 0000-0002-8745-1343 and Elson, Daniel S ORCID: 0000-0002-5578-3941
(2022) Simultaneous Depth Estimation and Surgical Tool Segmentation in Laparoscopic Images. IEEE transactions on medical robotics and bionics, 4 (2). pp. 335-338.

[img] Text
2022_TMRB_DepthEstimation.pdf - Author Accepted Manuscript

Download (513kB) | Preview


Surgical instrument segmentation and depth estimation are crucial steps to improve autonomy in robotic surgery. Most recent works treat these problems separately, making the deployment challenging. In this paper, we propose a unified framework for depth estimation and surgical tool segmentation in laparoscopic images. The network has an encoder-decoder architecture and comprises two branches for simultaneously performing depth estimation and segmentation. To train the network end to end, we propose a new multi-task loss function that effectively learns to estimate depth in an unsupervised manner, while requiring only semi-ground truth for surgical tool segmentation. We conducted extensive experiments on different datasets to validate these findings. The results showed that the end-to-end network successfully improved the state-of-the-art for both tasks while reducing the complexity during their deployment.

Item Type: Article
Uncontrolled Keywords: Deep learning, Multi-task learning, Self-supervised depth estimation, Surgical instrument segmentation
Divisions: Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User: Symplectic Admin
Date Deposited: 19 May 2022 13:57
Last Modified: 17 Mar 2024 13:57
DOI: 10.1109/tmrb.2022.3170215
Related URLs: