SideInfNet: A Deep Neural Network for Semi-Automatic Semantic Segmentation with Side Information

Jing Yu Koh1 Duc Thanh Nguyen2 Quang-Trung Truong1 Sai-Kit Yeung3 Alexander Binder1
1Singapore University of Technology and Design 2Deakin University 3Hong Kong University of Science and Technology

[Paper] [Supplementary] [Video]


Abstract

Fully-automatic execution is the ultimate goal for many Computer Vision applications. However, this objective is not always realistic in tasks associated with high failure costs, such as medical applications. For these tasks, semi-automatic methods allowing minimal effort from users to guide computer algorithms are often preferred due to desirable accuracy and performance. Inspired by the practicality and applicability of the semi-automatic approach, this paper proposes a novel deep neural network architecture, namely SideInfNet that effectively integrates features learnt from images with side information extracted from user annotations. To evaluate our method, we applied the proposed network to three semantic segmentation tasks and conducted extensive experiments on benchmark datasets. Experimental results and comparison with prior work have verified the superiority of our model, suggesting the generality and effectiveness of the model in semi-automatic semantic segmentation.


Paper

SideInfNet: A Deep Neural Network for Semi-Automatic Semantic Segmentation with Side Information

J.Y. Koh, D.T. Nguyen, Q.-T. Truong, S.-K. Yeung, A. Binder.

ECCV, 2020.

[Paper]     [Supplementary]     [arXiv]     [Bibtex]



Full Video


Acknowledgements

Duc Thanh Nguyen was partially supported by an internal SEBE 2019 RGS grant from Deakin University. Sai-Kit Yeung was partially supported by an internal grant from HKUST (R9429) and HKUST-WeBank Joint Lab. Alexander Binder was supported by the MoE Tier2 Grant MOE2016-T2- 2-154, Tier1 grant TDMD 2016-2, SUTD grant SGPAIRS1811, TL grant RTDST1907012. This webpage drew heavy inspiration from this template.