Cross-frame feature-saliency mutual reinforcing for weakly supervised video salient object detection

Wang, Jian, Yu, Siyue, Zhang, Bingfeng, Zhao, Xingiao, Garcia-Fernandez, Angel ORCID: 0000-0002-6471-8455, Lim, Eng Gee ORCID: 0000-0003-0199-7386 and Xiao, Jimin (2024) Cross-frame feature-saliency mutual reinforcing for weakly supervised video salient object detection Pattern Recognition, 150. p. 110302. ISSN 0031-3203, 1873-5142

Text Wang24_accepted.pdf - Author Accepted Manuscript Available under License Creative Commons Attribution. Download (2MB) | Preview

Abstract

Scribble annotations have recently become popular in video salient object detection. Previous methods only focus on utilizing shallow feature consistency for more integral predictions. However, there is potential for consistency between cross-frame deep features to be used to help regularize saliency predictions better. Besides, we have observed that leveraging saliency predictions as pseudo-supervision signals yields notable improvements in extracting both intra-frame and cross-frame deep features. This, in turn, leads to more precise and detailed object structural information. Thus, we propose a cross-frame feature-saliency mutual reinforcing training process to assist scribble annotations for integral video saliency predictions. Specifically, we design a cross-frame feature regularization head, which leverages intra-frame and cross-frame deep feature consistency to regularize saliency predictions as auxiliary supervision. Then, to help obtain more accurate feature consistency, we design a cross-frame saliency regularization head, where predicted saliency values are used as pseudo-supervision signals to acquire better feature consistency. In this way, our cross-frame feature and saliency regularization heads can benefit from each other to help the network learn more accurately. Extensive experiments show that our method can achieve better performances than the previous best methods. The project is available at https://github.com/muchengxue0911/CFMR.

Item Type:	Article
Uncontrolled Keywords:	Video salient object detection, Scribble annotations, Cross-frame feature consistency, Cross-frame saliency consistency
Divisions:	Faculty of Science & Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	06 Feb 2024 08:25
Last Modified:	28 Feb 2026 00:43
DOI:	10.1016/j.patcog.2024.110302
Related Websites:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3178400
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.