Wang, Jian, Yu, Siyue, Zhang, Bingfeng, Zhao, Xingiao, Garcia-Fernandez, Angel 
ORCID: 0000-0002-6471-8455, Lim, Eng Gee ORCID: 0000-0003-0199-7386 and Xiao, Jimin
(2024)
Cross-frame feature-saliency mutual reinforcing for weakly supervised video salient object detection.
Pattern Recognition, 150.
p. 110302.
|
Text
Wang24_accepted.pdf - Author Accepted Manuscript Available under License Creative Commons Attribution. Download (2MB) | Preview |
Abstract
Scribble annotations have recently become popular in video salient object detection. Previous methods only focus on utilizing shallow feature consistency for more integral predictions. However, there is potential for consistency between cross-frame deep features to be used to help regularize saliency predictions better. Besides, we have observed that leveraging saliency predictions as pseudo-supervision signals yields notable improvements in extracting both intra-frame and cross-frame deep features. This, in turn, leads to more precise and detailed object structural information. Thus, we propose a cross-frame feature-saliency mutual reinforcing training process to assist scribble annotations for integral video saliency predictions. Specifically, we design a cross-frame feature regularization head, which leverages intra-frame and cross-frame deep feature consistency to regularize saliency predictions as auxiliary supervision. Then, to help obtain more accurate feature consistency, we design a cross-frame saliency regularization head, where predicted saliency values are used as pseudo-supervision signals to acquire better feature consistency. In this way, our cross-frame feature and saliency regularization heads can benefit from each other to help the network learn more accurately. Extensive experiments show that our method can achieve better performances than the previous best methods. The project is available at https://github.com/muchengxue0911/CFMR.
| Item Type: | Article |
|---|---|
| Divisions: | Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science |
| Depositing User: | Symplectic Admin |
| Date Deposited: | 06 Feb 2024 08:25 |
| Last Modified: | 26 Feb 2024 03:22 |
| DOI: | 10.1016/j.patcog.2024.110302 |
| Related URLs: | |
| URI: | https://livrepository.liverpool.ac.uk/id/eprint/3178400 |
Dimensions
Dimensions
