Towards Better Robust Generalization with Shift Consistency Regularization

Zhang, Shufei, Qian, Zhuang, Huang, Kaizhu, Wang, Qiufeng, Zhang, Rui and Yi, Xinping ORCID: 0000-0001-5163-2364 (2021) Towards Better Robust Generalization with Shift Consistency Regularization. In: International Conference on Machine Learning.

Text zhang21p.pdf - Published version Download (3MB) | Preview

Abstract

While adversarial training becomes one of the most promising defending approaches against adversarial attacks for deep neural networks, the conventional wisdom through robust optimization may usually not guarantee good generalization for robustness. Concerning with robust generalization over unseen adversarial data, this paper investigates adversarial training from a novel perspective of shift consistency in latent space. We argue that the poor robust generalization of adversarial training is owing to the significantly dispersed latent representations generated by training and test adversarial data, as the adversarial perturbations push the latent features of natural examples in the same class towards diverse directions. This is underpinned by the theoretical analysis of the robust generalization gap, which is upper-bounded by the standard one over the natural data and a term of feature inconsistent shift caused by adversarial perturbation - a measure of latent dispersion. Towards better robust generalization, we propose a new regularization method - shift consistency regularization (SCR) - to steer the same-class latent features of both natural and adversarial data into a common direction during adversarial training. The effectiveness of SCR in adversarial training is evaluated through extensive experiments over different datasets, such as CIFAR-10, CIFAR-100, and SVHN, against several competitive methods.

Item Type:	Conference or Workshop Item (Unspecified)
Divisions:	Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	12 Jul 2021 09:53
Last Modified:	19 Jun 2023 11:32
Related URLs:	Author
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3129749