Low-bit Quantization Needs Good Distribution

Yu, Haibao, Wen, Tuopu, Cheng, Guangliang ORCID: 0000-0001-8686-9513, Sun, Jiankai, Han, Qi and Shi, Jianping (2020) Low-bit Quantization Needs Good Distribution. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020-6-14 - 2020-6-19.

Access the full-text of this item by clicking on the Open Access link.

Official URL: http://dx.doi.org/10.1109/cvprw50498.2020.00348

Abstract

Low-bit quantization is challenging to maintain high performance with limited model capacity (e.g., 4-bit for both weights and activations). Naturally, the distribution of both weights and activations in deep neural network are Gaussian-like. Nevertheless, due to the limited bitwidth of low-bit model, uniform-like distributed weights and activations have been proved to be more friendly to quantization while preserving accuracy. Motivated by this, we propose Scale-Clip, a Distribution Reshaping technique that can reshape weights or activations into a uniform-like distribution in a dynamic manner. Furthermore, to increase the model capability for a low-bit model, a novel Group- based Quantization algorithm is proposed to split the filters into several groups. Different groups can learn different quantization parameters, which can be elegantly merged into batch normalization layer without extra computational cost in the inference stage. Finally, we integrate Scale-Clip technique with Group-based Quantization algorithm and propose the Group-based Distribution Reshaping Quantization (GDRQ) framework to further improve the quantization performance. Experiments on various networks (e.g. VGGNet and ResNet) and vision tasks (e.g. classification, detection, and segmentation) demonstrate that our framework achieves much better performance than state-of-the- art quantization methods. Specifically, the ResNet-50 model with 2-bit weights and 4-bit activations obtained by our framework achieves less than 1% accuracy drop on ImageNet classification task, which is a new state-of-the-art to our best knowledge.

Item Type:	Conference or Workshop Item (Unspecified)
Divisions:	Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	14 Mar 2023 09:19
Last Modified:	24 Apr 2024 11:30
DOI:	10.1109/cvprw50498.2020.00348
Open Access URL:	https://openaccess.thecvf.com/content_CVPRW_2020/p...
Related URLs:	Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3169020