A Neighbourhood-Aware Differential Privacy Mechanism for StaticWord Embeddings

Bollegala, Danushka ORCID: 0000-0003-4476-7003, Otake, Shuichi, Machide, Tomoya and Kawarabayashi, Ken-ichi (2023) A Neighbourhood-Aware Differential Privacy Mechanism for StaticWord Embeddings In: Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings), 2023-11 - 2023-11, Bali, Indonesia.

PDF Diff_Privacy__IJCNLP_2023_.pdf - Author Accepted Manuscript Download (1MB) | Preview

Abstract

We propose a Neighbourhood-Aware Differential Privacy (NADP) mechanism considering the neighbourhood of a word in a pretrained static word embedding space to determine the minimal amount of noise required to guarantee a specified privacy level. We first construct a nearest neighbour graph over the words using their embeddings, and factorise it into a set of connected components (i.e. neighbourhoods). We then separately apply different levels of Gaussian noise to the words in each neighbourhood, determined by the set of words in that neighbourhood. Experiments show that our proposed NADP mechanism consistently outperforms multiple previously proposed DP mechanisms such as Laplacian, Gaussian, and Mahalanobis in multiple downstream tasks, while guaranteeing higher levels of privacy.

Item Type:	Conference Item (Unspecified)
Divisions:	Faculty of Science & Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	09 Nov 2023 09:17
Last Modified:	16 Jun 2026 20:18
DOI:	10.18653/v1/2023.findings-ijcnlp.7
Related Websites:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3176697
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.