On the Curious Case of ℓ2 norm of Sense Embeddings

Zhou, Y and Bollegala, D ORCID: 0000-0003-4476-7003 (2022) On the Curious Case of ℓ2 norm of Sense Embeddings In: Findings of the Association for Computational Linguistics: EMNLP 2022, 2022-12 - 2022-12, Abu Dabi.

Text L2_norm_and_frequency_of_sense_embeddings__camera_ready_version_.pdf - Author Accepted Manuscript Download (1MB) | Preview

Abstract

We show that the ℓ<inf>2</inf> norm of a static sense embedding encodes information related to the frequency of that sense in the training corpus used to learn the sense embeddings. This finding can be seen as an extension of a previously known relationship for word embeddings to sense embeddings. Our experimental results show that, in spite of its simplicity, the ℓ<inf>2</inf> norm of sense embeddings is a surprisingly effective feature for several word sense related tasks such as (a) most frequent sense prediction, (b) Word-in-Context (WiC), and (c) Word Sense Disambiguation (WSD). In particular, by simply including the ℓ<inf>2</inf> norm of a sense embedding as a feature in a classifier, we show that we can improve WiC and WSD methods that use static sense embeddings.

Item Type:	Conference Item (Unspecified)
Depositing User:	Symplectic Admin
Date Deposited:	12 Oct 2022 08:15
Last Modified:	23 May 2026 06:55
DOI:	10.18653/v1/2022.findings-emnlp.449
Related Websites:	Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3165407
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.