Zhou, Yi 
ORCID: 0000-0001-7009-8515 and Bollegala, Danushka ORCID: 0000-0003-4476-7003
(2021)
Learning Sense-Specific Static Embeddings using Contextualised Word
Embeddings as a Proxy.
In: 35th Pacific Asia Conference on Language, Information and Computation (PACLIC 35), 2021-11-5 - 2021-11-7, China.
|
Text
Sense_Embedding_PACLIC_2021 (1).pdf - Author Accepted Manuscript Download (520kB) | Preview |
Abstract
Contextualised word embeddings generated from Neural Language Models (NLMs), such as BERT, represent a word with a vector that considers the semantics of the target word as well its context. On the other hand, static word embeddings such as GloVe represent words by relatively low-dimensional, memory- and compute-efficient vectors but are not sensitive to the different senses of the word. We propose Context Derived Embeddings of Senses (CDES), a method that extracts sense related information from contextualised embeddings and injects it into static embeddings to create sense-specific static embeddings. Experimental results on multiple benchmarks for word sense disambiguation and sense discrimination tasks show that CDES can accurately learn sense-specific static embeddings reporting comparable performance to the current state-of-the-art sense embeddings.
| Item Type: | Conference or Workshop Item (Unspecified) |
|---|---|
| Additional Information: | Accepted to PACLIC 35 |
| Uncontrolled Keywords: | cs.CL, cs.CL |
| Divisions: | Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science |
| Depositing User: | Symplectic Admin |
| Date Deposited: | 08 Oct 2021 15:12 |
| Last Modified: | 18 Jan 2023 21:27 |
| Related URLs: | |
| URI: | https://livrepository.liverpool.ac.uk/id/eprint/3139728 |
Share
Share
