Learning Linear Transformations between Counting-based and Prediction-based Word Embeddings

Bollegala, D ORCID: 0000-0003-4476-7003, Hayashi, Kohei and Kawarabayashi, Ken-ichi
(2017) Learning Linear Transformations between Counting-based and Prediction-based Word Embeddings. PLoS One, 12 (9).

[img] Text
linear-trans.pdf - Submitted Version

Download (327kB)


Despite the growing interest in prediction-based word embedding learning methods, it remains unclear as to how the vector spaces learnt by the prediction-based methods differ from that of the counting-based methods, or whether one can be transformed into the other. To study the relationship between counting-based and prediction-based embeddings, we propose a method for learning a linear transformation between two given sets of word embeddings. Our proposal contributes to the word embedding learning research in three ways: (a) we propose an efficient method to learn a linear transformation between two sets of word embeddings, (b) using the transformation learnt in (a), we empirically show that it is possible to predict distributed word embeddings for novel unseen words, and (c) empirically it is possible to linearly transform counting-based embeddings to prediction-based embeddings, for frequent words, different POS categories, and varying degrees of ambiguities.

Item Type: Article
Depositing User: Symplectic Admin
Date Deposited: 05 Sep 2017 08:58
Last Modified: 27 Sep 2021 20:13
DOI: 10.1371/journal.pone.0184544
Open Access URL: http://doi.org/10.1371/journal.pone.0184544
Related URLs:
URI: https://livrepository.liverpool.ac.uk/id/eprint/3009299