Bollegala, Danushka ORCID: 0000-0003-4476-7003, Yoshida, Yuichi and Kawarabayashi, Ken-ichi
(2018)
Using k-Way Co-Occurrences for Learning Word Embeddings.
In: 32nd AAAI Conference on Artificial Intelligence, 2018-2-2 - 2018-2-7, New Orleans, Louisiana, USA.
This is the latest version of this item.
Text
context.pdf - Author Accepted Manuscript Download (581kB) |
|
Text
context.pdf - Author Accepted Manuscript Download (581kB) |
Abstract
<jats:p> Co-occurrences between two words provide useful insights into the semantics of those words.Consequently, numerous prior work on word embedding learning has used co-occurrences between two wordsas the training signal for learning word embeddings.However, in natural language texts it is common for multiple words to be related and co-occurring in the same context.We extend the notion of co-occurrences to cover k(≥2)-way co-occurrences among a set of k-words.Specifically, we prove a theoretical relationship between the joint probability of k(≥2) words, and the sum of l_2 norms of their embeddings. Next, we propose a learning objective motivated by our theoretical resultthat utilises k-way co-occurrences for learning word embeddings.Our experimental results show that the derived theoretical relationship does indeed hold empirically, anddespite data sparsity, for some smaller k(≤5) values, k-way embeddings perform comparably or better than 2-way embeddings in a range of tasks. </jats:p>
Item Type: | Conference or Workshop Item (Unspecified) |
---|---|
Uncontrolled Keywords: | Clinical Research |
Depositing User: | Symplectic Admin |
Date Deposited: | 18 Apr 2018 14:48 |
Last Modified: | 15 Mar 2024 02:25 |
DOI: | 10.1609/aaai.v32i1.12010 |
Related URLs: | |
URI: | https://livrepository.liverpool.ac.uk/id/eprint/3020270 |
Available Versions of this Item
-
Using k-way Co-occurrences for Learning Word Embeddings. (deposited 09 Jan 2018 10:12)
- Using k-Way Co-Occurrences for Learning Word Embeddings. (deposited 18 Apr 2018 14:48) [Currently Displayed]