Prompt-based Zero-shot Text Classification with Conceptual Knowledge

Wang, Y, Wang, W, Chen, Q, Huang, K, Nguyen, A ORCID: 0000-0002-1449-211X and De, S (2023) Prompt-based Zero-shot Text Classification with Conceptual Knowledge. .

Text 2023.acl-srw.4444.pdf - Author Accepted Manuscript Download (697kB) | Preview

Abstract

In recent years, pre-trained language models have garnered significant attention due to their effectiveness, which stems from the rich knowledge acquired during pre-training. To mitigate the inconsistency issues between pre-training tasks and downstream tasks and to facilitate the resolution of language-related issues, prompt-based approaches have been introduced, which are particularly useful in low-resource scenarios. However, existing approaches mostly rely on verbalizers to translate the predicted vocabulary to task-specific labels. The major limitations of this approach are the ignorance of potentially relevant domain-specific words and being biased by the pre-training data. To address these limitations, we propose a framework that incorporates conceptual knowledge for text classification in the extreme zero-shot setting. The framework includes prompt-based keyword extraction, weight assignment to each prompt keyword, and final representation estimation in the knowledge graph embedding space. We evaluated the method on four widely-used datasets for sentiment analysis and topic detection, demonstrating that it consistently outperforms recently-developed prompt-based approaches in the same experimental settings.

Item Type:	Conference or Workshop Item (Unspecified)
Divisions:	Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	23 Oct 2023 09:51
Last Modified:	23 Oct 2023 09:51
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3175779