Sentence Matching for Question Answering with Neural Networks

Wu, J (2019) Sentence Matching for Question Answering with Neural Networks. Doctor of Philosophy thesis, University of Liverpool.

Text 200907265_Sep2018.pdf - Unspecified Download (1MB)

Abstract

Natural Language Processing is an important area of artificial intelligence concerned with the interactions between computers and human language. Semantic matching requires accurately modeling the relevance between two portions of text and is widely used in various natural language processing tasks, such as paraphrase identification, machine translation, and question answering. In the past years, several works have continually progressed towards improving the ability to capture and analyse the text matching information (e.g. lexical, syntactic, etc.) through the application of various techniques. In this thesis, we examine the problem of developing efficient and reliable semantic matching methods for question answering and exploring the effects of the different deep learning algorithms. We split our work into three pieces, where each part refers to a different sentence matching structure. In the first part of the thesis, we propose a deep semantic similarity model to learn a distributed similarity representation for sentences pairs. Text matching can use lexical information without any consideration of semantics. Semantic similarities can be determined from human-curated knowledge, but such knowledge may not be available in every language. The novelty of the proposed architecture lies in an abstract representation of the pairwise similarities created by deep denoising stacked auto-encoders. Model training is accomplished through a greedy layer-wise training scheme, that incorporates both supervised and unsupervised learning. The proposed model is experimentally compared to state-of-the-art approaches on two different dataset types: the TREC library and the Yahoo! community question datasets. The experimental results show the proposed model outperforming other approaches. In the second part of the thesis, we focus on designing a new question-answer matching model, built upon a cross-sentence, context-aware, bi-directional long short-term memory architecture. Semantic matching between question and answer sentences involves recognizing whether a candidate’s response is relevant to a particular input question. Given the fact that semantic matching does not examine a question or an answer individually, context information outside the sentence should be considered with equal emphasis to the within-sentence syntactic context. An interactive attention mechanism is proposed which automatically select salient positional answer representations, that contribute more significantly towards the relevance of an answer to a given question. In the experiments, the proposed method is compared with the existing models, using four public community datasets. The results state that the proposed model is very competitive. In particular, it offers 1.0%-2.2% improvement over the best performing model for three out of four datasets, while for the remaining one performance is around 0.5% of the best performer. In the last part of this thesis, a much more complex deep memory network structure is considered. In particular, we aim at developing a novel memory network for storing and reading the relevant question answer pairs from the internal corpus. Traditionally, to provide more related sentence information, the external resources (e.g. knowledge base, related documents) are widely used in question answering leading to large computational costs. Thus, our goal in this work is to build a reliable model that can utilize the input corpus to build the memory network without the knowledge-based resources. In the experiments, our proposed method indeed improves the matching performance on three library/community question answering datasets, when compared with those methods relying on memory structures with document resources, it achieves better performances compared with the state-of-arts.

Item Type:	Thesis (Doctor of Philosophy)
Divisions:	Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	21 Aug 2019 15:14
Last Modified:	19 Jan 2023 00:52
DOI:	10.17638/03039147
Supervisors:	Goulermas, John Yannis
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3039147