Jain, Raghav, Saha, Tulika 
ORCID: 0000-0002-3252-0997 and Saha, Sriparna
(2023)
T-VAKS: A Tutoring-based Multimodal Dialog System
via Knowledge Selection.
.
|
Text
ECAI_2023_TVAKS.pdf - Author Accepted Manuscript Download (1MB) | Preview |
Abstract
<jats:p>Advancements in Conversational Natural Language Processing (NLP) have the potential to address critical social challenges, particularly in achieving the United Nations’ Sustainable Development Goal of quality education. However, the application of NLP in the educational domain, especially language learning, has been limited due to the inherent complexities of the field and the scarcity of available datasets. In this paper, we introduce T-VAKS (Tutoring Virtual Agent with Knowledge Selection), a novel language tutoring multimodal Virtual Agent (VA) designed to assist students in learning a new language, thereby promoting AI for Social Good. T-VAKS aims to bridge the gap between NLP and the educational domain, enabling more effective language tutoring through intelligent virtual agents. Our approach employs an information theory-based knowledge selection module built on top of a multimodal seq2seq generative model, facilitating the generation of appropriate, informative, and contextually relevant tutor responses. The knowledge selection module in turn consists of two sub-modules: (i) knowledge relevance estimation, and (ii) knowledge focusing framework. We evaluate the performance of our proposed end-to-end dialog system against various baseline models and the most recent state-of-the-art models, using multiple evaluation metrics. The results demonstrate that T-VAKS outperforms competing models, highlighting the potential of our approach in enhancing language learning through the use of conversational NLP and virtual agents, ultimately contributing to addressing social challenges and promoting well-being.</jats:p>
| Item Type: | Conference or Workshop Item (Unspecified) |
|---|---|
| Divisions: | Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science |
| Depositing User: | Symplectic Admin |
| Date Deposited: | 13 Oct 2023 07:48 |
| Last Modified: | 15 Mar 2024 19:44 |
| DOI: | 10.3233/faia230388 |
| Related URLs: | |
| URI: | https://livrepository.liverpool.ac.uk/id/eprint/3173677 |
Dimensions
Dimensions
