Unsupervised Evaluation of Human Translation Quality

Zhou, Yi ORCID: 0000-0001-7009-8515 and Bollegala, Danushka ORCID: 0000-0003-4476-7003 (2019) Unsupervised Evaluation of Human Translation Quality. In: 11th International Conference on Knowledge Discovery and Information Retrieval, 2019-9-17 - 2019-9-19.

Text Unsupervised Evalusation of Human Translation Quality.pdf - Author Accepted Manuscript Download (1MB) | Preview

Abstract

Even though machine translation (MT) systems have reached impressive performances in cross-lingual translation tasks, the quality of MT is still far behind professional human translations (HTs) due to the complexity in natural languages, especially for terminologies in different domains. Therefore, HTs are still widely demanded in practice. However, the quality of HT is also imperfect and vary significantly depending on the experience and knowledge of the translators. Evaluating the quality of HT in an automatic manner has faced many challenges. Although bilingual speakers are able to assess the translation quality, manually checking the accuracy of translations is expensive and time-consuming. In this paper, we propose an unsupervised method to evaluate the quality of HT without requiring any labelled data. We compare a range of methods for automatically grading HTs and observe the Bidirectional Minimum Word Mover’s distance (BiMWMD) to produce gradings that correlate well with humans.

Item Type:	Conference or Workshop Item (Unspecified)
Uncontrolled Keywords:	Translation Quality, Evaluation of Human Translations, Cross-lingual Word Embeddings, Word Mover's Distance, Bidirectional Minimum Word Mover's Distance
Depositing User:	Symplectic Admin
Date Deposited:	17 Aug 2020 08:41
Last Modified:	18 Jan 2023 23:37
DOI:	10.5220/0008064500550064
Related URLs:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3097319