Correcting crowdsourced annotations to improve detection of outcome types in evidence based medicine



Abaho, M, Bollegala, D ORCID: 0000-0003-4476-7003, Williamson, P ORCID: 0000-0001-9802-6636 and Dodd, S ORCID: 0000-0003-2851-3337
(2019) Correcting crowdsourced annotations to improve detection of outcome types in evidence based medicine. In: 4th International Workshop on Knowledge Discovery in Healthcare Data (KDH) at the 28th International Joint Conference on Artificial Intelligence, 2019-8-10 - 2019-8-16, Macao.

[thumbnail of abaho_kdh_2019.pdf] Text
abaho_kdh_2019.pdf - Author Accepted Manuscript

Download (159kB) | Preview

Abstract

The validity and authenticity of annotations in datasets massively influences the performance of Natural Language Processing (NLP) systems. In other words, poorly annotated datasets are likely to produce fatal results in at-least most NLP problems hence misinforming consumers of these models, systems or applications. This is a bottleneck in most domains, especially in healthcare where crowdsourcing is a popular strategy in obtaining annotations. In this paper, we present a framework that automatically corrects incorrectly captured annotations of outcomes, thereby improving the quality of the crowdsourced annotations. We investigate a publicly available dataset called EBM-NLP, built to power NLP tasks in support of Evidence based Medicine (EBM) primarily focusing on health outcomes.

Item Type: Conference or Workshop Item (Unspecified)
Depositing User: Symplectic Admin
Date Deposited: 24 Jun 2019 15:51
Last Modified: 21 Nov 2023 17:02
Related URLs:
URI: https://livrepository.liverpool.ac.uk/id/eprint/3047267