Evaluating ChatGPT text mining of clinical records for companion animal obesity monitoring

Fins, Ivo S, Davies, Heather, Farrell, Sean, Torres, Jose R, Pinchbeck, Gina ORCID: 0000-0002-5671-8623, Radford, Alan D ORCID: 0000-0002-4590-1334 and Noble, Peter-John ORCID: 0000-0002-2275-2014 (2024) Evaluating ChatGPT text mining of clinical records for companion animal obesity monitoring VETERINARY RECORD, 194 (3). e3669-. ISSN 0042-4900, 2042-7670

Access the full-text of this item by clicking on the Open Access link.

Abstract

Background: Veterinary clinical narratives remain a largely untapped resource for addressing complex diseases. Here we compare the ability of a large language model (ChatGPT) and a previously developed regular expression (RegexT) to identify overweight body condition scores (BCS) in veterinary narratives pertaining to companion animals. Methods: BCS values were extracted from 4415 anonymised clinical narratives using either RegexT or by appending the narrative to a prompt sent to ChatGPT, prompting the model to return the BCS information. Data were manually reviewed for comparison. Results: The precision of RegexT was higher (100%, 95% confidence interval [CI] 94.81%–100%) than that of ChatGPT (89.3%, 95% CI 82.75%–93.64%). However, the recall of ChatGPT (100%, 95% CI 96.18%–100%) was considerably higher than that of RegexT (72.6%, 95% CI 63.92%–79.94%). Limitations: Prior anonymisation and subtle prompt engineering are needed to improve ChatGPT output. Conclusions: Large language models create diverse opportunities and, while complex, present an intuitive interface to information. However, they require careful implementation to avoid unpredictable errors.

Item Type:	Article
Uncontrolled Keywords:	Animals, Obesity, Narration, Language, Data Mining, Pets
Divisions:	Faculty of Health & Life Sciences Faculty of Health & Life Sciences > Inst. Infection, Vet & Ecological Sciences
Depositing User:	Symplectic Admin
Date Deposited:	21 Dec 2023 13:37
Last Modified:	22 May 2026 19:55
DOI:	10.1002/vetr.3669
Open Access URL:	https://doi.org/10.1002/vetr.3669
Related Websites:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3177585
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.