Fins, Ivo S 
ORCID: 0000-0002-1519-7550, Davies, Heather ORCID: 0000-0001-6905-4718, Farrell, Sean, Torres, Jose R, Pinchbeck, Gina ORCID: 0000-0002-5671-8623, Radford, Alan D and Noble, Peter-John ORCID: 0000-0002-2275-2014
(2024)
Evaluating ChatGPT text mining of clinical records for companion animal obesity monitoring.
The Veterinary record, 194 (3).
e3669-.
Abstract
<h4>Background</h4>Veterinary clinical narratives remain a largely untapped resource for addressing complex diseases. Here we compare the ability of a large language model (ChatGPT) and a previously developed regular expression (RegexT) to identify overweight body condition scores (BCS) in veterinary narratives pertaining to companion animals.<h4>Methods</h4>BCS values were extracted from 4415 anonymised clinical narratives using either RegexT or by appending the narrative to a prompt sent to ChatGPT, prompting the model to return the BCS information. Data were manually reviewed for comparison.<h4>Results</h4>The precision of RegexT was higher (100%, 95% confidence interval [CI] 94.81%-100%) than that of ChatGPT (89.3%, 95% CI 82.75%-93.64%). However, the recall of ChatGPT (100%, 95% CI 96.18%-100%) was considerably higher than that of RegexT (72.6%, 95% CI 63.92%-79.94%).<h4>Limitations</h4>Prior anonymisation and subtle prompt engineering are needed to improve ChatGPT output.<h4>Conclusions</h4>Large language models create diverse opportunities and, while complex, present an intuitive interface to information. However, they require careful implementation to avoid unpredictable errors.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | Animals, Obesity, Narration, Language, Data Mining, Pets |
| Divisions: | Faculty of Health and Life Sciences Faculty of Health and Life Sciences > Institute of Infection, Veterinary and Ecological Sciences |
| Depositing User: | Symplectic Admin |
| Date Deposited: | 21 Dec 2023 13:37 |
| Last Modified: | 02 Apr 2024 09:26 |
| DOI: | 10.1002/vetr.3669 |
| Open Access URL: | https://doi.org/10.1002/vetr.3669 |
| Related URLs: | |
| URI: | https://livrepository.liverpool.ac.uk/id/eprint/3177585 |
Dimensions
Dimensions
