Developing clinical prediction models for nominal polytomous outcomes: a simulation study comparing available approaches


Pate, Alexander ORCID: 0000-0002-0849-3458, Riley, Richard, Sperrin, Matthew, Calster, Ben Van, Sergeant, Jamie, Peek, Niels, Staa, Tjeerd van, Mamas, Mamas, Lip, Gregory, O'Flaherty, Martin ORCID: 0000-0001-8944-4131 et al (show 2 more authors) (2023) Developing clinical prediction models for nominal polytomous outcomes: a simulation study comparing available approaches [Preprint]

Access the full-text of this item by clicking on the Open Access link.

Abstract

<title>Abstract</title><p>Background There are numerous methods available to develop clinical prediction models to estimate the risks of a nominal polytomous outcome. A comprehensive evaluation of the most appropriate method has not yet been undertaken. We compared the predictive performance of a range of models in a simulation study and illustrate how to implement them with shrinkage in a clinical example. Methods Performance of models [One-versus-All with normalisation (OvA-N), One-versus-One with pairwise coupling (OvO-PC), two types of continuation ratio regression (c-ratio and c-ratio-all) and multinomial logistic regression (MLR)] was evaluated by calibration, discrimination and magnitude of overfitting. We considered two data generating mechanisms and 4 underlying data structures to allow us to evaluate how robust each method was to model mis-specification. Results At large sample sizes OvO-PC and MLR had comparative calibration across all scenarios. When the models were misspecified, MLR and OvO-PC had the best calibration, followed by c-ratio-all, and then c-ratio. Discrimination of all methods was similar across most scenarios, however c-ratio had poor discrimination in certain scenarios. At small sample sizes MLR and c-ratio had a similar level of overfitting, and OvO-PC suffered from the highest levels of overfitting. MLR and c-ratio-all had the best calibration, followed by OvO-PC and then c-ratio. OvA-N had the worst performance at both large and small sample sizes. Conclusions We recommend MLR for developing clinical predictions models for the prediction of polytomous outcomes as it was the most robust to model misspecification and suffered from the joint smallest levels of overfitting.</p>

Item Type: Preprint
Uncontrolled Keywords: 49 Mathematical Sciences, 4905 Statistics
Divisions: Faculty of Health & Life Sciences
Faculty of Health & Life Sciences > Inst. Population Health
Depositing User: Symplectic Admin
Date Deposited: 20 Nov 2024 12:17
Last Modified: 20 Mar 2026 10:16
DOI: 10.21203/rs.3.rs-3121017/v1
Open Access URL: https://www.researchsquare.com/article/rs-3121017/...
Related Websites:
URI: https://livrepository.liverpool.ac.uk/id/eprint/3188775
Disclaimer: The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.
Altmetric
CORE (COnnecting REpositories)