Project #51:
Predicting risk of COVID-19 mortality: comparison of methods

Obtaining accurate estimates of risk is challenging in the context of changing levels of circulating infection, which standard risk prediction models commonly ignore.

We adopt a landmarking approach for risk prediction to dynamically incorporate time- and region-dependent information on infection prevalence. We compare this approach with more commonly used Cox models that do not take infection prevalence into account, in terms of discriminaton and calibration. We also compare simpler models to more richly specified ones.

The target population was adults registered at a general practice in England in March 2020. The outcome was 28-day COVID-19-related-death. Predictors included demographic characteristics and comorbidities. Three proxies of local infection prevalence were used: model-based estimates; rate of COVID-19-related attendances in emergency care; and rate of suspected COVID-19 cases in primary care.

Prediction models were developed on 11,972,947 individuals, of whom 7,999 experienced COVID-19-related-death. All models discriminated well between individuals who did and did not experience COVID-19 mortality, including simple models adjusting only for basic demographics and number of comorbidities: C-statistics 0.92-0.94. However, absolute risk estimates were substantially miscalibrated when infection prevalence was not explicitly modelled.

Our study suggests that models that ignore the infection prevalence provide poorly calibrated estimates of absolute risk; models that include time-varying measures of the infection prevalence can provide more accurate estimates. Simple models based only on number of comorbidities and basic demographics performed almost as well as more complex risk prediction models, both within models including infection prevalence and models that ignore this, suggesting that policies targeting population level reduction of COVID-19 mortality risk may not need to distinguish between all comorbidities in detail.