Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

Модели прогнозирования для диагностики и прогноза COVID-19: систематический обзор и критическая оценка
Ewoud Schuit, Johannes B. Reitsma, Karel G.M. Moons, Ewout Willem Steyerberg, Marc J. M. Bonten, Matthew Sperrin, Richard D Riley, Sander M. J. van Kuijk, Maarten van Smeden, Maarten De Vos, Professor Gary S. Collins, Jie Ma, Pauline Heus, Johanna AAG Damen, Lotty Hooft, Ioanna Tzoulaki, Brooke Levis, Nicole Skoetz, Joie Ensor, Kym I E Snell, Thomas P. A. Debray, Jack Wilkinson, Robert F. Wolff, Paula Dhiman, Ben VAN CALSTER, Georg Heinze, Laure Wynants, Jan Yvan Jos Verbakel, Glen Philip Martin, Liesbet Henckaerts, Luc J.M. Smits, Iwan C.C. van der Horst, René Spijker, Darren Lawrence Dahly, Jeroen Hoogland, Michael Oscar Harhay, Elena Albu, Banafsheh Arshi, Vanesa Bellou, Valentijn M. T. de Jong, Shan Gao, Maria C. Haller, Mohammed T Hudda, Kevin Jenniskens, Michael Kammer, Nina Kreuzberger, Anna Lohmann, Kim Luijken, David J. McLernon, Constanza L. Andaur Navarro, Jamie C. Sergeant, Chunhu Shi, Toshihiko Takada, Bas C. T. van Bussel, Kelly A. Reeve, Florien Sophie-Anne van Royen, Christine Wallisch
2020-04-07

COVID-19 prediction modelsDiagnostic modelsPROBASTPrognostic modelsSystematic review
Abstract Objective To review and appraise the validity and usefulness of published and preprint reports of prediction models for prognosis of patients with covid-19, and for detecting people in the general population at increased risk of covid-19 infection or being admitted to hospital or dying with the disease. Design Living systematic review and critical appraisal by the covid-PRECISE (Precise Risk Estimation to optimise covid-19 Care for Infected or Suspected patients in diverse sEttings) group. Data sources PubMed and Embase through Ovid, up to 17 February 2021, supplemented with arXiv, medRxiv, and bioRxiv up to 5 May 2020. Study selection Studies that developed or validated a multivariable covid-19 related prediction model. Data extraction At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool). Results 126 978 titles were screened, and 412 studies describing 731 new prediction models or validations were included. Of these 731, 125 were diagnostic models (including 75 based on medical imaging) and the remaining 606 were prognostic models for either identifying those at risk of covid-19 in the general population (13 models) or predicting diverse outcomes in those individuals with confirmed covid-19 (593 models). Owing to the widespread availability of diagnostic testing capacity after the summer of 2020, this living review has now focused on the prognostic models. Of these, 29 had low risk of bias, 32 had unclear risk of bias, and 545 had high risk of bias. The most common causes for high risk of bias were inadequate sample sizes (n=408, 67%) and inappropriate or incomplete evaluation of model performance (n=338, 56%). 381 models were newly developed, and 225 were external validations of existing models. The reported C indexes varied between 0.77 and 0.93 in development studies with low risk of bias, and between 0.56 and 0.78 in external validations with low risk of bias. The Qcovid models, the PRIEST score, Carr’s model, the ISARIC4C Deterioration model, and the Xie model showed adequate predictive performance in studies at low risk of bias. Details on all reviewed models are publicly available at https://www.covprecise.org/ . Conclusion Prediction models for covid-19 entered the academic literature to support medical decision making at unprecedented speed and in large numbers. Most published prediction model studies were poorly reported and at high risk of bias such that their reported predictive performances are probably optimistic. Models with low risk of bias should be validated before clinical implementation, preferably through collaborative efforts to also allow an investigation of the heterogeneity in their performance across various populations and settings. Methodological guidance, as provided in this paper, should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, prediction modellers should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline. Systematic review registration Protocol https://osf.io/ehc47/ , registration https://osf.io/wy245 . Readers’ note This article is the final version of a living systematic review that has been updated over the past two years to reflect emerging evidence. This version is update 4 of the original article published on 7 April 2020 ( BMJ 2020;369:m1328). Previous updates can be found as data supplements ( https://www.bmj.com/content/369/bmj.m1328/related#datasupp ). When citing this paper please consider adding the update number and date of access for clarity.
1
Common predictors included vital signs, age, comorbidities, and imaging features; flu-like symptoms were frequent diagnostic predictors, while sex, C-reactive protein, and lymphocyte counts were frequent prognostic factors.
2
Diagnostic models predominantly used medical imaging, whereas prognostic models predicted mortality, severe disease progression, intensive care admission, ventilation, intubation, or hospital stay duration.
3
Reported C-index values from the strongest available validation ranged from 0.71 to 0.99, although the abstract emphasizes critical appraisal of model validity and usefulness.
4
Seven models targeted infection or hospitalization risk in the general population, while 118 diagnostic and 107 prognostic models addressed patients with suspected or confirmed COVID-19.
5
The living systematic review included 169 studies describing 232 COVID-19 prediction models identified from 37,421 screened titles.

Published and preprint multivariable COVID-19 prediction models for diagnosis, prognosis, and population risk detection

Validity, predictive performance, and risk of bias of models for detecting COVID-19, predicting clinical outcomes, and identifying people at increased risk

Publication Details
Publication Date
2020-04-07
Journal
Publisher
ISSN
Cited by
3062
Access Type
Author Information
Authors
Ewoud Schuit
Johannes B. Reitsma
Karel G.M. Moons
Ewout Willem Steyerberg
Marc J. M. Bonten
Matthew Sperrin
Richard D Riley
Sander M. J. van Kuijk
Maarten van Smeden
Maarten De Vos
Professor Gary S. Collins
Jie Ma
Pauline Heus
Johanna AAG Damen
Lotty Hooft
Ioanna Tzoulaki
Brooke Levis
Nicole Skoetz
Joie Ensor
Kym I E Snell
Thomas P. A. Debray
Jack Wilkinson
Robert F. Wolff
Paula Dhiman
Ben VAN CALSTER
Georg Heinze
Laure Wynants
Jan Yvan Jos Verbakel
Glen Philip Martin
Liesbet Henckaerts
Luc J.M. Smits
Iwan C.C. van der Horst
René Spijker
Darren Lawrence Dahly
Jeroen Hoogland
Michael Oscar Harhay
Elena Albu
Banafsheh Arshi
Vanesa Bellou
Valentijn M. T. de Jong
Shan Gao
Maria C. Haller
Mohammed T Hudda
Kevin Jenniskens
Michael Kammer
Nina Kreuzberger
Anna Lohmann
Kim Luijken
David J. McLernon
Constanza L. Andaur Navarro
Jamie C. Sergeant
Chunhu Shi
Toshihiko Takada
Bas C. T. van Bussel
Kelly A. Reeve
Florien Sophie-Anne van Royen
Christine Wallisch
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%