Prognostic model research

Sample size for developing a prognostic model

​

  • Calculating the sample size required for developing a clinical prediction model (PDF)

​​

  • A note on estimating the Cox‐Snell R2 from a reported C statistic (AUROC) to inform sample size calculations for developing a prediction model with a binary outcome (PDF)

  • ​Minimum sample size for developing a multivariable prediction model: PART I ‐ continuous outcomes  (PDF)

  • Minimum sample size for developing a multivariable prediction model: PART II ‐ binary & time‐to‐event outcomes (PDF)

​

  • Sample size for binary logistic prediction models: Beyond events per variable criteria (PDF)

  • No rationale for 1 variable per 10 events criterion for binary logistic regression analysis (PDF)

​​

  • The problems with using a split-sample for model development and validation (blog)

​​

  • Adaptive sample size determination for the development of clinical prediction models (PDF)

​

  • How Can Machine Learning be Reliable When the Sample is Adequate for Only One Feature? (blog)

  • Modern Modelling Techniques Are Data Hungry: A Simulation Study for Predicting Dichotomous Endpoints (PDF)

​

Videos on sample size for model development available here

​

"Why the EPV ≥ 10 sample size rule is rubbish and what to use instead" - slides by Dr Maarten van Smeden available here

​

Sample size for external validation of a prognostic model

​

  • Minimum sample size for external validation of a clinical prediction model with a binary outcome (in-press, Stat Med)

  • External validation of clinical prediction models: simulation-based sample size calculations were more reliable than rules of thumb (PDF)

  • Minimum sample size for external validation of a clinical prediction model with a continuous outcome (PDF)

(also see corresponding videos here)​

​​

  • Sample size considerations for the external validation of a multivariable prognostic model: a resampling study  (PDF)

  • A calibration hierarchy for risk models was defined: from utopia to empirical data (PDF)

  • Substantial effective sample sizes were required for external validation studies of predictive logistic regression models (PDF)

  • Discrimination-based sample size calculations for multivariable prognostic models for time-to-event data (PDF)

​

Stages of prognostic model research
​

  • Prognosis and prognostic research: what, why, and how?  (PDF)

  • Prognosis and prognostic research: developing a prognostic model (PDF) 

  • Prognosis and prognostic research: Validating a prognostic model (PDF)

  • Prognosis and prognostic research: application and impact of prognostic models in clinical practice (PDF)

​

  • Guide to presenting clinical prediction models for use in clinical settings (PDF)

  • Presentation of multivariate data for clinical use: The Framingham Study risk score functions (PDF)

​

Improving prognostic model research

​​

​​

  • Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, ... (PDF)

  • Towards better clinical prediction models: seven steps for development and an ABCD for validation (PDF)

  • Everything you always wanted to know about evaluating prediction models (but were too afraid to ask) (PDF)

​

  • Variable selection – A review and recommendations for the practicing statistician (PDF)

​

Notes of caution:

​

  • Penalization and shrinkage methods produced unreliable clinical prediction models especially when sample size was small (PDF) (also see video here)

  • Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis (PDF)

  • Poor performance of clinical prediction models: the harm of commonly applied methods (PDF)

  • Regression shrinkage methods for clinical prediction models do not guarantee improved performance ... (PDF)

  • Three myths about risk thresholds for prediction models (PDF)

  • Quantifying the impact of different approaches for handling continuous predictors on the performance of a prognostic model (PDF)

​

Video on controversies in prediction modelling using statistical methods and machine learning available here

Video on COVID-19 related prediction models available here

​

Evaluating the performance of a prognostic model

​

  • A calibration hierarchy for risk models was defined: from utopia to empirical data (PDF)

  • Calibration: the Achilles heel of predictive analytics (PDF)

​​

  • Internal validation of predictive models: efficiency of some procedures for logistic regression analysis (PDF)

  • Prediction models need appropriate internal, internal-external, and external validation (PDF)

  • Construction and validation of a prognostic model across several studies ... (PDF)

​​

  • Assessment of predictive performance in incomplete data by combining internal validation & multiple imputation (PDF)

​​

  • Validation and updating of risk models based on multinomial logistic regression (PDF)

  • Assessing calibration of multinomial risk prediction models (PDF)

  • A spline-based tool to assess and visualize the calibration of multiclass risk predictions (PDF)

​

  • External validation of a Cox prognostic model: principles and methods (PDF)

  • Tools for checking calibration of a Cox model in external validation: Approach based on individual event probabilities (PDF)

  • Graphical calibration curves and the integrated calibration index (ICI) for survival models (PDF)

  • External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis (PDF)

​

  • Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests (PDF)

  • Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers (PDF)

  ​

Improving prognostic survival models

​

  • Temporal recalibration for improving prognostic model development and risk predictions ... (PDF)

  • Prognostic Models With Competing Risks: Methods and Application to Coronary Risk Prediction (PDF)

  • Validation, calibration, revision and combination of prognostic survival models (PDF)

  • Flexible Parametric Survival Analysis Using Stata: Beyond the Cox Model (PDF)

​
​​
 
 
​
 
 
​
​
​