Controversies in prediction modeling

Prof Ben Van Calster gave the following talk discussing the "enemies" of reliable risk prediction

 

 

 

 

 

 

Prof Frank Harrell Jr gave the following talk discussing controversies in prediction modeling, considering statistical methods and machine learning (talk starts at 4 mins 25 secs)

Prof Gary Collins gave a talk entitled "Clinical prediction models: a field in crisis", in which he outlines concerns about the quality and methodological standards of current prediction model research. It can be viewed here.

Prof Nan van Geloven gave the following talk "Why most prediction models cannot support treatment decisions". Currently, in the development of prediction models ad hoc approaches are used for patients who start a treatment. Patients may for instance be excluded or censored or the treatments are ignored. Consequences of such analysis choices on the interpretation of the calculated risks are typically neglected which may lead to wrongly interpreted risks. In particular, risks may not provide the input to treatment decisions they are expected to give. In this talk, Nan proposes 'predictimand' framework to deal with treatment in clinical prediction models. Analogous to the ICH estimand guideline for clinical trials, the ‘predictimand' framework is comprised of different questions that may be of interest when predicting risk in relation to treatment started after baseline. Nan provides definitions of the estimands matching these questions, giving examples of settings in which each is useful.