The importance of prediction model validation and assessment in obesity and nutrition research

Andrada Ivanescu, P. Li, B. George, A. W. Brown, S. W. Keith, D. Raju, D. B. Allison

Research output: Contribution to journalReview article

24 Citations (Scopus)

Abstract

Deriving statistical models to predict one variable from one or more other variables, or predictive modeling, is an important activity in obesity and nutrition research. To determine the quality of the model, it is necessary to quantify and report the predictive validity of the derived models. Conducting validation of the predictive measures provides essential information to the research community about the model. Unfortunately, many articles fail to account for the nearly inevitable reduction in predictive ability that occurs when a model derived on one data set is applied to a new data set. Under some circumstances, the predictive validity can be reduced to nearly zero. In this overview, we explain why reductions in predictive validity occur, define the metrics commonly used to estimate the predictive validity of a model (for example, coefficient of determination (R 2), mean squared error, sensitivity, specificity, receiver operating characteristic and concordance index) and describe methods to estimate the predictive validity (for example, cross-validation, bootstrap, and adjusted and shrunken R 2). We emphasize that methods for estimating the expected reduction in predictive ability of a model in new samples are available and this expected reduction should always be reported when new predictive models are introduced.

Original languageEnglish
Pages (from-to)887-894
Number of pages8
JournalInternational Journal of Obesity
Volume40
Issue number6
DOIs
StatePublished - 1 Jun 2016

Fingerprint

Obesity
Statistical Models
Research
ROC Curve
Sensitivity and Specificity
Datasets

Cite this

Ivanescu, Andrada ; Li, P. ; George, B. ; Brown, A. W. ; Keith, S. W. ; Raju, D. ; Allison, D. B. / The importance of prediction model validation and assessment in obesity and nutrition research. In: International Journal of Obesity. 2016 ; Vol. 40, No. 6. pp. 887-894.
@article{335b58310218449d89162c3e5bd25de9,
title = "The importance of prediction model validation and assessment in obesity and nutrition research",
abstract = "Deriving statistical models to predict one variable from one or more other variables, or predictive modeling, is an important activity in obesity and nutrition research. To determine the quality of the model, it is necessary to quantify and report the predictive validity of the derived models. Conducting validation of the predictive measures provides essential information to the research community about the model. Unfortunately, many articles fail to account for the nearly inevitable reduction in predictive ability that occurs when a model derived on one data set is applied to a new data set. Under some circumstances, the predictive validity can be reduced to nearly zero. In this overview, we explain why reductions in predictive validity occur, define the metrics commonly used to estimate the predictive validity of a model (for example, coefficient of determination (R 2), mean squared error, sensitivity, specificity, receiver operating characteristic and concordance index) and describe methods to estimate the predictive validity (for example, cross-validation, bootstrap, and adjusted and shrunken R 2). We emphasize that methods for estimating the expected reduction in predictive ability of a model in new samples are available and this expected reduction should always be reported when new predictive models are introduced.",
author = "Andrada Ivanescu and P. Li and B. George and Brown, {A. W.} and Keith, {S. W.} and D. Raju and Allison, {D. B.}",
year = "2016",
month = "6",
day = "1",
doi = "10.1038/ijo.2015.214",
language = "English",
volume = "40",
pages = "887--894",
journal = "International Journal of Obesity",
issn = "0307-0565",
publisher = "Nature Publishing Group",
number = "6",

}

The importance of prediction model validation and assessment in obesity and nutrition research. / Ivanescu, Andrada; Li, P.; George, B.; Brown, A. W.; Keith, S. W.; Raju, D.; Allison, D. B.

In: International Journal of Obesity, Vol. 40, No. 6, 01.06.2016, p. 887-894.

Research output: Contribution to journalReview article

TY - JOUR

T1 - The importance of prediction model validation and assessment in obesity and nutrition research

AU - Ivanescu, Andrada

AU - Li, P.

AU - George, B.

AU - Brown, A. W.

AU - Keith, S. W.

AU - Raju, D.

AU - Allison, D. B.

PY - 2016/6/1

Y1 - 2016/6/1

N2 - Deriving statistical models to predict one variable from one or more other variables, or predictive modeling, is an important activity in obesity and nutrition research. To determine the quality of the model, it is necessary to quantify and report the predictive validity of the derived models. Conducting validation of the predictive measures provides essential information to the research community about the model. Unfortunately, many articles fail to account for the nearly inevitable reduction in predictive ability that occurs when a model derived on one data set is applied to a new data set. Under some circumstances, the predictive validity can be reduced to nearly zero. In this overview, we explain why reductions in predictive validity occur, define the metrics commonly used to estimate the predictive validity of a model (for example, coefficient of determination (R 2), mean squared error, sensitivity, specificity, receiver operating characteristic and concordance index) and describe methods to estimate the predictive validity (for example, cross-validation, bootstrap, and adjusted and shrunken R 2). We emphasize that methods for estimating the expected reduction in predictive ability of a model in new samples are available and this expected reduction should always be reported when new predictive models are introduced.

AB - Deriving statistical models to predict one variable from one or more other variables, or predictive modeling, is an important activity in obesity and nutrition research. To determine the quality of the model, it is necessary to quantify and report the predictive validity of the derived models. Conducting validation of the predictive measures provides essential information to the research community about the model. Unfortunately, many articles fail to account for the nearly inevitable reduction in predictive ability that occurs when a model derived on one data set is applied to a new data set. Under some circumstances, the predictive validity can be reduced to nearly zero. In this overview, we explain why reductions in predictive validity occur, define the metrics commonly used to estimate the predictive validity of a model (for example, coefficient of determination (R 2), mean squared error, sensitivity, specificity, receiver operating characteristic and concordance index) and describe methods to estimate the predictive validity (for example, cross-validation, bootstrap, and adjusted and shrunken R 2). We emphasize that methods for estimating the expected reduction in predictive ability of a model in new samples are available and this expected reduction should always be reported when new predictive models are introduced.

UR - http://www.scopus.com/inward/record.url?scp=84946106320&partnerID=8YFLogxK

U2 - 10.1038/ijo.2015.214

DO - 10.1038/ijo.2015.214

M3 - Review article

C2 - 26449421

AN - SCOPUS:84946106320

VL - 40

SP - 887

EP - 894

JO - International Journal of Obesity

JF - International Journal of Obesity

SN - 0307-0565

IS - 6

ER -