Abstract
Background Net Reclassification Indices (NRI) have recently become popular statistics for measuring the prediction increment of new biomarkers.
Methods In this review, we examine the various types of NRI statistics and their correct interpretations. We evaluate the advantages and disadvantages of the NRI approach. For pre-defined risk categories, we relate NRI to existing measures of the prediction increment. We also consider statistical methodology for constructing confidence intervals for NRI statistics and evaluate the merits of NRI-based hypothesis testing.
Conclusions Investigators using NRI statistics should report them separately for events (cases) and nonevents (controls). When there are two risk categories, the NRI components are the same as the changes in the true and false positive rates. We advocate use of true and false positive rates and suggest it is more useful for investigators to retain the existing, descriptive terms. When there are three or more risk categories, we recommend against NRI statistics because they do not adequately account for clinically important differences in movements among risk categories. The category-free NRI is a new descriptive device designed to avoid pre-defined risk categories. The category-free NRI suffers from many of the same problems as other measures such as the area under the receiver operating characteristic curve. In addition, the category-free NRI can mislead investigators by overstating the incremental value of a biomarker, even in independent validation data. When investigators want to test a null hypothesis of no prediction increment, the well-established tests for coefficients in the regression model are superior to the NRI. If investigators want to use NRI measures, their confidence intervals should be calculated using bootstrap methods rather than published variance formulas. The preferred single-number summary of the prediction increment is the improvement in the Net Benefit.
Disciplines
Biostatistics | Clinical Epidemiology
Suggested Citation
Kerr, Kathleen F.; Wang, Zheyu; Janes, Holly; McClelland, Robyn; Psaty, Bruce M.; and Pepe, Margaret S., "Net Reclassification Indices for Evaluating Risk Prediction Instruments: A Critical Review" (August 2013). UW Biostatistics Working Paper Series. Working Paper 393.
https://biostats.bepress.com/uwbiostat/paper393
Comments
Financial Support: The MESA study was supported by contracts N01-HC-95159 through N01-HC-95169 from the National Heart, Lung, and Blood Institute. This work was also supported by grant NIH GM054438 to MSP, grant NIH R01 CA152089 to HJ, and a subcontract to the University of Washington from NIH grant HL085757-07 to KFK.