Abstract
Many applications aim to learn a high dimensional parameter of a data generating distribution based on a sample of independent and identically distributed observations. For example, the goal might be to estimate the conditional mean of an outcome given a list of input variables. In this prediction context, Breiman (1996a) introduced bootstrap aggregating (bagging) as a method to reduce the variance of a given estimator at little cost to bias. Bagging involves applying the estimator to multiple bootstrap samples, and averaging the result across bootstrap samples. In order to deal with the curse of dimensionality, typical practice has been to apply bagging to estimators which themselves use cross-validation, thereby using cross-validation within a bootstrap sample to select fine-tuning parameters trading off bias and variance of the bootstrap sample-specific candidate estimators. In this article we point out that in order to achieve the correct bias variance trade-off for the parameter of interest, one should apply the cross-validation selector externally to candidate bagged estimators indexed by these fine-tuning parameters. In addition we define variable importance as a summary measure of the parameter of interest, and present a novel bootstrap method to achieve statistical inference and p-values based on the (externally) cross-validated bagged estimator. We illustrate the new cross-validated bagging method with a data analysis and investigate the performance of the variable importance measures in a small simulation study.
Disciplines
Statistical Methodology | Statistical Models | Statistical Theory
Suggested Citation
van der Laan, Mark J.; Sinisi, Sandra E.; and Petersen, Maya L., "Cross-validated Bagged Learning" (June 2005). U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 182.
https://biostats.bepress.com/ucbbiostat/paper182