Abstract
In logistic regression, separation refers to the situation in which a linear combination of predictors perfectly discriminates the binary outcome. Because finite-valued maximum likelihood parameter estimates do not exist under separation, Bayesian regressions with informative shrinkage of the regression coefficients offer a suitable alternative. Little focus has been given on whether and how to shrink the intercept parameter. Based upon classical studies of separation, we argue that efficiency in estimating regression coefficients may vary with the intercept prior. We adapt alternative prior distributions for the intercept that downweight implausibly extreme regions of the parameter space rendering less sensitivity to separation. Through simulation and the analysis of exemplar datasets, we quantify differences across priors stratified by established statistics measuring the degree of separation. Relative to diffuse priors, our recommendations generally result in more efficient estimation of the regression coefficients themselves when the data are nearly separated. They are equally efficient in non-separated datasets, making them suitable for default use. Modest differences were observed with respect to out-of-sample discrimination. Our work also highlights the interplay between priors for the intercept and the regression coefficients: numerical results are more sensitive to the choice of intercept prior when using a weakly informative prior on the regression coefficients than an informative shrinkage prior.
Disciplines
Applied Statistics | Categorical Data Analysis | Statistical Models
Suggested Citation
Boonstra, Philip S.; Barbaro, Ryan P.; and Sen, Ananda, "Default Priors for the Intercept Parameter in Logistic Regressions" (March 2018). The University of Michigan Department of Biostatistics Working Paper Series. Working Paper 123.
https://biostats.bepress.com/umichbiostat/paper123
Included in
Applied Statistics Commons, Categorical Data Analysis Commons, Statistical Models Commons