Variance parameters in mixed or multilevel models can be difficult to estimate, especially when the number of groups is small. We propose a maximum penalized likelihood approach which is equivalent to estimating variance parameters by their marginal posterior mode, given a weakly informative prior distribution. By choosing the prior from the gamma family with at least 1 degree of freedom, we ensure that the prior density is zero at the boundary and thus the marginal posterior mode of the group-level variance will be positive. The use of a weakly informative prior allows us to stabilize our estimates while remaining faithful to the data.
Statistical Methodology | Statistical Theory
Chung, Yeojin; Rabe-Hesketh, Sophia; Gelman, Andrew; Liu, Jingchen; and Dorie, Vincent, "Avoiding Boundary Estimates in Linear Mixed Models Through Weakly Informative Priors" (February 2012). U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 284.