Ascent-Based Monte Carlo EM
Abstract
The EM algorithm is a popular tool for maximizing likelihood functions in the presence of missing data. Unfortunately, in many practical problems EM requires the evaluation of analytically intractable and high-dimensional integrals. The Monte Carlo EM (MCEM) algorithm is the natural extension of EM that employs Monte Carlo methods to estimate the relevant integrals. Typically, a very large Monte Carlo sample size is required to estimate these integrals within an acceptable tolerance when the algorithm is near convergence. Even if this sample size were known at the onset of implementation of MCEM, its use throughout all iterations is wasteful, especially when accurate starting values are not available. We propose a data-driven strategy for controlling Monte Carlo resources in MCEM. The proposed algorithm improves on similar existing methods by: (i) recovering EM’s ascent (i.e., likelihood-increasing) property with high probability, (ii) being more robust to the impact of user defined inputs, and (iii) handling classical Monte Carlo and Markov chain Monte Carlo within a common framework. Because of (i) we refer to the algorithm as “Ascent-based MCEM.” We apply Ascent-based MCEM to a variety of examples, including the one where it is used to dramatically accelerate the convergence of deterministic EM.