Abstract
This paper proposes a statistical methodology for comparing the performance of stochastic optimization algorithms that iteratively generate candidate optima. The fundamental data structure of the results of these algorithms is a time series. Algorithmic differences may be assessed through a procedure of statistical sampling and multiple hypothesis testing of time series data. Shilane et al. propose a general framework for performance comparison of stochastic optimization algorithms that result in a single candidate optimum. This project seeks to extend this framework to assess performance in time series data structures. The proposed methodology analyzes empirical data to determine the generation intervals in which algorithmic performance differences exist and may be used to guide the selection and design of optimization procedures for the task at hand. Such comparisons may be drawn for general performance metrics of any iterative stochastic optimization algorithm under any (typically unknown) data generating distribution. Additionally, this paper proposes a data reduction procedure to estimate performance differences in a more computationally feasible manner. In doing so, we provide a statistical framework to assess the performance of stochastic optimization algorithms and to design improved procedures for the task at hand.
Disciplines
Numerical Analysis and Computation
Suggested Citation
Shilane, David; Martikainen, Jarno; and Ovaska, Seppo, "Time-Dependent Performance Comparison of Stochastic Optimization Algorithms" (August 2007). U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 224.
https://biostats.bepress.com/ucbbiostat/paper224