Bringing Game Theory to Hypothesis Testing: Establishing Finite Sample Bounds on Inference
Download the Paper Forward to a colleague
Abstract:
Small sample properties are of fundamental interest when only limited data is available. Exact inference is limited by constraints imposed by specific nonrandomized tests and of course also by lack of more data. These effects can be separated as we propose to evaluate a test by comparing its type II error to the minimal type II error among all tests for the given sample. Game theory is used to establish this minimal type II error, the associated randomized test is characterized as part of a Nash equilibrium of a fictitious game against nature. We use this method to investigate sequential tests for the difference between two means when outcomes are constrained to belong to a given bounded set. Tests of inequality and of noninferiority are included. We find that inference in terms of type II error based on a balanced sample cannot be improved by sequential sampling or even by observing counter factual evidence providing there is a reasonable gap between the hypotheses.
Subject Area:
Statistical Theory and Methods
Suggested Citation:
Karl H. Schlag, "Bringing Game Theory to Hypothesis Testing: Establishing Finite Sample Bounds on Inference" (June 2008). COBRA Preprint Series. Article 59.
http://biostats.bepress.com/cobra/ps/art59