There is considerable interest in community interventions for health promotion, where the community is the experimental unit. Because such interventions are expensive, the number of experimental units (communities) is usually very small, yielding a study with low power. We examined the ability of a process known as “pooling” or “preliminary significance testing” to improve the power of community variations. In this process, one first tests whether there is significant community variation, using type 1 error of perhaps 0.25. If there is significant variation, the usual community-level test is performed. If not, a person-level test is performed. We found through Monte Carlo simulation that for studies with 2, 3, or 4 communities per group, this procedure could improve power somewhat in situations where the community by time variation is known to be small. Estimates of community by time variation for a variety of health variables are also presented. Because of the limited information available on community variances, and the probable difficulties in defending a person-level analysis, we recommend against the pooling procedure at this time.
Diehr, Paula; Lystig, Ted; Andrilla, Holly; and Feng, Ziding, "Pooling Community Data for Community Interventions When the Number of Pairs is Small" (May 1997). UW Biostatistics Working Paper Series. Working Paper 149.