POSITIVE FALSE DISCOVERY PROPORTIONS: INTRINSIC BOUNDS AND ADAPTIVE CONTROL

Zhiyi Chi, Department of Statistics, University of Connecticut
Zhiqiang Tan, Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health

Abstract

Multiple hypothesis testing aims to assess several or even thousands of null hypotheses simultaneously. The overall error can be controlled in terms of different compound measures on false discoveries. False discovery rate (FDR) is the expectation of false discovery proportion (FDP), while positive false discovery rate (pFDR) is the conditional expectation of FDP given that at least one discovery is made. False discovery excessive probability (FDEP) refers to the probability that FDP exceeds a specified level, while positive false discovery excessive probability (pFDEP) refers to the corresponding conditional probability. By definition, pFDR and pFDEP are more relevant to follow-up studies once positive findings have been obtained. In this article, we first investigate the controllability of pFDR and pFDEP under a mixture model, and establish that given a multiple testing problem, there exist possibly positive bounds on pFDR and pFDEP. The bounds are intrinsic to the problem, and applicable to any multiple testing procedure: no procedure can attain a pFDR or pFDEP below the corresponding lower bound. Second, we propose several procedures to control pFDR and pFDEP, and investigate the asymptotic behaviors of the procedures under the mixture model. The proposed procedures are adaptive in the sense that given a control level, they asymptotically achieve control when the level is attainable, and make no rejection otherwise. Moreover, the proposed procedures can achieve adaptive control of pFDR and pFDEP under certain sparsity condition where the fraction of false null hypotheses is increasingly close to zero.