The University of North Carolina at Chapel Hill Department of Biostatistics Technical Report Series

Feature Elimination in Empirical Risk Minimization and Support Vector Machines

Sayan Dasgupta et al. — Thu, 18 Apr 2013 13:16:51 PDT

We develop an approach for feature elimination in empirical risk minimization and support vector machines, based on recursive elimination of features. We present theoretical properties of this method and show that this is uniformly consistent in finding the correct feature space under certain generalized assumptions. We present case studies to show that the assumptions are met in most practical situations and also present simulation studies to demonstrate performance of the proposed approach.

Latent Supervised Learning

Susan Wei et al. — Mon, 18 Mar 2013 05:13:29 PDT

A new machine learning task is introduced, called latent supervised learning, where the goal is to learn a binary classifier from continuous training labels which serve as surrogates for the unobserved class labels. A specific model is investigated where the surrogate variable arises from a two-component Gaussian mixture with unknown means and variances, and the component membership is determined by a hyperplane in the covariate space. The estimation of the separating hyperplane and the Gaussian mixture parameters forms what shall be referred to as the change-line classification problem. A data-driven sieve maximum likelihood estimator for the hyperplane is proposed, which in turn can be used to estimate the parameters of the Gaussian mixture. The estimator is shown to be consistent. Simulations as well as empirical data show the estimator has high classification accuracy.

Cross-Validation for Nonlinear Mixed Effects Models

Emily Colby et al. — Thu, 14 Mar 2013 06:15:36 PDT

Cross-validation is frequently used for model selection in a variety of applications. However, it is difficult to apply cross-validation to mixed effects models (including the nonlinear mixed effects models) due to the fact that cross-validation requires “out-of-sample” predictions of the outcome variable, which cannot be easily calculated when random effects are present.We describe two novel variants of cross-validation that can be applied to nonlinear mixed effects models. One variant, where out-of-sample predictions are based on post hoc estimates of the random effects, can be used to select the overall structural model. Another variant, where cross-validation seeks to minimize the estimated random effects rather than the estimated residuals, can be used to select covariates to include in the model.We show that these methods produce accurate results in a variety of simulated data sets and apply them to two publicly available population pharmacokinetic data sets.

Parameter Estimation in Cox Proportional Hazard Models with Missing Censoring Indicators

Naomi Brownstein et al. — Mon, 28 Jan 2013 13:56:01 PST

In a prospective cohort study, examining all participants for incidence of the condition of interest may be prohibitively expensive. For example, the ``gold standard'' for diagnosing temporomandibular disorder (TMD) is a clinical examination by an expert dentist. In a large study, examining all subjects in this manner is infeasible. Instead, it is common to use a cheaper (and less reliable) examination to screen for possible incident cases and perform the ``gold standard'' examination only on participants who screen positive on this simpler examination. Unfortunately, some subjects may leave the study before receiving the ``gold standard'' examination. Within the framework of survival analysis, this results in missing censoring indicators. Motivated by the Orofacial Pain: Prospective Evaluation and Risk Assessment(OPPERA) study, a large cohort study of TMD, we propose a method for parameter estimation in survival models with missing censoring indicators. We estimate the probability of being a case for those with no ``gold standard'' examination using logistic regression. These predicted probabilities are used to generate multiple imputations of each missing case status and estimate the hazard ratios associated with each putative risk factor. The variance introduced by the procedure is estimated using multiple imputation. We simulate data with missing censoring indicators and show that our method performs as well as or better than the competing methods. Finally, we apply the proposed method to analyze data from the OPPERA study.

Reinforcement Learning Trees

Ruoqing Zhu et al. — Thu, 10 Jan 2013 07:20:28 PST

In this paper, we introduce a new type of tree-based regression method, reinforcement learning trees (RLT), which exhibits significantly improved performance over traditional methods such as random forests (Breiman, 2001). The innovations are three-fold. First, the new method implements reinforcement learning at each selection of a splitting variable during the tree construction processes. By splitting on the variable that brings the greatest future improvement in later splits, rather than choosing the one with largest marginal effect from the immediate split, the constructed tree utilizes the available samples in a more efficient way. Moreover, such an approach can be adapted to make high-dimensional cuts available at a relatively small computational cost. Second, we propose a variable screening method that progressively mutes noise variables during the construction of each individual tree. The muting procedure also takes advantage of reinforcement learning and prevents noise variables from being considered in the search for splitting rules, so that towards a terminal node when the sample size is small, the splitting rules are still constructed from only strong variables. Last, we investigate asymptotic properties of the proposed method. We can show that under the proposed splitting variable selection procedure, the constructed trees are consistent. The error bounds for the proposed RLT are shown to depend on a pre-selected number p₀, where p₀ is an educated guess of the number of strong variables which is usually much smaller than the total number of variables p but at least as large as the true number of strong variables p₁. Hence when p₀ is properly chosen, the error bounds can be significantly improved.

Identification of biologically relevant subtypes via preweighted sparse clustering

Sheila Gaynor et al. — Tue, 04 Dec 2012 04:10:31 PST

Cluster analysis methods are used to identify homogeneous subgroups in a data set. Frequently one applies cluster analysis in order to identify biologically interesting subgroups. In particular, one may wish to identify subgroups that are associated with a particular outcome of interest. Conventional clustering methods often fail to identify such subgroups, particularly when there are a large number of high-variance features in the data set. Conventional methods may identify clusters associated with these high-variance features when one wishes to obtain secondary clusters that are more interesting biologically or more strongly associated with a particular outcome of interest. We describe a modification of the sparse clustering method of Witten and Tibshirani (2010) can be used to identify such secondary clusters or clusters associated with an outcome of interest. We show that this method can correctly identify such clusters of interest in several simulation scenarios. The method is also applied to a large case-control study of temporomandibular disorder and a breast cancer microarray data set.

Simple and accurate trend tests using a permutation approximation

Yi-Hui Zhou et al. — Tue, 16 Oct 2012 11:17:08 PDT

Permutation is an attractive approach to assess association between two vectors x and y, by comparing the observed statistic to the distribution induced by random permutation of one of the vectors. For a number of “standard” statistics, equivalent testing can be performed by using the sample Pearson correlation. Applications include the standard tests applied in the two-sample problem, simple linear regression, several generalized linear models, linear categorical trend tests, and rank-based association. We describe a simple approximation to the distribution of the correlation under permutation, providing accurate p-values that can be quickly computed for a variety of data types. The approximation may be especially useful in high-throughput applications in which a series of x-vectors is compared to one or more y-vectors.

Developing Adaptive Personalized Therapy for Cystic Fibrosis using Reinforcement Learning

Yiyun Tang et al. — Fri, 28 Sep 2012 14:15:29 PDT

Optimal clinical management of inherited chronic diseases, such as Cystic Fibrosis (CF), requires a dynamic approach which updates treatments to cope with the evolving course of illness and to tailor medicines and dosages for individual patient. In this paper, we examine the problem of computing optimal adaptive personalized therapy for CF patients. A temporal difference reinforcement learning method called fitted Q-iteration is utilized to discover the optimal treatment regimen directly from clinical data. We conduct a simulation study of virtual cystic fibrosis patients with Pseudomonas aeruginosa infection and antibiotic therapy with parameters tuned to approximately match published data from CF patients. Our simulation results indicate that reinforcement learning can be an effective tool in developing personalized therapy which optimises the benefit-risk trade off in multi-stage decision making and improves long term outcomes in chronic diseases.

Reader Reaction: On Variance Estimation for the Fine-Gray Model

Chenxi Li et al. — Mon, 20 Aug 2012 08:37:33 PDT

Geskus (2011, Biometrics, 67, 39-49) studied estimation of the Fine-Gray model for the cumulative incidence function with left truncated right censored competing risks data. The limiting distribution for an estimator base on weighting inversely using weights involving estimates of the joint distribution of the truncation and censoring times was derived via classical martingale theory with variance estimation based on martingale results. In this note, we demonstrate that martingale theory is not applicable and that other theoretical arguments, like those in Fine and Gray (1999), are needed to rigorously establish the asymptotic properties of the estimators and to construct valid variance estimators. For inverse probability of censoring weighted estimators, the common wisdom is that martingale theory fails because of estimation of the censoring distribution in the weights. For the Fine-Gray model, alternative theoretical developments are needed even with a known censoring distribution.

A Comparison of Methods for Generating Correlated Binary Variates with Specified Marginal Means and Correlations

John S. Preisser Jr. et al. — Wed, 08 Aug 2012 08:00:16 PDT

Simulation studies employed to study properties of estimators for parameters in population-averaged models for clustered or longitudinal data require suitable algorithms for data generation. The most useful algorithms for generating correlated binary data are those that allow general specifications of the marginal mean and correlation structures, while being able to generate clusters of moderate to large size. Such methods, however, cannot reproduce data for all possible multivariate binary distributions. Given a vector of marginal means, they often place restrictions on the range of correlations beyond the natural restrictions applicable to any multivariate binary distribution. Motivated by problems in biostatistics, we compare the algorithms of Emrich and Piedmonte (1991) and Qaqish (2003) with respect to range restrictions induced on correlations. Examples include generating longitudinal binary data and generating correlated binary data compatible with specified marginal means and covariance structures for bivariate, overdispersed binomial outcomes. Results show that both algorithms generally have good coverage with Qaqish's method giving a wider range of correlations for longitudinal data having autocorrelated within-subject associations and Emrich and Piedmonte's method giving a wider range of correlations for clustered data having exchangeable-type correlations. Practical considerations for generating data with varying cluster sizes or for subjects in longitudinal studies with missing data are also discussed.

A Multistage Non‐inferiority Study Analysis Plan to Evaluate Successively More Stringent Criteria for a Clinical Trial with Rare Events

Siying Li et al. — Mon, 18 Jun 2012 05:57:46 PDT

We address a multistage clinical trial to assess a sequence of hypotheses in the noninferiority and also rare events setting. Three successive hypotheses are used to evaluate whether the new treatment meets the criteria for new drug approval. Sample sizes for a five stage trial for all hypotheses are calculated using Poisson and Logrank sample size methods. Three strategies and corresponding analysis plans are developed to evaluate the sequential hypotheses. Simulations show the design is satisfactory with respect to controlled Type I error, sufficient power, and early success at interim analyses.

Change-Point Models to Estimate the Limit of Detection

Ryan C. May et al. — Mon, 05 Mar 2012 10:22:05 PST

In many biological and environmental studies, measured data is subject to a limit of detection. The limit of detection is generally defined as the lowest concentration of analyte that can be differentiated from a blank sample with some certainty. Data falling below the limit of detection is left-censored, falling below a level that is easily quantified by a measuring device. A great deal of interest lies in estimating the limit of detection for a particular measurement device. In this paper we propose an innovative change-point model to estimate the limit of detection using data from an experiment with known analyte concentrations. Estimation of the limit of detection proceeds by way of a two-stage maximum likelihood method. The proposed methodology is analyzed via simulation, and is applied to copy number data from an HIV pilot study. This method is shown to lead to improved estimation of the limit of detection.

Keywords: Change Point; Linear Calibration Curve; Limit of Detection; Two-Stage Maximum Likelihood

Research Methods for Clinical Trials in Personalized Medicine: A Systematic Review

Zheng Ren et al. — Thu, 09 Feb 2012 13:17:06 PST

Background: Personalized medicine, the notion that an individual’s genetic and other characteristics can be used to individualize the diagnosis, treatment and prevention of disease, is an active and exciting area of research, with tremendous potential to improve the health of society.

Methods: Seventy-six studies using personalized medicine analysis techniques published from 2006 to 2010 in six high-impact journals - Journal of the American Medical Association, Journal of the National Cancer Institute, Lancet, Nature, Nature Medicine, and the New England Journal of Medicine - were reviewed. Selected articles were manually selected based on reporting of the use of genetic information to stratify subjects and on analyses of the association between biomarkers and patient clinical outcomes.

Results: We found considerable variability and limited consensus in approaches. Approaches could largely be classified as data-driven, seeking discovery through statistical analysis of data, or knowledge-driven, relying heavily on prior biological information. Some studies took a hybrid approach. Eliminating two articles that were retracted after publication, 56 of the remaining 74 (76%) were cancer-related.

Conclusions: Much work is needed to standardize and improve statistical methods for finding biomarkers, validating results, and efficiently optimizing better individual treatment strategies. Several promising new analytic approaches are available and should be considered in future studies of personalized medicine.

Empirical Pathway Analysis, without Permutation

Yi-Hui Zhou et al. — Mon, 09 Jan 2012 10:06:30 PST

Resampling-based expression pathway analysis techniques have been shown to preserve type I error, in contrast to simple gene-list approaches which implicitly assume independence of genes in ranked lists. However, resampling is intensive in computation time and memory requirements. We describe highly accurate analytic approximations to permutations of score statistics, including novel approaches for Pearson correlation and summed score statistics, that have good performance for even relatively small sample sizes. In addition, the approach provides insight into the permutation approach itself, and summary properties of the data that largely determine the behavior of the statistics. Within the framework of the SAFE pathway analysis procedure, our approach preserves the essence of permutation analysis, but with greatly reduced computation. Extensions to include covariates are described, and we test the performance of our procedures using simulations based on real datasets of modest size.

Keywords: gene set analysis; permutation; hypothesis testing.

ORTH: R and SAS Software for Regression Models of Correlated Binary Data Based on Orthogonalized Residuals and Alternating Logistic Regressions

Kunthel By et al. — Wed, 20 Jul 2011 08:13:53 PDT

In this article, we describe a new software for modeling correlated binary data based on orthogonalized residuals (Zink and Qaqish, 2009), a recently developed estimating equations approach that includes, as a special case, alternating logistic regressions (Carey et al., 1993). The software is flexible with respect to fitting in that the user can choose estimating equations for the association model based on alternating logistic regressions or orthogonalized residuals, the latter choice providing a non-diagonal working covariance matrix for second moment parameters providing potentially greater efficiency. Regression diagnostics based on this method are also implemented in the software. The mathematical details of the procedure are briefly reviewed and the software is applied to medical data sets.

Deletion Diagnostics for Alternating Logistic Regressions

John S. Preisser et al. — Tue, 19 Jul 2011 08:09:27 PDT

Deletion diagnostics are introduced for the regression analysis of clustered binary outcomes estimated with alternating logistic regressions, an implementation of generalized estimating equations (GEE) that estimates regression coefficients in a marginal mean model and in a model for the intracluster association given by the log odds ratio. The diagnostics are developed within an estimating equations framework that recasts the esti- mating functions for association parameters based upon conditional resid- uals into equivalent functions based upon marginal residuals. Extensions of earlier work on GEE diagnostics follow directly, including computational formulae for one-step deletion diagnostics that measure the influence of a cluster of observations on the estimated regression parameters and on the overall marginal mean or association model fit. The diagnostic formulae are evaluated with simulations studies and with an application concerning an as- sessment of factors associated with health maintenance visits in primary care medical practices. The application and the simulations demonstrate that the proposed cluster-deletion diagnostics for alternating logistic regressions are good approximations of their exact fully iterated counterparts.

Causal Inference for Vaccine Effects on Infectiousness

M. Elizabeth Halloran et al. — Fri, 29 Apr 2011 12:10:39 PDT

If a vaccine does not protect individuals completely against infection, it could still reduce infectiousness of infected vaccinated individuals for others. Typically, vaccine efficacy for infectiousness is estimated based on contrasts between the transmission risk to susceptible individuals from infected vaccinated individuals compared with that from infected unvaccinated individuals. Such estimates are problematic, however, because they are subject to selection bias and do not have a causal interpretation. Here we develop causal estimands for vaccine efficacy for infectiousness for four different scenarios of populations of transmission units of size two. These causal estimands incorporate both principal stratification based on the joint potential infection outcomes under vaccine and control and interference between individuals within transmission units. In the most general scenario, both individuals can be exposed to infection outside the transmission unit and both can be assigned either vaccine or control. The three other scenarios are special cases of the general scenario where only one individual is exposed outside the transmission unit or can be assigned vaccine. For each scenario, the principal stratification based on the joint potential infection outcomes under vaccine and control of the individuals exposed outside the transmission unit is developed. The causal estimands for vaccine efficacy for infectiousness are well defined only within certain principal strata and, in general, are identifiable only with strong unverifiable assumptions. Nonetheless, the observed data do provide some information, and we derive large sample bounds on the causal vaccine efficacy for infectiousness estimands. For each scenario, several other causal vaccine efficacy estimands and estimators are defined. An example of the type of data typically observed in a study to estimate vaccine efficacy for infectiousness is analyzed in the causal inference framework we develop.

The Interactive Decision Committee for Chaemical Toxicity Analysis

Chaeryon Kang et al. — Thu, 16 Dec 2010 11:38:25 PST

We introduce the Interactive Decision Committee method for classification when high-dimensional feature variables are grouped into feature categories. The proposed method uses the interactive re- lationships among feature categories to build base classifiers which are combined using decision committees. A two-stage 5-fold cross- validation technique is utilized to decide the total number of base classifiers to be combined. The proposed procedure is useful for clas- sifying biochemicals on the basis of toxicity activity, where the feature space consists of chemical descriptors and the responses are binary indicators of toxicity activity. Each descriptor belongs to at least one descriptor category. The support vector machine algorithm is utilized as a classifier inducer. Forward selection is used to select the best combinations of the base classifiers given the number of base classifiers. We applied the proposed method to two chemical toxic- ity data sets. For these data sets, the proposed method outperforms other decision committee methods including adaboost, bagging, random forests, the univariate decision committee, and a single large, unaggregated classifier.

Randomization-Based Inference within Principal Strata

Tracy L. Nolen et al. — Fri, 20 Aug 2010 11:20:19 PDT

In randomized studies, treatment comparisons conditional on intermediate post-randomization outcomes using standard analytic methods do not have a causal interpretation. An alternate approach entails treatment comparisons within principal strata by the potential outcomes for the intermediate outcome that would be observed under each treatment assignment. In this paper, we develop methods for randomization-based inference within principal strata. The proposed methods are compared with existing large-sample methods as well as traditional intent-to- treat approaches. This research is motivated by HIV prevention studies where few infections are expected and inference is desired within the always-infected principal stratum, i.e., all individuals who would become infected regardless of randomization assignment.

Group Testing for Case Identification with Correlated Responses

Samuel D. Lendle et al. — Fri, 20 Aug 2010 11:06:24 PDT

This paper examines group testing procedures where units within a group (or pool) may be correlated. The expected number of tests per unit (i.e., efficiency) of hierarchical and matrix based procedures is derived based on a class of models of exchangeable binary random variables. The effect of the arrangement of correlated units within pools on efficiency is then examined. In general, when correlated units are arranged in the same pool, the expected number of tests per unit decreases, sometimes substantially, relative to arrangements which ignore information about correlation.