#### Title

Examination of Effects of Lethality and Mortality Rates on the Tests of Carcinogenicity Using the ED01 Study

#### Abstract

Due to the occult nature of tumors observed in animal carcinogenicity experiments, information about the rate of tumor development is confounded with the rate of mortality and the lethality of tumors. Common practice for analysis of data from such experiments involves making extreme assumptions about the lethality of tumors. Alternatively, survival/sacrifice experiments could be used to remove such confounding without making any assumptions about the lethality of tumors. Such a design however requires larger size experiments. In this paper we compare the efficiency of (i) the Hoel-Walburg or the prevalence test which assumes that tumors are nonlethal, (ii) the log rank test comparing death rates with tumor incidence rates under the assumptions that tumors are rapidly lethal and (iii) a nonparametric test of comparing the tumor incidence rate without making any assumption about the lethality of tumors suggested by Malani and Van Ryzin (1988).

A complete analysis of data from the ED01 experiment is presented for 15 different types of tumors using these three tests. Results indicate that for rapidly lethal tumors, a prevalence test indicates a false negative trend. For nonlethal and some intermediately lethal tumors the log-rank test gives the most significant results. However, this seems to be due to increased mortality at higher dose instead of increased power of the test to detect differences in the rate of tumor development. The nonparametric test based on the tumor incidence rate seems to give reasonable results in all situations.

In addition, small sample properties of these three tests are investigated using simulations based on smoothed ED01 data. Again the results indicate that for nonlethal and intermediately lethal tumors, the prevalence test suggested by Hoel and Walburg performs quite well; however for rapidly lethal tumors, the type I error rate in the simulation was less than the nominal 5% level and the prevalence test was not as powerful as the other two tests. The log-rank test performed well for rapidly lethal tumors but the type I rate was higher than 5% for most other tumors due to an increase in mortality. The test suggested by Malani and Van Ryzin was not as powerful as the other two tests in most situations. However, the test seems to preserve the type I error rate of 5% in most situations.

#### Disciplines

Epidemiology | Statistical Methodology | Statistical Theory

#### Suggested Citation

Malani, Hina M. and Lu, Ying, "Examination of Effects of Lethality and Mortality Rates on the Tests of Carcinogenicity Using the ED01 Study" (August 1990). *U.C. Berkeley Division of Biostatistics Working Paper Series.* Working Paper 15.

https://biostats.bepress.com/ucbbiostat/paper15

## Comments

Published in Communications in Statistics: Theory and Methods, 1993, 22(6), 1557-1584.