This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Bruner, L H Articles by Curren, R D Search for Related Content PubMed PubMed Citation Articles by Bruner, L H Articles by Curren, R D Social Bookmarking                         What's this?

An investigation of new toxicity test method performance in validation studies: 1. toxicity test methods that have predictive capacity no greater than chance

The Gillette Company, Gillette Environment, Health & Safety, Needham, Massachusetts 02492, USA; Gillette Environment, Health & Safety, 37 A Street, Needham, Massachusetts 02492, USA; leon_bruner{at}gillette.com

G J Carr

The Procter & Gamble Company, Miami Valley Laboratories, Cincinnati, Ohio, USA

J W Harbell

R D Curren

The Institute for In Vitro Sciences, 21 Firstfield Road, Gaithersburg, Maryland 20878, USA

An approach commonly used to measure new toxicity test method (NTM) performance in validation studies is to divide toxicity results into positive and negative classifications, and then identify true positive (TP), true negative (TN), false positive (FP) and false negative (FN) results. After this step is completed, the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) are calculated. Although these statistics are widely used and often the only statistics used to assess the performance of toxicity test methods, there is little specific guidance in the validation literature on what values for these statistics indicate adequate performance. The purpose of this study was to begin developing data-based answers to this question by characterizing the CPS obtained from an NTM whose data have a completely random association with a reference test method (RTM). Determining the CPS of this worst-case scenario is useful because it provides a lower baseline from which the performance of an NTM can be judged in future validation studies. It also provides an indication of relationships in the CPS that help identify random or near-random relationships in the data. The results from this study of randomly associated tests show that the values obtained for the statistics vary significantly depending on the cut-offs chosen, that high values can be obtained for individual statistics, and that the different measures cannot be considered independently when evaluating the performance of an NTM. When the association between results of an NTM and RTM is random the sum of the complementary pairs of statistics (sensitivity+ specificity, NPV+PPV) is approximately 1, and the prevalence (i.e., the proportion of toxic chemicals in the population of chemicals) and PPV are equal. Given that combinations of high sensitivity–low specificity or low specificity–high sensitivity (i.e., the sum of the sensitivity and specificity equal to approximately 1) indicate lack of predictive capacity, an NTM having these performance characteristics should be considered no better for predicting toxicity than by chance alone.

Key Words: correlation • negative predictive value • positive predictive value • prediction interval • prevalence • random association • sensitivity • specificity • toxicity test battery • toxicity test performance measurement • toxicity test validation • validation

Human & Experimental Toxicology, Vol. 21, No. 6, 305-312 (2002)
DOI: 10.1191/0960327102ht252oa


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				L H Bruner, G J Carr, J W Harbell, and R D Curren An investigation of new toxicity test method performance in validation studies: 1. toxicity test methods that have predictive capacity no greater than chance Human and Experimental Toxicology, June 1, 2002; 21(6): 305 - 312. [Abstract] [PDF]

				L H Bruner, G J Carr, J W Harbell, and R D Curren An investigation of new toxicity test method performance in validation studies: 3. sensitivity and specificity are not independent of prevalence or distribution of toxicity Human and Experimental Toxicology, June 1, 2002; 21(6): 325 - 334. [Abstract] [PDF]