CommentaryNeurotoxicity test validation, positive controls and proficiency: Are chemicals necessary?
Introduction
The United States Environmental Protection Agency (USEPA; the Agency) neurotoxicity guidelines typically require the use of chemicals to “detect chemically induced increases and decreases in [motor] activity” [19], to evaluate “effects in neonatal animals perinatally exposed to chemicals” [20], to “provide evidence that the experimental procedures are sensitive to substances known to affect operant behavior” [21], to “provide evidence that the experimental procedures are sensitive to substances or procedures known to affect peripheral nerve function” [22], or to “demonstrate the capability of the laboratory performing the testing to conduct the procedure” [23]. These represent a spectrum of guideline studies; some are required for registration, others are required if results from other studies suggest a significant potential for neurotoxicity. A summary of the endpoints for each of these studies is presented in Table 1.
The USEPA addresses the nature of the positive control chemicals and indicates that “Permanently injurious substances need not be used for the behavioral tests.” [19], [21]. Acrylamide and trimethyl tin are suggested for nervous system pathology [19]. In the “Peripheral nerve function” [22] and the “Neurophysiology: sensory evoked potentials” [23] guidelines, the USEPA states that the positive control data do not have to include chemicals. The Agency recommends “procedures” and states that: “Temperature change could be used as a positive control procedure …” [22] or “ … it should be demonstrated that the mean of an amplitude-sensitive dependent measure increases monotonically as a function of stimulus intensity …” [23].
Regarding the USEPA developmental neurotoxicity guidelines mentioned above [20], the Agency has reported that some positive control data packages submitted to the Agency were poor and lacked adequate information [5]. The main deficiencies encountered within the context of the USEPA developmental neurotoxicity guideline included absence of positive control data, positive control data without effects, inappropriate control chemical, inappropriate dose and/or route of administration, lack of statistical analysis and use of one sex.
Several reasons have been given in the guidelines by the Agency for requesting positive control data, but these reasons are neither uniform nor fully developed. The stated reasons are to address the “sensitivity” of the test, its “reliability”, the “competence” of the laboratory, the “ability to detect increases and decreases” and/or the “significance” of effects seen. Further, the Agency offers little guidance in selecting specific chemicals (or procedural manipulations) to be used as positive controls for specific tests, leaving important details (dose, route, timing, etc.) up to the registrant. Though this lack of prescription allows some freedom in how validation studies are conducted, it also increases the chances of inappropriate selections and decisions, which may result in wasted use of laboratory resources and animals, and invalid positive control studies.
Further evaluation of the guideline statements, the Agency's specific comments to registrants over the years, as well as some reflection on scientific test requirements have led us to rephrase the questions that should be asked about a neurotoxicology test, as follows:
- 1.
“What does the test measure?”
- 2.
“Does the laboratory have the competence to perform the test?”
- 3.
“Do the complementary data submitted in support of neurotoxicity studies conducted with the test material provide enough context for the interpretation of the biological significance of an effect?”
The purpose of this commentary is to provide a conceptual framework for the evaluation of what needs to be performed before a test (old or new) can be satisfactorily used within the context of the guidelines.
Section snippets
What does the test measure?
This first question addresses two important concepts referred to in the guidelines: “sensitivity” and “reliability”.
First, for any test to be useful, it must be reliable, i.e., reproducible. When applied under similar conditions, the test should give results of substantial equivalence. This is the primary and sine qua non condition for any test.
After documenting the reliability of a test, the next question to be resolved during the initial construction of a test is its “validity”. The
Conclusions
Reliability and validity of a test are expected prerequisites before recommending its use within the context of the USEPA neurotoxicity testing guidelines. The two key aspects of the validity of a test must have been documented, i.e., 1) does the test measure what it is supposed to measure? (sensitivity) 2) does the test not measure what it is not supposed to measure? (specificity). This validation process can be performed with chemicals as well as procedures (e.g., parametric manipulation).
Acknowledgments
The authors want to acknowledge Dr. Joel L. Mattsson, Dow AgroSciences LLC, for numerous discussions about this topic.
References (24)
- et al.
Hyperuricemia and locomotor activity in developing rats
Pharm. Biochem. Behav.
(1989) - et al.
A qualitative analysis of positive control data in developmental neurotoxicity studies
Neurotoxicol. Teratol.
(2004) - et al.
Validation of an auditory startle response system using chemicals or parametric modulation as positive controls
Neurotoxicol. Teratol.
(2004) - et al.
Factors affecting grip strength testing
Neurotoxicol. Teratol.
(2003) On the importance of test validity and the experimental doubt
Neurotoxicol. Teratol.
(2005)Approaches for assessing the validity of a functional observational battery
Neurotoxicol. Teratol.
(1990)- et al.
The validity and utility of geotaxis in young rodents
Neurotoxicol. Teratol.
(2005) - American Education Research Association, American Psychological Association and National Council on Measurement in...
- et al.
Effects of drugs of abuse and scopolamine on memory in rats: delayed spatial alternation and matching to position
Psychopharmacology
(1998) - et al.
Neonatal medial prefrontal lesions and recovery of spatial delayed alternation in the rat: effects of delay interval
Dev. Psychobiol.
(1995)
Pharmacological mechanisms and animal models of cognition
Behav. Pharmacol.
Cited by (10)
Ethical considerations
2020, Information Resources in Toxicology, Volume 1: Background, Resources, and ToolsBehavioral Screening for Toxicology and Safety Pharmacology
2018, Comprehensive Toxicology: Third EditionBehavioral Screening for Toxicology
2010, Comprehensive Toxicology, Second EditionUndertaking positive control studies as part of developmental neurotoxicity testing. A report from the ILSI Research Foundation/Risk Science Institute expert working group on neurodevelopmental endpoints
2008, Neurotoxicology and TeratologyCitation Excerpt :They help demonstrate the dynamic range of a biological response beyond that observed in control animals, create a realistic environment to verify training proficiency of laboratory personnel, and characterize inter-observer, intra-laboratory, and inter-laboratory reliability [24,25,40,43,63,64,72,154,155,168,181,199,221]. Positive control data, along with historical control and other proficiency data, are often used to establish confidence in test results from new laboratories or help interpret data from previously untested chemicals [43,57,79,89,107,130,138,139,180]. Characterizing the effects of positive controls in animals provides valuable information on normal variability in animals [24,40,43,58,63,132,138,154,168,219,221,227,228].
Determining normal variability in a developmental neurotoxicity test A report from the ILSI Research Foundation/Risk Science Institute expert working group on neurodevelopmental endpoints
2008, Neurotoxicology and TeratologyCitation Excerpt :Test administration procedures can vary as a function of operational parameters such as level or type of background noise, stimulus strength (e.g., decibel level), length of the acclimation period, number of trials/session, or length of the inter-trial interval. To determine readiness for the conduct of a DNT study, preliminary proficiency studies play an important role in determining the performance of equipment, personnel, procedural effectiveness, and appropriateness of responses obtained [64,100]. In summary, unwanted variability can be reduced by the use of well-trained laboratory personnel, automated and well-calibrated testing equipment, and strict adherence to well-defined, standardized testing procedures [1,4,23].
- 1
Current address: Syngenta Crop Protection, Inc., NAFTA Human Safety Assessment, Greensboro, NC 27410.