Developing scientific confidence in HTS-derived prediction models: lessons learned from an endocrine case study.
High throughput (HTS) and high content (HCS) screening methods show great promise in changing how hazard and risk assessments are undertaken, but scientific confidence in such methods and associated prediction models needs to be established prior to regulatory use. Using a case study of HTS-derived models for predicting in vivo androgen (A), estrogen (E), thyroid (T) and steroidogenesis (S) endpoints in endocrine screening assays, we compare classification (fitting) models to cross validation (prediction) models. The more robust cross validation models (based on a set of endocrine ToxCast™ assays and guideline in vivo endocrine screening studies) have balanced accuracies from 79% to 85% for A and E, but only 23% to 50% for T and S. Thus, for E and A, HTS results appear promising for initial use in setting priorities for endocrine screening. However, continued research is needed to expand the domain of applicability and to develop more robust HTS/HCS-based prediction models prior to their use in other regulatory applications. Based on the lessons learned, we propose a framework for documenting scientific confidence in HTS assays and the prediction models derived therefrom. The documentation, transparency and the scientific rigor involved in addressing the elements in the proposed Scientific Confidence Framework could aid in discussions and decisions about the prediction accuracy needed for different applications.
Cox Associates, 503 Franklin St, Denver, CO 80218, USA. Electronic address: tcoxdenver@aol.com.
Louis Anthony
LA
Cox
Cox Associates, 503 Franklin St, Denver, CO 80218, USA. Electronic address: doug@cox-associates.com.
Douglas
D
Popken
The Dow Chemical Company, Toxicology & Environmental Research & Consulting, 1803 Building Washington Street, Midland, MI 48674, USA. Electronic address: MSMarty@dow.com.
M Sue
MS
Marty
The Dow Chemical Company, Toxicology & Environmental Research & Consulting, 1803 Building Washington Street, Midland, MI 48674, USA. Electronic address: JCRowlands@dow.com.
J Craig
JC
Rowlands
DuPont Haskell Global Centers for Health and Environmental Sciences, Stine 320/212, 1090 Elkton Road, Newark, DE 19711, USA. Electronic address: patlewig@hotmail.com.
Grace
G
Patlewicz
ExxonMobil Biomedical Sciences Inc., 1545 Route 22, East Annandale, NJ 08801, USA. Electronic address: katy.o.goyak@exxonmobil.com.
Katy O
KO
Goyak
Regulatory and Technical Affairs Department, American Chemistry Council (ACC), 700, 2nd Street NE, Washington, DC 20002, USA. Electronic address: Rick_Becker@americanchemistry.com.
Richard A
RA
Becker
D000728:Androgens / Q000737:chemistry; D052244:Endocrine Disruptors / Q000009:adverse effects* / Q000737:chemistry*; D004703:Endocrine System / Q000187:drug effects*; D004785:Environmental Pollutants / Q000009:adverse effects* / Q000737:chemistry*; D004967:Estrogens / Q000737:chemistry; D057166:High-Throughput Screening Assays / Q000379:methods*; D006801:Humans; D008962:Models, Theoretical; D018570:Risk Assessment; D013256:Steroids / Q000737:chemistry; D013961:Thyroid Gland / Q000737:chemistry
D000728:Androgens; D052244:Endocrine Disruptors; D004785:Environmental Pollutants; D004967:Estrogens; D013256:Steroids
Adverse outcome pathways; Endocrine; High throughput/high content assays; Prediction models; Validation framework