title: Correcting for bias in the selection and validation of informative diagnostic tests.
abstract: When developing a new diagnostic test for a disease, there are often multiple candidate classifiers to choose from, and it is unclear if any will offer an improvement in performance compared with current technology. A two-stage design can be used to select a promising classifier (if one exists) in stage one for definitive validation in stage two. However, estimating the true properties of the chosen classifier is complicated by the first stage selection rules. In particular, the usual maximum likelihood estimator (MLE) that combines data from both stages will be biased high. Consequently, confidence intervals and p-values flowing from the MLE will also be incorrect. Building on the results of Pepe et al. (SIM 28:762-779), we derive the most efficient conditionally unbiased estimator and exact confidence intervals for a classifier's sensitivity in a two-stage design with arbitrary selection rules; the condition being that the trial proceeds to the validation stage. We apply our estimation strategy to data from a recent family history screening tool validation study by Walter et al. (BJGP 63:393-400) and are able to identify and successfully adjust for bias in the tool's estimated sensitivity to detect those at higher risk of breast cancer.
authors: MRC Biostatistics Unit, Institute of Public Health, Cambridge, UK.
authors: David S
authors: DS
authors: Robertson
authors: A Toby
authors: AT
authors: Prevost
authors: Jack
authors: J
authors: Bowden
mesh_terms: D000704:Analysis of Variance; D015982:Bias; D015415:Biomarkers / Q000032:analysis; D001943:Breast Neoplasms / Q000175:diagnosis*; D003198:Computer Simulation; D016001:Confidence Intervals; D055088:Early Detection of Cancer / Q000379:methods / Q000592:standards* / Q000706:statistics & numerical data; D015340:Epidemiologic Research Design; D005192:Family Health / Q000706:statistics & numerical data*; D005260:Female; D006801:Humans; D016013:Likelihood Functions; D015203:Reproducibility of Results; D018570:Risk Assessment / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data; D018709:Statistics, Nonparametric; D011795:Surveys and Questionnaires / Q000592:standards
chemical_list: D015415:Biomarkers
keywords: diagnostic tests; family history; group sequential design; uniformly minimum variance unbiased estimator
references: Stat Med. 2012 Feb 28;31(5):420-35
references: 22238117
references: J Natl Cancer Inst. 2001 Jul 18;93(14):1054-61
references: 11459866
references: Comput Stat Data Anal. 2009 Jan 1;53(3):586-595
references: 20047003
references: Stat Med. 2010 Apr 30;29(9):959-71
references: 20191605
references: Stat Med. 2013 May 10;32(10):1677-90
references: 22744936
references: Stat Med. 2005 Dec 30;24(24):3697-714
references: 16320264
references: Bioinformatics. 2009 Oct 15;25(20):2700-7
references: 19648135
references: Stat Med. 2013 Mar 15;32(6):1027-37
references: 23413213
references: Nature. 2002 Jan 31;415(6871):530-6
references: 11823860
references: Stat Med. 2003 Mar 15;22(5):689-703
references: 12587100
references: Biom J. 2014 Mar;56(2):332-49
references: 24353149
references: Biom J. 2008 Aug;50(4):515-27
references: 18663760
references: Cancer Prev Res (Phila). 2013 Jul;6(7):656-65
references: 23694962
references: J Natl Cancer Inst. 2001 Mar 7;93(5):334-5
references: 11238688
references: Br J Gen Pract. 2013 Jun;63(611):e393-400
references: 23735410
references: Stat Med. 2010 Dec 30;29(30):3118-25
references: 21170906
references: J Cancer. 2010 Oct 06;1:150-77
references: 20975847
references: Stat Med. 2009 Feb 28;28(5):762-79
references: 19097251
references: Stat Med. 2013 Jul 30;32(17):2893-910
references: 23413228
references: Stat Med. 2004 Mar 30;23(6):881-96
references: 15027078
references: Biom J. 2005 Jun;47(3):257-68; discussion 269-81
references: 16053251