title: Estimating negative likelihood ratio confidence when test sensitivity is 100%: A bootstrapping approach.
abstract: Objectives Assessing high-sensitivity tests for mortal illness is crucial in emergency and critical care medicine. Estimating the 95% confidence interval (CI) of the likelihood ratio (LR) can be challenging when sample sensitivity is 100%. We aimed to develop, compare, and automate a bootstrapping method to estimate the negative LR CI when sample sensitivity is 100%. Methods The lowest population sensitivity that is most likely to yield sample sensitivity 100% is located using the binomial distribution. Random binomial samples generated using this population sensitivity are then used in the LR bootstrap. A free R program, "bootLR," automates the process. Extensive simulations were performed to determine how often the LR bootstrap and comparator method 95% CIs cover the true population negative LR value. Finally, the 95% CI was compared for theoretical sample sizes and sensitivities approaching and including 100% using: (1) a technique of individual extremes, (2) SAS software based on the technique of Gart and Nam, (3) the Score CI (as implemented in the StatXact, SAS, and R PropCI package), and (4) the bootstrapping technique. Results The bootstrapping approach demonstrates appropriate coverage of the nominal 95% CI over a spectrum of populations and sample sizes. Considering a study of sample size 200 with 100 patients with disease, and specificity 60%, the lowest population sensitivity with median sample sensitivity 100% is 99.31%. When all 100 patients with disease test positive, the negative LR 95% CIs are: individual extremes technique (0,0.073), StatXact (0,0.064), SAS Score method (0,0.057), R PropCI (0,0.062), and bootstrap (0,0.048). Similar trends were observed for other sample sizes. Conclusions When study samples demonstrate 100% sensitivity, available methods may yield inappropriately wide negative LR CIs. An alternative bootstrapping approach and accompanying free open-source R package were developed to yield realistic estimates easily. This methodology and implementation are applicable to other binomial proportions with homogeneous responses.
authors.affiliation: 1 Department of Emergency Medicine, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Keith A
authors.initials: KA
authors.lastname: Marill
authors.affiliation: 2 Department of Medicine, Harvard Medical School, Boston, MA, USA.
authors.forename: Yuchiao
authors.initials: Y
authors.lastname: Chang
authors.affiliation: 3 Department of Chemistry, Center for Simulation and Modeling, University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Kim F
authors.initials: KF
authors.lastname: Wong
authors.affiliation: 4 Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA.
authors.forename: Ari B
authors.initials: AB
authors.lastname: Friedman
mesh_terms: D016010:Binomial Distribution; D016001:Confidence Intervals; D003422:Critical Care / Q000379:methods; D003955:Diagnostic Tests, Routine / Q000379:methods* / Q000592:standards*; D006801:Humans; D016013:Likelihood Functions; D009010:Monte Carlo Method; D011379:Prognosis; D018401:Sample Size; D012680:Sensitivity and Specificity; D012984:Software
keywords: Monte Carlo method; Sensitivity and specificity; biostatistics; bootstrapping; confidence intervals; data interpretation; statistical