Statistical methods for the assessment of EQAPOL proficiency testing: ELISpot, Luminex, and Flow Cytometry.
In September 2011 Duke University was awarded a contract to develop the National Institutes of Health/National Institute of Allergy and Infectious Diseases (NIH/NIAID) External Quality Assurance Program Oversight Laboratory (EQAPOL). Through EQAPOL, proficiency testing programs are administered for Interferon-γ (IFN-γ) Enzyme-linked immunosorbent spot (ELISpot), Intracellular Cytokine Staining Flow Cytometry (ICS) and Luminex-based cytokine assays. One of the charges of the EQAPOL program was to apply statistical methods to determine overall site performance. We utilized various statistical methods for each program to find the most appropriate for assessing laboratory performance using the consensus average as the target value. Accuracy ranges were calculated based on Wald-type confidence intervals, exact Poisson confidence intervals, or via simulations. Given the nature of proficiency testing data, which has repeated measures within donor/sample made across several laboratories; the use of mixed effects models with alpha adjustments for multiple comparisons was also explored. Mixed effects models were found to be the most useful method to assess laboratory performance with respect to accuracy to the consensus. Model based approaches to the proficiency testing data in EQAPOL will continue to be utilized. Mixed effects models also provided a means of performing more complex analyses that would address secondary research questions regarding within and between laboratory variability as well as longitudinal analyses.
Duke Human Vaccine Institute, Duke University Medical Center, Durham, NC, USA. Electronic address: wes.rountree@dm.duke.edu.
Wes
W
Rountree
Duke Human Vaccine Institute, Duke University Medical Center, Durham, NC, USA.
Nathan
N
Vandergrift
Duke Human Vaccine Institute, Duke University Medical Center, Durham, NC, USA.
John
J
Bainbridge
Duke Human Vaccine Institute, Duke University Medical Center, Durham, NC, USA.
Ana M
AM
Sanchez
Duke Human Vaccine Institute, Duke University Medical Center, Durham, NC, USA.
Thomas N
TN
Denny
D015415:Biomarkers / Q000097:blood; D016207:Cytokines / Q000097:blood*; D003627:Data Interpretation, Statistical; D058501:Enzyme-Linked Immunospot Assay / Q000592:standards* / Q000706:statistics & numerical data; D005434:Flow Cytometry / Q000592:standards* / Q000706:statistics & numerical data; D019983:Guideline Adherence / Q000592:standards; D006801:Humans; D059425:Interferon-gamma Release Tests / Q000592:standards* / Q000706:statistics & numerical data; D007753:Laboratories / Q000592:standards* / Q000706:statistics & numerical data; D059021:Laboratory Proficiency Testing / Q000592:standards* / Q000706:statistics & numerical data; D015233:Models, Statistical; D015166:Monitoring, Immunologic / Q000592:standards* / Q000706:statistics & numerical data; D015588:Observer Variation; D017410:Practice Guidelines as Topic / Q000592:standards; D011237:Predictive Value of Tests; D016730:Program Development; D015397:Program Evaluation; D011786:Quality Control; D019984:Quality Indicators, Health Care / Q000592:standards; D015203:Reproducibility of Results; D013048:Specimen Handling / Q000592:standards
D015415:Biomarkers; D016207:Cytokines
ELISpot; Flow Cytometry; Luminex; Mixed effects models; Poisson; Proficiency testing