title: Adapting the posterior probability of diagnosis index to enhance evidence-based screening: an application to ADHD in primary care.
abstract: This study adapts the Posterior Probability of Diagnosis (PPOD) Index for use with screening data. The original PPOD Index, designed for use in the context of comprehensive diagnostic assessments, is overconfident when applied to screening data. To correct for this overconfidence, we describe a simple method for adjusting the PPOD Index to improve its calibration when used for screening. Specifically, we compare the adjusted PPOD Index to the original index and naïve Bayes probability estimates on two dimensions of accuracy, discrimination and calibration, using a clinical sample of children and adolescents (N = 321) whose caregivers completed the Vanderbilt Assessment Scale to screen for attention-deficit/hyperactivity disorder and who subsequently completed a comprehensive diagnostic assessment. Results indicated that the adjusted PPOD Index, original PPOD Index, and naïve Bayes probability estimates are comparable using traditional measures of accuracy (sensitivity, specificity, and area under the curve), but the adjusted PPOD Index showed superior calibration. We discuss the importance of calibration for screening and diagnostic support tools when applied to individual patients.
authors.affiliation: University of Pittsburgh, Pittsburgh, PA, USA lindhiemoj@upmc.edu.
authors.forename: Oliver
authors.initials: O
authors.lastname: Lindhiem
authors.affiliation: University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: Lan
authors.initials: L
authors.lastname: Yu
authors.affiliation: University of Connecticut, Storrs, CT, USA.
authors.forename: Damion J
authors.initials: DJ
authors.lastname: Grasso
authors.affiliation: University of Pittsburgh, Pittsburgh, PA, USA.
authors.forename: David J
authors.initials: DJ
authors.lastname: Kolko
authors.affiliation: University of North Carolina at Chapel Hill, NC, USA.
authors.forename: Eric A
authors.initials: EA
authors.lastname: Youngstrom
mesh_terms: D001289:Attention Deficit Disorder with Hyperactivity / Q000175:diagnosis*; D001499:Bayes Theorem; D002648:Child; D002675:Child, Preschool; D055317:Evidence-Based Practice; D005163:Factor Analysis, Statistical; D005260:Female; D006801:Humans; D008297:Male; D010290:Parents; D010414:Pennsylvania; D011320:Primary Health Care; D011336:Probability; D011569:Psychiatric Status Rating Scales / Q000592:standards* / Q000706:statistics & numerical data*; D012680:Sensitivity and Specificity
keywords: ADHD; calibration; evidence-based assessment; item response theory; screening