Assessing the predictive value of the American Board of Family Practice In-training Examination.
OBJECTIVE
The American Board of Family Practice In-training Examination (ABFP ITE) is a cognitive examination similar in content to the ABFP Certification Examination (CE). The ABFP ITE is widely used in family medicine residency programs. It was originally developed and intended to be used for assessment of groups of residents. Despite lack of empirical support, however, some residency programs are using ABFP ITE scores as individual resident performance indicators. This study's objective was to estimate the positive predictive value of the ABFP ITE for identifying residents at risk for poor performance on the ABFP CE or a subsequent ABFP ITE.


METHODS
We used a normal distribution model for correlated test scores and Monte Carlo simulation to investigate the effect of test reliability (measurement errors) on the positive predictive value of the ABFP ITE.


RESULTS
The positive predictive value of the composite score was .72. The positive predictive value of the eight specialty subscales ranged from .26 to .57.


CONCLUSIONS
Only the composite score of the ABFP ITE has acceptable positive predictive value to be used as part of a comprehension resident evaluation system. The ABFP ITE specialty subscales do not have sufficient positive predictive value or reliability to warrant use as performance indicators.
Replogle
William H
WH
Department of Family Medicine, University of Mississippi Medical Center, Jackson, 32916, USA. wrepogle@familymed.umsmed.edu
Johnson
William D
WD
D002568:Certification; D004503:Education, Medical, Graduate; D004521:Educational Measurement / Q000379:methods; D005194:Family Practice / Q000193:education*; D006801:Humans; D007396:Internship and Residency / Q000592:standards; D011897:Random Allocation; D013337:Students, Medical; D014481:United States