Reliability of the Part II Board Certification Examination in Psychiatry: interexaminer consistency.
OBJECTIVE
The aim of the study was to examine the reliability (interexaminer consistency) of the American Board of Psychiatry and Neurology (ABPN) Part II (oral) examination in psychiatry.


METHODS
Grades were assigned independently by two examiners who observed the same examination in a 1-year cycle (1,422 candidates, two examinations each). The consistency between these pairs of grades (pass, condition, fail) was analyzed using a weighted kappa statistic.


RESULTS
There was perfect agreement between examiners in 67% of examinations, minor disagreement in 26%, and major disagreement in 7% (weighted kappa = 0.54-0.56).


CONCLUSIONS
The Part II ABPN examination demonstrates fair to good reliability as measured by interexaminer consistency. Development of more explicit grading criteria should further improve examiner agreement in future examinations.
McDermott
J F
JF
Department of Psychiatry, University of Hawaii at Manoa, Honolulu 96826.
Tanguay
P E
PE
Scheiber
S C
SC
Juul
D
D
Shore
J H
JH
Tucker
G J
GJ
McCurdy
L
L
Terr
L C
LC
D004521:Educational Measurement / Q000592:standards* / Q000706:statistics & numerical data; D006801:Humans; D011570:Psychiatry / Q000592:standards*; D015203:Reproducibility of Results; D013044:Specialty Boards / Q000592:standards*; D014481:United States