The reliability and validity of the American Board of Internal Medicine Monthly Evaluation Form.
OBJECTIVE
To report indicators of reliability and validity of the American Board of Internal Medicine Evaluation Form (ABIM-MEF) at one institution (Wright-Patterson Medical Center).


METHODS
Completed ABIM-MEFs from 1990-1999 were reviewed. Reliability measures included Cronbach alpha, interrater reliability, and rating consistency between different types of staff and rotations. Construct validity was investigated by tracking ABIM-MEF scores over time and with factor analysis. Predictive validity was assessed by correlating ABIM-MEF scores with the In-training Examination and ABIM Certifying Examination results


RESULTS
The 71 residents averaged 12 ABIM-MEFs per year. The forms had a Cronbach alpha of 0.96 and high interrater reliability (intraclass correlation coefficients > 0.80). Ratings did not differ by type of attending or rotation, except that noninternists rated residents lower on procedural skills. ABIM-MEF questions about judgment, knowledge, and clinical skills showed significant improvement from month to month during each academic year as well as year to year. In contrast, questions on professional attitudes, humanism, and procedural skills sections improved between postgraduate year 1 and postgraduate year 2 only. ABIM-MEF questions collapsed into two domains in factor analysis: judgment-knowledge-skills and attitude-humanism. ABIM-MEF questions from judgment and knowledge sections modestly predicted In-training Examination and ABIM Certifying Examination results. In contrast, professional attitude, humanism, and clinical as well as procedural skill questions had little discriminative ability.


CONCLUSIONS
The ABIM-MEF appears to be reliable and valid. Further, factor analysis results support the ABIM's movement to simplify the monthly evaluation form to the new Accreditation Council for Graduate Medical Education core competencies.
Durning
Steven J
SJ
Department of Medicine (NEP), Uniformed Services University of the Health Sciences, Bethesda, MD 20814, USA. sdurning@usuhs.mil
Cation
Lannie J
LJ
Jackson
Jeffrey L
JL
D000046:Academic Medical Centers; D000328:Adult; D005260:Female; D006801:Humans; D007388:Internal Medicine / Q000193:education* / Q000592:standards*; D008297:Male; D011361:Professional Competence; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D013044:Specialty Boards; D011795:Surveys and Questionnaires