Longitudinal case-based evaluation of diagnostic competency among pathology residents: a statistical approach.
BACKGROUND
Accreditation Council for Graduate Medical Education guidelines require the evaluation of residents in 6 competencies. Pathology residents demonstrate medical knowledge, patient care, communication, and practice-based learning competencies in their attainment of competency in surgical pathology diagnosis.


OBJECTIVE
To implement a prospective case-based approach to longitudinally evaluate the acquisition of competency in surgical pathology diagnosis by trainees.


METHODS
Each resident made his or her surgical pathology diagnosis on cases before faculty review of the cases. Faculty members scored each resident diagnosis as to whether they agree, partially agree, or disagree with the diagnosis. Forty-three months of surgical pathology reports (August 2001 through January 2005) and 22,252 surgical pathology cases were analyzed.


METHODS
Pathology residency program.


METHODS
Thirteen faculty members and 21 trainees.


METHODS
Time and training year trends for the number of cases reviewed and the percent agreement between faculty and trainees on the diagnoses.


RESULTS
A mean of 146 cases (range, 12-327 cases) was reviewed during each month-long rotation. The number of cases reviewed increased through postgraduate year 4. The percent agreement on the diagnoses was 78% (range, 56%- 99%) for all trainees, with improvement by postgraduate year, although the improvement attenuated by postgraduate year 3. Residents were less likely to preview the most complex cases. Faculty rank and sex and resident sex did not significantly affect outcomes. The overall agreement on the diagnoses increased over time. Residents experiencing difficulty could be identified clearly and early.


CONCLUSIONS
Individual resident performance was easily tracked over time. The review of hundreds of reports increases systems accountability and allows more objectivity than traditional evaluations. The use of case-based evaluation fosters earlier identification and remediation of deficiencies.
Ducatman
Barbara S
BS
Department of Pathology, Institute for Occupational and Environmental Health, Robert C. Byrd Health Sciences Center, West Virginia University, Morgantown, WV 26506-9203, USA. bducatman@hsc.wvu.edu
Ducatman
Alan M
AM
D019090:Case Management / Q000592:standards* / Q000706:statistics & numerical data; D002983:Clinical Competence / Q000592:standards* / Q000706:statistics & numerical data; D003162:Competency-Based Education; D004493:Education; D004521:Educational Measurement; D005180:Faculty, Medical; D006801:Humans; D007396:Internship and Residency / Q000592:standards* / Q000706:statistics & numerical data; D010340:Pathology, Surgical / Q000193:education / Q000592:standards / Q000706:statistics & numerical data; D011446:Prospective Studies