Reliability of the physical medicine and rehabilitation resident observation and competency assessment tool: a multi-institution study.
OBJECTIVE
To assess the psychometric qualities of a method of resident physician evaluation by faculty.


METHODS
Multicenter study by seven Physical Medicine and Rehabilitation training programs. Faculty physicians observed residents in brief patient encounters or teaching sessions, rated specific competencies, and provided residents with immediate feedback. The resident observation and competency assessment form included competencies in patient care, professionalism, interpersonal and communication skills, systems-based practice, and practice-based learning and improvement. Residents and faculty rated satisfaction with the process.


RESULTS
Three hundred sixty-two ratings were completed on 88 different residents. Each resident received an average of 3.8 ratings across two academic years. Overall internal consistency reliability was high (0.98); reliability of the individual competencies ranged from 0.74 to 0.76. Item means were correlated with year of training for two skill sets, with higher means for more experienced residents. The majority of participants gave high ratings of satisfaction; correlation between satisfaction ratings of attending and resident physicians was 0.63 (P < 0.01).


CONCLUSIONS
The resident observation and competency assessment is a reliable method to assess resident skills in five of six general competencies. Construct validity of the tool is supported by the fact that faculty rated two skill sets higher for senior residents.
Musick
David W
DW
Department of Medical Education, Physical Medicine and Rehabilitation, Brody School of Medicine, East Carolina University, Greenville, North Carolina 27834, USA.
Bockenek
William L
WL
Massagli
Teresa L
TL
Miknevich
Mary A
MA
Poduri
K Rao
KR
Sliwa
James A
JA
Steiner
Monica
M
D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D005180:Faculty, Medical; D005246:Feedback; D006801:Humans; D007396:Internship and Residency; D010810:Physical and Rehabilitation Medicine / Q000193:education* / Q000592:standards; D011594:Psychometrics; D015203:Reproducibility of Results; D014481:United States