title: Reliability of the physical medicine and rehabilitation resident observation and competency assessment tool: a multi-institution study.
abstract: OBJECTIVE
  To assess the psychometric qualities of a method of resident physician evaluation by faculty.
  
  
  METHODS
  Multicenter study by seven Physical Medicine and Rehabilitation training programs. Faculty physicians observed residents in brief patient encounters or teaching sessions, rated specific competencies, and provided residents with immediate feedback. The resident observation and competency assessment form included competencies in patient care, professionalism, interpersonal and communication skills, systems-based practice, and practice-based learning and improvement. Residents and faculty rated satisfaction with the process.
  
  
  RESULTS
  Three hundred sixty-two ratings were completed on 88 different residents. Each resident received an average of 3.8 ratings across two academic years. Overall internal consistency reliability was high (0.98); reliability of the individual competencies ranged from 0.74 to 0.76. Item means were correlated with year of training for two skill sets, with higher means for more experienced residents. The majority of participants gave high ratings of satisfaction; correlation between satisfaction ratings of attending and resident physicians was 0.63 (P < 0.01).
  
  
  CONCLUSIONS
  The resident observation and competency assessment is a reliable method to assess resident skills in five of six general competencies. Construct validity of the tool is supported by the fact that faculty rated two skill sets higher for senior residents.
authors.affiliation: Department of Medical Education, Physical Medicine and Rehabilitation, Brody School of Medicine, East Carolina University, Greenville, North Carolina 27834, USA.
authors.forename: David W
authors.initials: DW
authors.lastname: Musick
authors.forename: William L
authors.initials: WL
authors.lastname: Bockenek
authors.forename: Teresa L
authors.initials: TL
authors.lastname: Massagli
authors.forename: Mary A
authors.initials: MA
authors.lastname: Miknevich
authors.forename: K Rao
authors.initials: KR
authors.lastname: Poduri
authors.forename: James A
authors.initials: JA
authors.lastname: Sliwa
authors.forename: Monica
authors.initials: M
authors.lastname: Steiner
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D005180:Faculty, Medical; D005246:Feedback; D006801:Humans; D007396:Internship and Residency; D010810:Physical and Rehabilitation Medicine / Q000193:education* / Q000592:standards; D011594:Psychometrics; D015203:Reproducibility of Results; D014481:United States