title: Reliability of a 360-degree evaluation to assess resident competence.
abstract: OBJECTIVE
  To determine the feasibility and psychometric qualities of a 360-degree evaluation of physical medicine and rehabilitation (PM&R) residents' competence.
  
  
  METHODS
  Nurses, allied health staff, and medical students completed a 12-item questionnaire after each PM&R resident rotation from January 2002 to December 2004. The items were derived from five of the six competencies defined by the Accreditation Council for Graduate Medical Education (ACGME).
  
  
  RESULTS
  Nine hundred thirty evaluations of 56 residents were completed. The alpha reliability coefficient for the instrument was 0.89. Ratings did not vary significantly by resident gender. Senior residents had higher ratings than junior residents. A reliability of >0.8 could be achieved by ratings from just five nurses or allied health staff, compared with 23 ratings from medical students. Factor analysis revealed all items clustered on one factor, accounting for 84% of the variance. In a subgroup of residents with low scores, raters were able to differentiate among skills.
  
  
  CONCLUSIONS
  Resident assessment tools should be valid, reliable, and feasible. This Web-based 360-degree evaluation tool is a feasible way to obtain reliable ratings from rehabilitation staff about resident behaviors. The assignment of higher ratings for senior residents than junior residents is evidence for the general validity of this 360-degree evaluation tool in the assessment of resident performance. Different rater groups may need distinct instruments based on the exposure of rater groups to various resident activities and behaviors.
authors: Department of Rehabilitation Medicine, University of Washington, Seattle, Washington 98105, USA.
authors: Teresa L
authors: TL
authors: Massagli
authors: Jan D
authors: JD
authors: Carline
mesh_terms: D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods*; D005163:Factor Analysis, Statistical; D005240:Feasibility Studies; D005260:Female; D006801:Humans; D007396:Internship and Residency; D008297:Male; D010810:Physical and Rehabilitation Medicine / Q000193:education*; D011594:Psychometrics; D015203:Reproducibility of Results; D014861:Washington