title: How well do internal medicine faculty members evaluate the clinical skills of residents?
abstract: OBJECTIVE
  To determine the accuracy of faculty evaluations of residents' clinical skills and whether a structured form and instructional videotape improve accuracy.
  
  
  METHODS
  Randomized, controlled trial.
  
  
  METHODS
  Twelve university and community teaching hospitals.
  
  
  METHODS
  A total of 203 faculty internists.
  
  
  METHODS
  Participants watched a videotape of one of two residents performing new patient workups. Participants were assigned to one of three groups: They used either an open-ended evaluation form or a structured form that prompted detailed observations; some participants used the structured form after seeing a videotape showing good evaluation techniques.
  
  
  METHODS
  Faculty observations of strengths and weaknesses in the residents' performance were scored. An accuracy score consisting of clinical skills of critical importance for a competent history and physical examination was calculated for each participant by raters blinded to the participants' hospital, training, subspecialty, and experience as observers.
  
  
  RESULTS
  When observations were not prompted, participants recorded only 30% of the residents' strengths and weaknesses; accuracy among participants using structured forms increased to 60% or greater. Faculty in university hospitals were more accurate than those in community hospitals, and general internists were more accurate than subspecialists; the structured form improved performance in all groups. However, participants disagreed markedly about the residents' overall clinical competence: Thirty-one percent assessed one resident's clinical skills as unsatisfactory or marginal, whereas 69% assessed them as satisfactory or superior; 48% assessed the other resident's clinical skills as unsatisfactory or marginal, whereas 52% assessed them as satisfactory or superior. Participants also disagreed about the residents' humanistic qualities. The instructional videotape did not improve accuracy.
  
  
  CONCLUSIONS
  A structured form improved the accuracy of observations of clinical skills, but faculty still disagreed in their assessments of clinical competence. If program directors are to certify residents' clinical competence, better and more standardized evaluation is needed.
authors.affiliation: Dartmouth Medical School, Hanover, NH.
authors.forename: G L
authors.initials: GL
authors.lastname: Noel
authors.forename: J E
authors.initials: JE
authors.lastname: Herbers
authors.forename: M P
authors.initials: MP
authors.lastname: Caplow
authors.forename: G S
authors.initials: GS
authors.lastname: Cooper
authors.forename: L N
authors.initials: LN
authors.lastname: Pangaro
authors.forename: J
authors.initials: J
authors.lastname: Harvey
mesh_terms: D002983:Clinical Competence; D005069:Evaluation Studies as Topic; D005180:Faculty, Medical; D006764:Hospitals, Community; D006784:Hospitals, Teaching; D006785:Hospitals, University; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D015588:Observer Variation; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires; D014481:United States; D014743:Videotape Recording