title: Evaluating Surgical Residents Quickly and Easily Against the Milestones Using Electronic Formative Feedback.
abstract: OBJECTIVE
  This study was conducted to assess the effectiveness of a newly implemented electronic web-based review system created at our institution for evaluating resident performance relative to established milestones.
  
  
  METHODS
  Retrospective review of data collected from a survey of general surgery faculty and residents.
  
  
  METHODS
  Tertiary care teaching hospital system and independent academic medical center.
  
  
  METHODS
  A total of 12 general surgery faculty and 17 general surgery residents participated in this study. The survey queried the level of satisfaction before and after the adoption of QuickNotes using several statements scored on a 5-point scale, with 1 being the lowest rating as "not satisfied," and 5 being the highest rating as "completely satisfied."
  
  
  RESULTS
  The weighted average improvements from pre- to post-QuickNotes implementation for the faculty responding to the survey ranged from 10% to 40%; weighted average improvements for the residents responding to the survey ranged from 5% to 73%. For the survey of faculty, both sets of weighted averages tended to be higher than the weighted average for the resident's survey responses. The highest rated topic was the faculty's level of satisfaction with the "frequency to provide feedback" with a post-QuickNotes implementation weighted average of 4.25, closely followed by the residents' level of satisfaction with the "evaluation includes positive feedback" with a post-QuickNotes implementation weighted average of 4.24. The most notable increases in weighted averages from preimplementation to postimplementation were noted for "overall satisfaction" (20% increase for faculty, 37% for residents), "reflects actual criteria that matter" (36% increase for faculty, 73% for residents), faculty "opportunity for follow-up" (increase of 40%), resident "reflects overall trends" (increase of 37%), and resident "provides new information about my performance" (increase of 37%).
  
  
  CONCLUSIONS
  Our institutional adoption of QuickNotes into the resident evaluation process has been associated with an overall increased level of satisfaction in the evaluation process by both faculty and residents. The design of QuickNotes facilitates its integration into the resident training environment, as it is web based, easy to use, and has no additional cost over the standard New Innovations subscription. Although it is designed to capture snapshots of trainee behavior and performance, monthly reports through QuickNotes can be used effectively in conjunction with the more traditional end-of-rotation evaluations to show trends, identify areas of strength that should be reinforced, demonstrate areas needing improvement, allow for a more tailored individual education plan to be developed, and permit a more accurate determination of milestone progression.
authors.affiliation: Department of Surgery, Mount Carmel Health System, Columbus, Ohio.
authors.forename: Thomas H
authors.initials: TH
authors.lastname: Hartranft
authors.affiliation: Department of Surgery, Mount Carmel Health System, Columbus, Ohio.
authors.forename: Kurt
authors.initials: K
authors.lastname: Yandle
authors.affiliation: Mount Carmel Health Department of Family Practice, Mount Carmel Health System, Columbus, Ohio.
authors.forename: Tim
authors.initials: T
authors.lastname: Graham
authors.affiliation: Department of Surgery, Mount Carmel Health System, Columbus, Ohio.
authors.forename: Charles
authors.initials: C
authors.lastname: Holden
authors.affiliation: Department of Surgery, Mount Carmel Health System, Columbus, Ohio. Electronic address: lchambers@mchs.com.
authors.forename: Lowell W
authors.initials: LW
authors.lastname: Chambers
mesh_terms: D000046:Academic Medical Centers; D002983:Clinical Competence; D003430:Cross-Sectional Studies; D004503:Education, Medical, Graduate / Q000379:methods; D005260:Female; D000068156:Formative Feedback; D013502:General Surgery / Q000193:education*; D006801:Humans; D020407:Internet; D007396:Internship and Residency / Q000458:organization & administration*; D008297:Male; D008505:Medical Staff, Hospital / Q000706:statistics & numerical data; D012189:Retrospective Studies; D062606:Tertiary Care Centers; D014481:United States
keywords: Professionalism; graduate medical education; milestones; resident evaluation