title: Comprehensive Observations of Resident Evolution: A Novel Method for Assessing Procedure-Based Residency Training.
abstract: BACKGROUND
  Assessment of surgical skills in the operating room remains a challenge. Increasing documentation requirements of the Accreditation Council for Graduate Medical Education are necessitating mechanisms to document trainee competence without hindering operative turnover. The authors created a comprehensive electronic resource to facilitate plastic surgery training program compliance with changes mandated by Next Accreditation System Milestones and the ACGME.
  
  
  METHODS
  In 2013, the authors implemented the Comprehensive Observations of Resident Evolution, or CORE, a Web-based tool to assess plastic surgery residents. It comprises a rapid electronic assessment of resident operating room performance completed after each surgery; a data dashboard displaying graphical summaries of resident progress by case, Milestone, or current procedural terminology code; and an electronic Milestones tracker (MileMarker), which enables ongoing trainee assessments.
  
  
  RESULTS
  From January through October of 2014, 24 residents completed nearly 1300 Operative Entrustability Assessments. Thirty-eight percent of residents reported more immediate feedback regarding operative performance. The assessment demonstrates construct validity, which distinguishes novice residents from experienced residents. Individual case data identify resident-specific operative strengths and weaknesses. Using assessment data, the first two Clinical Competency Committee reviews were 81 percent and 87 percent shorter than Milestones pilot test site reports (average, 11.5 and 8 minutes versus 60 minutes per resident, respectively).
  
  
  CONCLUSIONS
  Comprehensive Observations of Resident Evolution is capable of capturing operative performance data on all operating room cases by primary current procedural terminology code. It increases immediate attending/trainee feedback and assessment transparency, enables trainee self-monitoring, and informs end-of-rotation reviews, programwide assessments, and tailoring of training to address specific needs. It is a valuable resource for tracking resident progress in real-time while maintaining compliance with evolving ACGME requirements.
authors.affiliation: Baltimore, Md. From the Department of Plastic and Reconstructive Surgery, Johns Hopkins University School of Medicine; and the Division of Plastic and Reconstructive Surgery, University of Maryland Medical Center.
authors.forename: Carisa M
authors.initials: CM
authors.lastname: Cooney
authors.forename: Damon S
authors.initials: DS
authors.lastname: Cooney
authors.forename: Ricardo J
authors.initials: RJ
authors.lastname: Bello
authors.forename: Branko
authors.initials: B
authors.lastname: Bojovic
authors.forename: Richard J
authors.initials: RJ
authors.lastname: Redett
authors.forename: Scott D
authors.initials: SD
authors.lastname: Lifchez
mesh_terms: D000068:Accreditation; D007396:Internship and Residency / Q000379:methods* / Q000639:trends*; D013518:Surgery, Plastic / Q000193:education*; D014481:United States
references.citation: Faulkner H, Regehr G, Martin J, Reznick R. Validation of an objective structured assessment of technical skill for surgical residents. Acad Med. 1996;71:1363–1365
references.citation: Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84:273–278
references.citation: Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ. The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE): A tool to assess surgical competence. Acad Med. 2012;87:1401–1407
references.citation: Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system–rationale and benefits. N Engl J Med. 2012;366:1051–1056
references.citation: Carter WA Jr. Milestone myths and misperceptions. J Grad Med Educ. 2014;6:18–20
references.citation: Cooney CM, Redett RJ III, Dorafshar AH, Zarrabi B, Lifchez SD. Integrating the NAS Milestones and handheld technology to improve residency training and assessment. J Surg Educ. 2014;71:39–42
references.citation: Lifchez SD, Cooney CM, Redett RJ III. The standardized professional encounter: A new model to assess professionalism and communication skills. J Grad Med Educ. 2014;7:230–233
references.citation: Nasca TJ, Weiss KB, Bagian JP, Brigham TP. The accreditation system after the “next accreditation system.” Acad Med. 2014;89:27–29