title: Use of cognitive task analysis to guide the development of performance-based assessments for intraoperative decision making.
abstract: BACKGROUND
  There is a paucity of performance-based assessments that focus on intraoperative decision making. The purpose of this article is to review the performance outcomes and usefulness of two performance-based assessments that were developed using cognitive task analysis (CTA) frameworks.
  
  
  METHODS
  Assessment-A used CTA to create a "think aloud" oral examination that was administered while junior residents (PGY 1-2's, N = 69) performed a porcine-based laparoscopic cholecystectomy. Assessment-B used CTA to create a simulation-based, formative assessment of senior residents' (PGY 4-5's, N = 29) decision making during a laparoscopic ventral hernia repair. In addition to survey-based assessments of usefulness, a multiconstruct evaluation was performed using eight variables.
  
  
  RESULTS
  When comparing performance outcomes, both approaches revealed major deficiencies in residents' intraoperative decision-making skills. Multiconstruct evaluation of the two CTA approaches revealed assessment method advantages for five of the eight evaluation areas: (1) Cognitive Complexity, (2) Content Quality, (3) Content Coverage, (4) Meaningfulness, and (5) Transfer and Generalizability.
  
  
  CONCLUSIONS
  The two CTA performance assessments were useful in identifying significant training needs. While there are pros and cons to each approach, the results serve as a useful blueprint for program directors seeking to develop performance-based assessments for intraoperative decision making.
authors.affiliation: Associate Professor of Surgery, Vice Chair of Education and Patient Safety, University of Wisconsin, Department of Surgery, 600 Highland Ave-CSC 785B, Madison, WI 53792.
authors.forename: Carla M
authors.initials: CM
authors.lastname: Pugh
authors.forename: Debra A
authors.initials: DA
authors.lastname: DaRosa
mesh_terms: D002763:Cholecystectomy / Q000193:education / Q000592:standards; D002983:Clinical Competence; D003198:Computer Simulation; D003657:Decision Making; D004501:Education, Medical; D004521:Educational Measurement / Q000379:methods; D059685:Herniorrhaphy / Q000193:education / Q000592:standards; D006801:Humans; D013647:Task Performance and Analysis