title: Qualitative and quantitative outcomes of audience response systems as an educational tool in a plastic surgery residency program.
abstract: BACKGROUND
  In-training evaluations in graduate medical education have typically been challenging. Although the majority of standardized examination delivery methods have become computer-based, in-training examinations generally remain pencil-paper-based, if they are performed at all. Audience response systems present a novel way to stimulate and evaluate the resident-learner. The purpose of this study was to assess the outcomes of audience response systems testing as compared with traditional testing in a plastic surgery residency program.
  
  
  METHODS
  A prospective 1-year pilot study of 10 plastic surgery residents was performed using audience response systems-delivered testing for the first half of the academic year and traditional pencil-paper testing for the second half. Examination content was based on monthly "Core Quest" curriculum conferences. Quantitative outcome measures included comparison of pretest and posttest and cumulative test scores of both formats. Qualitative outcomes from the individual participants were obtained by questionnaire.
  
  
  RESULTS
  When using the audience response systems format, pretest and posttest mean scores were 67.5 and 82.5 percent, respectively; using traditional pencil-paper format, scores were 56.5 percent and 79.5 percent. A comparison of the cumulative mean audience response systems score (85.0 percent) and traditional pencil-paper score (75.0 percent) revealed statistically significantly higher scores with audience response systems (p = 0.01). Qualitative outcomes revealed increased conference enthusiasm, greater enjoyment of testing, and no user difficulties with the audience response systems technology.
  
  
  CONCLUSIONS
  The audience response systems modality of in-training evaluation captures participant interest and reinforces material more effectively than traditional pencil-paper testing does. The advantages include a more interactive learning environment, stimulation of class participation, immediate feedback to residents, and immediate tabulation of results for the educator. Disadvantages include start-up costs and lead-time preparation.
authors: Vancouver, British Columbia, Canada; and Detroit, Mich. From the Division of Plastic Surgery, British Columbia Children's Hospital and University of British Columbia, and the Departments of Otolaryngology and Surgery and the School of Medicine, Wayne State University.
authors: Jugpal S
authors: JS
authors: Arneja
authors: Kailash
authors: K
authors: Narasimhan
authors: David
authors: D
authors: Bouwman
authors: Patrick D
authors: PD
authors: Bridge
mesh_terms: D000328:Adult; D002983:Clinical Competence; D003162:Competency-Based Education; D003194:Computer-Assisted Instruction; D004503:Education, Medical, Graduate / Q000379:methods; D004521:Educational Measurement / Q000379:methods*; D005069:Evaluation Studies as Topic; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000458:organization & administration*; D008297:Male; D015397:Program Evaluation; D011446:Prospective Studies; D013518:Surgery, Plastic / Q000193:education*; D013664:Teaching Materials