title: Comparing the use of global rating scale with checklists for the assessment of central venous catheterization skills using simulation.
abstract: The use of checklists is recommended for the assessment of competency in central venous catheterization (CVC) insertion. To explore the use of a global rating scale in the assessment of CVC skills, this study seeks to compare its use with two checklists, within the context of a formative examination using simulation. Video-recorded performances of CVC insertion by 34 first-year medical residents were reviewed by two independent, trained evaluators. Each evaluator used three assessment tools: a ten-item checklist, a 21-item checklist, and a nine-item global rating scale. Exploratory principal component analysis of the global rating scale revealed two factors, accounting for 84.1% of the variance: technical ability and safety. The two checklist scores correlated positively with the weighted factor score on technical ability (0.49 [95% CI 0.17-0.71] for the 10-item checklist; 0.43 [95% CI 0.10-0.67] for the 21-item checklist) and negatively with the weighted factor score on safety (-0.17 [95% CI -0.48-0.18] for the 10-item checklist; -0.13 [95% CI -0.45-0.22] for the 21-item checklist). A checklist score of <80% was strong indication of incompetence. However, a high checklist score did not preclude incompetence. Ratings using the global rating scale identified an additional 11 candidates (32%) who were deemed incompetent despite scoring >80% on both checklists. All these candidates committed serious errors. In conclusion, the practice of universal adoption of checklists as the preferred method of assessment of procedural skills should be questioned. The inclusion of global rating scales should be considered.
authors.affiliation: Department of Medicine, University of Calgary, 3330 Hospital Drive NW, Calgary, AB T2N 4N1, Canada. ima@ucalgary.ca
authors.forename: Irene W Y
authors.initials: IW
authors.lastname: Ma
authors.forename: Nadia
authors.initials: N
authors.lastname: Zalunardo
authors.forename: George
authors.initials: G
authors.lastname: Pachev
authors.forename: Tanya
authors.initials: T
authors.lastname: Beran
authors.forename: Melanie
authors.initials: M
authors.lastname: Brown
authors.forename: Rose
authors.initials: R
authors.lastname: Hatala
authors.forename: Kevin
authors.initials: K
authors.lastname: McLaughlin
mesh_terms: D001955:British Columbia; D002405:Catheterization, Central Venous / Q000379:methods / Q000592:standards*; D057189:Checklist / Q000379:methods / Q000592:standards; D002983:Clinical Competence / Q000592:standards*; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D004521:Educational Measurement / Q000379:methods / Q000592:standards*; D005260:Female; D006801:Humans; D007388:Internal Medicine / Q000193:education / Q000592:standards; D007396:Internship and Residency / Q000379:methods / Q000592:standards*; D008297:Male; D008953:Models, Anatomic; D015588:Observer Variation; D015203:Reproducibility of Results; D014743:Videotape Recording