The MacTRAUMA TTL Assessment Tool: Developing a Novel Tool for Assessing Performance of Trauma Trainees: Initial Reliability Testing.
OBJECTIVE
To develop a novel assessment tool for trainees-led trauma resuscitation. Assess psychometric properties of the proposed tool. Evaluate feasibility and utility of the tool.


BACKGROUND
Trauma resuscitation is a structured and complex process involving unique sets of skills. There is currently no published structured formative evaluation tool for trauma trainees. Therefore, many trauma trainees rely upon limited, unstructured feedback on their performance. We developed a tool to assess trainee performance while leading a trauma resuscitation and to assist faculty in providing trainee feedback after the encounter.


METHODS
This study was conducted in a level I trauma centre in Ontario, Canada. Principles of learning theories, literature review, and clinical expert opinions were used to design a tool to assess clinical competence required to lead the resuscitation. In total, 5 critical domains were identified. High-fidelity simulation-based environment was used to test interrater reliability using intraclass correlation coefficients. To gauge feasibility, practicality, and utility of the tool, an online survey was sent to raters and trainees at the end of the study.


RESULTS
We found "excellent" agreement for "initial critical assessment" domain (0.80) and "moderate to good" agreement for the "communication and leadership" (0.67) and "clinical performance" domains (0.53). "Poor" agreement was identified for the "decision-making" domain (0.33). The coefficients for individual items reached "good" agreement for 5 items, and "moderate" agreement for 8 items. Intraclass correlation coefficients for the remaining 7 items were "fair" or "poor." Most raters agreed that items in the medical training domain were not applicable. Feedback from raters and trainees confirmed the feasibility and acceptability of the tool for formative feedback, in addition to some suggestions to enhance the tool.


CONCLUSIONS
MacTrauma TTL assessment tool is a novel tool for formative feedback for trainees' performance during trauma resuscitation. Initial psychometric property testing is promising. Further reliability and validity testing of the modified tool is needed. The tool has been shown to be feasible and acceptable by both trainees and faculty as a formative assessment tool.
Department of Surgery, McMaster University, Hamilton, Ontario, Canada.
Aussama
A
Nassar
Department of Surgery, McMaster University, Hamilton, Ontario, Canada; Trauma Program, Hamilton Health Sciences, Hamilton, Ontario, Canada.
Angela
A
Coates
Oklahoma University College of Medicine, Tulsa, Oklahoma.
Faiz
F
Tuma
Department of Surgery, McMaster University, Hamilton, Ontario, Canada.
Forough
F
Farrokhyar
Department of Surgery, McMaster University, Hamilton, Ontario, Canada. Electronic address: reid@mcmaster.ca.
Susan
S
Reid
D002983:Clinical Competence / Q000592:standards*; D003430:Cross-Sectional Studies; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement; D005240:Feasibility Studies; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000379:methods*; D008297:Male; D015588:Observer Variation; D009864:Ontario; D018794:Problem-Based Learning; D011594:Psychometrics; D015203:Reproducibility of Results; D000066908:Simulation Training / Q000379:methods*; D014193:Trauma Centers; D014194:Traumatology / Q000193:education*; D014947:Wounds and Injuries / Q000175:diagnosis / Q000628:therapy
Interpersonal and Communication Skills; Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; Systems-Based Practice; assessment; evaluation; performance; simulation; trauma