title: Determination of the psychometric properties of a behavioural marking system for obstetrical team training using high-fidelity simulation.
abstract: BACKGROUND
  To determine the effectiveness of high-fidelity simulation for team training, a valid and reliable tool is required. This study investigated the internal consistency, inter-rater reliability and test-retest reliability of two newly developed tools to assess obstetrical team performance.
  
  
  METHODS
  After research ethics board approval, multidisciplinary obstetrical teams participated in three sessions separated by 5-9 months and managed four high-fidelity simulation scenarios. Two tools, an 18-item Assessment of Obstetric Team Performance (AOTP) and a six-item Global Assessment of Obstetric Team Performance (GAOTP) were used.(5) Eight reviewers rated the DVDs of all teams' performances.
  
  
  RESULTS
  Two AOTP items were consistently incomplete and omitted from the analyses. Cronbach's α for the 16-item AOTP was 0.96, and 0.91 for the six-item GAOTP. The eight-rater α for the GAOTP was 0.81 (single-rater intra-class correlation coefficient, 0.34) indicating acceptable inter-rater reliability. The 'four-scenario' α for the 12 teams was 0.79 for session 1, 0.88 for session 2, and 0.86 for session 3, suggesting that performance is not being strongly affected by the context specificity of the cases. Pearson's correlation of team performance scores for the four scenarios were 0.59, 0.35, 0.40 and 0.33, and for the total score across scenarios it was 0.47, indicating moderate test-retest reliability.
  
  
  CONCLUSIONS
  The results from this study indicate that the GAOTP would be a sufficient assessment tool for obstetrical team performance using simulation provided that it is used to assess teams with at least eight raters to ensure a sufficiently stable score. This could allow the quantitative evaluation of an educational intervention.
authors: Department of Anesthesia, Women's College Hospital, 76 Grenville Street, Toronto, ON M5S 1B2, Canada. pam.morgan@utoronto.ca
authors: Pamela J
authors: PJ
authors: Morgan
authors: Deborah
authors: D
authors: Tregunno
authors: Richard
authors: R
authors: Pittini
authors: Jordan
authors: J
authors: Tarshis
authors: Glenn
authors: G
authors: Regehr
authors: Susan
authors: S
authors: Desousa
authors: Matt
authors: M
authors: Kurrek
authors: Ken
authors: K
authors: Milne
mesh_terms: D002983:Clinical Competence / Q000592:standards; D004521:Educational Measurement / Q000379:methods; D006801:Humans; D033183:Interdisciplinary Communication; D009774:Obstetrics / Q000193:education* / Q000458:organization & administration / Q000592:standards; D010348:Patient Care Team / Q000592:standards; D000078329:Workforce