title: Development and Validation of an Assessment Tool for Competency in Critical Care Ultrasound.
abstract: BACKGROUND
  Point-of-care ultrasound is an emerging technology in critical care medicine. Despite requirements for critical care medicine fellowship programs to demonstrate knowledge and competency in point-of-care ultrasound, tools to guide competency-based training are lacking.
  
  
  OBJECTIVE
  We describe the development and validity arguments of a competency assessment tool for critical care ultrasound.
  
  
  METHODS
  A modified Delphi method was used to develop behaviorally anchored checklists for 2 ultrasound applications: "Perform deep venous thrombosis study (DVT)" and "Qualify left ventricular function using parasternal long axis and parasternal short axis views (Echo)." One live rater and 1 video rater evaluated performance of 28 fellows. A second video rater evaluated a subset of 10 fellows. Validity evidence for content, response process, and internal consistency was assessed.
  
  
  RESULTS
  An expert panel finalized checklists after 2 rounds of a modified Delphi method. The DVT checklist consisted of 13 items, including 1.00 global rating step (GRS). The Echo checklist consisted of 14 items, and included 1.00 GRS for each of 2 views. Interrater reliability evaluated with a Cohen kappa between the live and video rater was 1.00 for the DVT GRS, 0.44 for the PSLA GRS, and 0.58 for the PSSA GRS. Cronbach α was 0.85 for DVT and 0.92 for Echo.
  
  
  CONCLUSIONS
  The findings offer preliminary evidence for the validity of competency assessment tools for 2 applications of critical care ultrasound and data on live versus video raters.
authors.forename: Paru
authors.initials: P
authors.lastname: Patrawalla
authors.forename: Lewis Ari
authors.initials: LA
authors.lastname: Eisen
authors.forename: Ariel
authors.initials: A
authors.lastname: Shiloh
authors.forename: Brijen J
authors.initials: BJ
authors.lastname: Shah
authors.forename: Oleksandr
authors.initials: O
authors.lastname: Savenkov
authors.forename: Wendy
authors.initials: W
authors.lastname: Wise
authors.forename: Laura
authors.initials: L
authors.lastname: Evans
authors.forename: Paul
authors.initials: P
authors.lastname: Mayo
authors.forename: Demian
authors.initials: D
authors.lastname: Szyld
mesh_terms: D057189:Checklist / Q000379:methods; D002983:Clinical Competence / Q000592:standards*; D003422:Critical Care; D003697:Delphi Technique; D004503:Education, Medical, Graduate / Q000592:standards; D004521:Educational Measurement / Q000379:methods; D006801:Humans; D008991:Monitoring, Physiologic / Q000295:instrumentation / Q000592:standards; D019095:Point-of-Care Systems / Q000592:standards*; D011446:Prospective Studies; D011787:Quality of Health Care; D015203:Reproducibility of Results; D014463:Ultrasonography; D014743:Videotape Recording