title: Multi-Institution Validation of an Emergency Ultrasound Image Rating Scale-A Pilot Study.
abstract: BACKGROUND
  As bedside ultrasound (BUS) is being increasingly taught and incorporated into emergency medicine practice, measurement of BUS competency is becoming more important. The commonly adopted experiential approach to BUS competency has never been validated on a large scale, and has some limitations by design.
  
  
  OBJECTIVE
  Our aim was to introduce and report preliminary testing of a novel emergency BUS image rating scale (URS).
  
  
  METHODS
  Gallbladder BUS was selected as the test case. Twenty anonymous BUS image sets (still images and clips) were forwarded electronically to 16 reviewers (13 attendings, 3 fellows) at six training sites across the United States. Each reviewer rated the BUS sets using the pilot URS that consisted of three components, with numerical values assigned to each of the following aspects: Landmarks, Image Quality, and Annotations. Reviewers also decided whether or not each BUS set would be "Clinically Useful." Kendall τs were calculated as a measure of concordance among the reviewers.
  
  
  RESULTS
  Among the 13 attendings, image review experience ranged from 2-15 years, 5-300 scans per week, and averaged 7.8 years and 60 images. Kendall τs for each aspect of the URS were: Landmarks: 0.55; Image Quality: 0.57; Annotation: 0.26; Total Score: 0.63, and Clinical Usefulness: 0.45. All URS elements correlated significantly with clinical usefulness (p < 0.001). The correlation coefficient between each attending reviewer and the entire group ranged from 0.48-0.69, and was independent of image review experience beyond fellowship training.
  
  
  CONCLUSIONS
  Our novel URS had moderate-to-good inter-rater agreement in this pilot study. Based on these results, the URS will be modified for use in future investigations.
authors.affiliation: Department of Emergency Medicine, Advocate Christ Medical Center, Oak Lawn, Illinois; Department of Emergency Medicine, University of Illinois at Chicago, Chicago, Illinois.
authors.forename: Samuel H F
authors.initials: SH
authors.lastname: Lam
authors.affiliation: Department of Emergency Medicine, Cook County (Stroger) Hospital, Chicago, Illinois.
authors.forename: John
authors.initials: J
authors.lastname: Bailitz
authors.affiliation: Department of Emergency Medicine, University of Massachusetts Medical School, Worcester, Massachusetts.
authors.forename: David
authors.initials: D
authors.lastname: Blehar
authors.affiliation: Department of Emergency Medicine, York Hospital (Wellspan Health), York, Pennsylvania.
authors.forename: Brent A
authors.initials: BA
authors.lastname: Becker
authors.affiliation: Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts.
authors.forename: Beatrice
authors.initials: B
authors.lastname: Hoffmann
authors.affiliation: Department of Emergency Medicine, Massachusetts General Hospital, Boston, Massachusetts.
authors.forename: Andrew S
authors.initials: AS
authors.lastname: Liteplo
authors.affiliation: Rush Institute for Healthy Aging, Department of Internal Medicine, Rush University, Chicago, Illinois.
authors.forename: Kumar B
authors.initials: KB
authors.lastname: Rajan
authors.affiliation: Department of Emergency Medicine, Advocate Christ Medical Center, Oak Lawn, Illinois.
authors.forename: Michael
authors.initials: M
authors.lastname: Lambert
mesh_terms: D059925:Anatomic Landmarks / Q000000981:diagnostic imaging; D002983:Clinical Competence / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D004635:Emergency Medicine / Q000193:education*; D005704:Gallbladder / Q000000981:diagnostic imaging; D006801:Humans; D015588:Observer Variation; D010865:Pilot Projects; D019095:Point-of-Care Systems; D014463:Ultrasonography / Q000592:standards*
keywords: bedside ultrasound; competency; image rating scale; ultrasound education