title: Validity and feasibility of the EMG direct observation tool (EMG-DOT).
abstract: OBJECTIVE
  To develop a new workplace-based EMG direct observation tool (EMG-DOT) and gather validity evidence supporting its use for assessing electrodiagnostic skills among postgraduate medical trainees.
  
  
  METHODS
  The EMG-DOT was developed by experts using an iterative process. Validity evidence from content, response process, internal structure, relations to other variables, and consequences of testing was collected during the 2013-2014 academic year.
  
  
  RESULTS
  Of 3,412 studies performed by trainees during the study period, 299 (9%) were assessed using the EMG-DOT. Of these, 203 (68%) involved a physician rater and 96 (32%) involved a technician rater. The 14-item EMG-DOT had excellent internal-consistency reliability (Cronbach α 0.94). Correlations between individual items and criterion-referenced global ratings of performance ranged from 0.36 to 0.72 (all p < 0.001). Mean total scores increased from 70% to 80% over 4 months of the EMG rotation (p < 0.001) despite a corresponding significant increase in case complexity (0.21-0.74 on a 3-point rating scale; p < 0.001). Trainees reported that the observational assessment exercise improved their knowledge or skills in 82% of encounters (188/230) and that feedback generated by the EMG-DOT improved the quality of care provided to patients in 58% (133/230). Trainees were "satisfied" or "very satisfied" with the observational assessment exercise in 96% of encounters (234/243).
  
  
  CONCLUSIONS
  This study provides validity evidence supporting the use of EMG-DOT scores to assess electrodiagnostic skills of residents and fellows. The EMG-DOT can be used to inform milestone-based assessments of trainee performance in neurology, child neurology, physical medicine and rehabilitation, neuromuscular, and clinical neurophysiology training programs.
authors.affiliation: From the Department of Neurology (A.N.L.H., R.S.L., E.J.S., J.C.W., L.K.J.), Mayo Clinic College of Medicine, Rochester, MN; Department of Neurology (D.I.R.), Mayo Clinic College of Medicine, Jacksonville, FL; and Department of Medical Education (D.J., Y.S.P.), University of Illinois College of Medicine, Chicago, IL. leep.andrea@mayo.edu.
authors.forename: Andrea N
authors.initials: AN
authors.lastname: Leep Hunderfund
authors.affiliation: From the Department of Neurology (A.N.L.H., R.S.L., E.J.S., J.C.W., L.K.J.), Mayo Clinic College of Medicine, Rochester, MN; Department of Neurology (D.I.R.), Mayo Clinic College of Medicine, Jacksonville, FL; and Department of Medical Education (D.J., Y.S.P.), University of Illinois College of Medicine, Chicago, IL.
authors.forename: Devon I
authors.initials: DI
authors.lastname: Rubin
authors.affiliation: From the Department of Neurology (A.N.L.H., R.S.L., E.J.S., J.C.W., L.K.J.), Mayo Clinic College of Medicine, Rochester, MN; Department of Neurology (D.I.R.), Mayo Clinic College of Medicine, Jacksonville, FL; and Department of Medical Education (D.J., Y.S.P.), University of Illinois College of Medicine, Chicago, IL.
authors.forename: Ruple S
authors.initials: RS
authors.lastname: Laughlin
authors.affiliation: From the Department of Neurology (A.N.L.H., R.S.L., E.J.S., J.C.W., L.K.J.), Mayo Clinic College of Medicine, Rochester, MN; Department of Neurology (D.I.R.), Mayo Clinic College of Medicine, Jacksonville, FL; and Department of Medical Education (D.J., Y.S.P.), University of Illinois College of Medicine, Chicago, IL.
authors.forename: Eric J
authors.initials: EJ
authors.lastname: Sorenson
authors.affiliation: From the Department of Neurology (A.N.L.H., R.S.L., E.J.S., J.C.W., L.K.J.), Mayo Clinic College of Medicine, Rochester, MN; Department of Neurology (D.I.R.), Mayo Clinic College of Medicine, Jacksonville, FL; and Department of Medical Education (D.J., Y.S.P.), University of Illinois College of Medicine, Chicago, IL.
authors.forename: James C
authors.initials: JC
authors.lastname: Watson
authors.affiliation: From the Department of Neurology (A.N.L.H., R.S.L., E.J.S., J.C.W., L.K.J.), Mayo Clinic College of Medicine, Rochester, MN; Department of Neurology (D.I.R.), Mayo Clinic College of Medicine, Jacksonville, FL; and Department of Medical Education (D.J., Y.S.P.), University of Illinois College of Medicine, Chicago, IL.
authors.forename: Lyell K
authors.initials: LK
authors.lastname: Jones
authors.affiliation: From the Department of Neurology (A.N.L.H., R.S.L., E.J.S., J.C.W., L.K.J.), Mayo Clinic College of Medicine, Rochester, MN; Department of Neurology (D.I.R.), Mayo Clinic College of Medicine, Jacksonville, FL; and Department of Medical Education (D.J., Y.S.P.), University of Illinois College of Medicine, Chicago, IL.
authors.forename: Dorthea
authors.initials: D
authors.lastname: Juul
authors.affiliation: From the Department of Neurology (A.N.L.H., R.S.L., E.J.S., J.C.W., L.K.J.), Mayo Clinic College of Medicine, Rochester, MN; Department of Neurology (D.I.R.), Mayo Clinic College of Medicine, Jacksonville, FL; and Department of Medical Education (D.J., Y.S.P.), University of Illinois College of Medicine, Chicago, IL.
authors.forename: Yoon Soo
authors.initials: YS
authors.lastname: Park
mesh_terms: D002983:Clinical Competence; D004503:Education, Medical, Graduate; D004521:Educational Measurement / Q000379:methods*; D004576:Electromyography; D005240:Feasibility Studies; D005246:Feedback; D007722:Health Knowledge, Attitudes, Practice; D006801:Humans; D007396:Internship and Residency; D007858:Learning; D009462:Neurology / Q000193:education / Q000379:methods; D010810:Physical and Rehabilitation Medicine / Q000193:education; D011787:Quality of Health Care; D015203:Reproducibility of Results