title: Mobile technology for the facilitation of direct observation and assessment of student performance.
abstract: BACKGROUND
  We developed, implemented, and assessed a web-based clinical evaluation application (i.e., CEX app) for Internet-enabled mobile devices, including mobile phones. The app displays problem-specific checklists that correspond to training problems created by the Clerkship Directors in Internal Medicine.
  
  
  OBJECTIVE
  We hypothesized that use of the CEX app for directly observing students' clinical skills would be feasible and acceptable, and would demonstrate adequate reliability and validity.
  
  
  METHODS
  Between July 2010 and February 2012, 266 third-year medical students completed 5 to 10 formative CEXs during their internal medicine clerkship. The observers (attendings and residents), who performed the CEX, used the app to guide and document their observations, record their time observing and giving feedback to the students, and their overall satisfaction with the CEX app. Interrater reliability and validity were assessed with 17 observers who viewed 6 videotaped student-patient encounters, and by measuring the correlation between student CEX scores and their scores on subsequent standardized-patient Objective Structured Clinical Examination (OSCE) exams.
  
  
  RESULTS
  A total of 2,523 CEXs were completed by 411 observers. The average number of evaluations per student was 9.8 (± 1.8 SD), and the average number of CEXs completed per observer was 6 (± 11.8 SD). Observers spent less than 10 min on 45.3% of the CEXs and 68.6% of the feedback sessions. An overwhelming majority of observers (90.6%) reported satisfaction with the CEX. Interrater reliability was measured at 0.69 among the observers viewing the videotapes, and their ratings discriminated between competent and noncompetent performances. Student CEX grades, however, did not correlate with their end of 3rd-year OSCE scores.
  
  
  CONCLUSIONS
  The use of this CEX app is feasible and it captures students' clinical performance data with a high rate of user satisfaction. Our embedded checklists had adequate interrater reliability and concurrent validity. The grades measured on this app, however, were not predictive of subsequent student performance.
authors.affiliation: a Department of Medicine , Michigan State University College of Human Medicine , East Lansing , Michigan , USA.
authors.forename: Gary S
authors.initials: GS
authors.lastname: Ferenchick
authors.forename: David
authors.initials: D
authors.lastname: Solomon
authors.forename: Jami
authors.initials: J
authors.lastname: Foreback
authors.forename: Basim
authors.initials: B
authors.lastname: Towfiq
authors.forename: Kevin
authors.initials: K
authors.lastname: Kavanaugh
authors.forename: Larry
authors.initials: L
authors.lastname: Warbasse
authors.forename: James
authors.initials: J
authors.lastname: Addison
authors.forename: Frances
authors.initials: F
authors.lastname: Chames
authors.forename: Alvin
authors.initials: A
authors.lastname: Dandan
authors.forename: Asad
authors.initials: A
authors.lastname: Mohmand
mesh_terms: D057189:Checklist; D002982:Clinical Clerkship; D002983:Clinical Competence / Q000592:standards*; D005240:Feasibility Studies; D006801:Humans; D007388:Internal Medicine / Q000193:education; D008824:Michigan; D063731:Mobile Applications; D019370:Observation / Q000379:methods*; D013337:Students, Medical