title: A Tool to Assess Family Medicine Residents' Patient Encounters Using Secure Messaging.
abstract: BACKGROUND
  Secure messages exchanged between patients and family medicine residents via an electronic health record (EHR) could be used to assess residents' clinical and communication skills, but the mechanism is not well understood.
  
  
  OBJECTIVE
  To design and test a secure messaging competency assessment for family medicine residents in a patient-centered medical home (PCMH).
  
  
  METHODS
  Using the existing literature and evidence-based guidelines, we designed an assessment tool to evaluate secure messaging competency for family medicine residents training in a PCMH. Core faculty performed 2-stage validity and reliability testing (n = 2 and n = 9, respectively). A series of randomly selected EHR secure messages (n = 45) were assessed from a sample of 10 residents across all years of training.
  
  
  RESULTS
  The secure message assessment tool provided data on a set of competencies and a framework for resident feedback. Assessment showed 10% (n = 2) of residents at the novice level, 50% (n = 10) as progressing, and 40% (n = 8) as proficient. The most common deficiencies for residents' secure messages related to communication rather than clinical competencies (n = 37 [90%] versus n = 4 [10%]). Interrater reliability testing ranged from 60% to 78% agreement and 20% to 44% disagreement. Disagreement centered on interpersonal communication factors. After 2 stages of testing, the assessment using residents' secure messages was incorporated into our existing evaluation process.
  
  
  CONCLUSIONS
  Assessing family medicine residents' secure messaging for patient encounters closed an evaluation gap in our family medicine program, and offered residents feedback on their clinical and communication skills in a PCMH.
authors.forename: Jung G
authors.initials: JG
authors.lastname: Kim
authors.forename: Carl G
authors.initials: CG
authors.lastname: Morris
authors.forename: Fred E
authors.initials: FE
authors.lastname: Heidrich
mesh_terms: D002983:Clinical Competence / Q000592:standards; D003142:Communication; D003479:Curriculum; D004521:Educational Measurement / Q000379:methods*; D057286:Electronic Health Records / Q000592:standards*; D005194:Family Practice / Q000193:education*; D005246:Feedback; D006801:Humans; D007396:Internship and Residency; D015588:Observer Variation; D018802:Patient-Centered Care; D010820:Physicians; D015203:Reproducibility of Results