title: Reducing Missed Primary Care Appointments in a Learning Health System: Two Randomized Trials and Validation of a Predictive Model.
abstract: BACKGROUND
  Collaborations between clinical/operational leaders and researchers are advocated to develop "learning health systems," but few practical examples are reported.
  
  
  OBJECTIVE
  To describe collaborative efforts to reduce missed appointments through an interactive voice response and text message (IVR-T) intervention, and to develop and validate a prediction model to identify individuals at high risk of missing appointments.
  Random assignment of 8804 adults with primary care appointments to a single IVR-T reminder or no reminder at an index clinic (IC) and 7497 at a replication clinic (RC) in an integrated health system in Denver, CO.
  
  
  METHODS
  Proportion of missed appointments; demographic, clinical, and appointment-specific predictors of missed appointments.
  
  
  RESULTS
  Patients receiving IVR-T had a lower rate of missed appointments than those receiving no reminder at the IC (6.5% vs. 7.5%, relative risk=0.85, 95% confidence interval, 0.72-1.00) and RC (8.2% vs. 10.5%, relative risk=0.76, 95% confidence interval, 0.65-0.89). A 10-variable prediction model for missed appointments demonstrated excellent discrimination (C-statistic 0.90 at IC, 0.89 at RC) and calibration (P=0.99 for Osius and McCullagh tests). Patients in the 3 lowest-risk quartiles missed 0.4% and 0.4% of appointments at the IC and RC, respectively, whereas patients in the highest-risk quartile missed 24.1% and 28.9% of appointments, respectively.
  
  
  CONCLUSIONS
  A single IVR-T call reduced missed appointments, whereas a locally validated prediction model accurately identified patients at high risk of missing appointments. These rigorous studies promoted dissemination of the intervention and prompted additional research questions from operational leaders.
authors.affiliation: *Institute for Health Research †Skyline Medical Office, Kaiser Permanente Colorado, Denver, CO.
authors.forename: John F
authors.initials: JF
authors.lastname: Steiner
authors.forename: Michael R
authors.initials: MR
authors.lastname: Shainline
authors.forename: Mary Christine
authors.initials: MC
authors.lastname: Bishop
authors.forename: Stan
authors.initials: S
authors.lastname: Xu
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D001071:Appointments and Schedules; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D008962:Models, Theoretical; D010349:Patient Compliance; D011320:Primary Health Care; D055815:Young Adult