title: Adaptive dynamic programming algorithms for sequential appointment scheduling with patient preferences.
abstract: OBJECTIVE
  A well-developed appointment system can help increase the utilization of medical facilities in an outpatient department. This paper outlines the development of an appointment system that can make an outpatient department work more efficiently and improve patient satisfaction level.
  
  
  METHODS
  A Markov decision process model is proposed to schedule sequential appointments with the consideration of patient preferences in order to maximize the patient satisfaction level. Adaptive dynamic programming algorithms are developed to avoid the curse of dimensionality. These algorithms can dynamically capture patient preferences, update the value of being a state, and thus improve the appointment decisions.
  
  
  RESULTS
  Experiments were conducted to investigate the performance of the algorithms. The convergence behaviors under different settings, including the number of iterations needed for convergence and the accuracy of results, were examined. Bias-adjusted Kalman filter step-sizes were found to lead to the best convergence behavior, which stabilized within 5000 iterations. As for the effects of exploration and exploitation, it resulted in the best convergence behavior when the probability of taking a myopically optimal action equaled 0.9. The performance of value function approximation algorithm was greatly affected by the combination of basis functions. Under different combinations, errors varied from 2.7% to 8.3%. More preferences resulted in faster convergence, but required longer computation time.
  
  
  CONCLUSIONS
  System parameters are adaptively updated as bookings are confirmed. The proposed appointment scheduling system could certainly contribute to better patient satisfaction level during the booking periods.
authors.affiliation: Department of Systems Engineering and Engineering Management, City University of Hong Kong, 83 Tat Chee Ave, Kowloon, Hong Kong.
authors.forename: Jin
authors.initials: J
authors.lastname: Wang
authors.affiliation: Department of Systems Engineering and Engineering Management, City University of Hong Kong, 83 Tat Chee Ave, Kowloon, Hong Kong. Electronic address: richard.fung@cityu.edu.hk.
authors.forename: Richard Y K
authors.initials: RY
authors.lastname: Fung
mesh_terms: D000465:Algorithms; D001071:Appointments and Schedules; D003198:Computer Simulation; D003199:Computer Systems; D017598:Efficiency, Organizational; D006801:Humans; D008390:Markov Chains; D009874:Operations Research; D010044:Outpatient Clinics, Hospital / Q000458:organization & administration*; D057240:Patient Preference; D017060:Patient Satisfaction; D013269:Stochastic Processes; D013997:Time Factors; D057188:Workflow
keywords: Adaptive dynamic programming; Appointment scheduling; Markov decision process; Outpatient department; Patient preferences