title: An architecture for a continuous, user-driven, and data-driven application of clinical guidelines and its evaluation.
abstract: OBJECTIVE
  Design, implement, and evaluate a new architecture for realistic continuous guideline (GL)-based decision support, based on a series of requirements that we have identified, such as support for continuous care, for multiple task types, and for data-driven and user-driven modes.
  
  
  METHODS
  We designed and implemented a new continuous GL-based support architecture, PICARD, which accesses a temporal reasoning engine, and provides several different types of application interfaces. We present the new architecture in detail in the current paper. To evaluate the architecture, we first performed a technical evaluation of the PICARD architecture, using 19 simulated scenarios in the preeclampsia/toxemia domain. We then performed a functional evaluation with the help of two domain experts, by generating patient records that simulate 60 decision points from six clinical guideline-based scenarios, lasting from two days to four weeks. Finally, 36 clinicians made manual decisions in half of the scenarios, and had access to the automated GL-based support in the other half. The measures used in all three experiments were correctness and completeness of the decisions relative to the GL.
  
  
  RESULTS
  Mean correctness and completeness in the technical evaluation were 1±0.0 and 0.96±0.03 respectively. The functional evaluation produced only several minor comments from the two experts, mostly regarding the output's style; otherwise the system's recommendations were validated. In the clinically oriented evaluation, the 36 clinicians applied manually approximately 41% of the GL's recommended actions. Completeness increased to approximately 93% when using PICARD. Manual correctness was approximately 94.5%, and remained similar when using PICARD; but while 68% of the manual decisions included correct but redundant actions, only 3% of the actions included in decisions made when using PICARD were redundant.
  
  
  CONCLUSIONS
  The PICARD architecture is technically feasible and is functionally valid, and addresses the realistic continuous GL-based application requirements that we have defined; in particular, the requirement for care over significant time frames. The use of the PICARD architecture in the domain we examined resulted in enhanced completeness and in reduction of redundancies, and is potentially beneficial for general GL-based management of chronic patients.
authors.affiliation: Medical Informatics Research Center, Department of Information Systems Engineering, Ben Gurion University of the Negev, Beer-Sheva, Israel. Electronic address: erezsh@bgu.ac.il.
authors.forename: Erez
authors.initials: E
authors.lastname: Shalom
authors.affiliation: Medical Informatics Research Center, Department of Information Systems Engineering, Ben Gurion University of the Negev, Beer-Sheva, Israel. Electronic address: yshahar@bgu.ac.il.
authors.forename: Yuval
authors.initials: Y
authors.lastname: Shahar
authors.affiliation: Soroka Medical Center, Department of OB/GYN, Faculty of Health Sciences, Ben Gurion University of the Negev, Beer-Sheva, Israel. Electronic address: lunenfld@bgu.ac.il.
authors.forename: Eitan
authors.initials: E
authors.lastname: Lunenfeld
mesh_terms: D020000:Decision Support Systems, Clinical; D006801:Humans; D008491:Medical Informatics Applications; D017410:Practice Guidelines as Topic; D017216:Telemedicine / Q000379:methods*; D014584:User-Computer Interface
keywords: Clinical decision support systems; Clinical practice guidelines; Computer-assisted decision-making; Evaluation; Guideline execution engine; Guideline simulation; Health care quality assurance; Knowledge representation; Telemedicine