title: Validation of the Economic and Health Outcomes Model of Type 2 Diabetes Mellitus (ECHO-T2DM).
abstract: The Economic and Health Outcomes Model of Type 2 Diabetes Mellitus (ECHO-T2DM) was developed to address study questions pertaining to the cost-effectiveness of treatment alternatives in the care of patients with type 2 diabetes mellitus (T2DM). Naturally, the usefulness of a model is determined by the accuracy of its predictions. A previous version of ECHO-T2DM was validated against actual trial outcomes and the model predictions were generally accurate. However, there have been recent upgrades to the model, which modify model predictions and necessitate an update of the validation exercises.
  
  
  
  The objectives of this study were to extend the methods available for evaluating model validity, to conduct a formal model validation of ECHO-T2DM (version 2.3.0) in accordance with the principles espoused by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the Society for Medical Decision Making (SMDM), and secondarily to evaluate the relative accuracy of four sets of macrovascular risk equations included in ECHO-T2DM.
  
  
  
  We followed the ISPOR/SMDM guidelines on model validation, evaluating face validity, verification, cross-validation, and external validation. Model verification involved 297 'stress tests', in which specific model inputs were modified systematically to ascertain correct model implementation. Cross-validation consisted of a comparison between ECHO-T2DM predictions and those of the seminal National Institutes of Health model. In external validation, study characteristics were entered into ECHO-T2DM to replicate the clinical results of 12 studies (including 17 patient populations), and model predictions were compared to observed values using established statistical techniques as well as measures of average prediction error, separately for the four sets of macrovascular risk equations supported in ECHO-T2DM. Sub-group analyses were conducted for dependent vs. independent outcomes and for microvascular vs. macrovascular vs. mortality endpoints.
  
  
  
  All stress tests were passed. ECHO-T2DM replicated the National Institutes of Health cost-effectiveness application with numerically similar results. In external validation of ECHO-T2DM, model predictions agreed well with observed clinical outcomes. For all sets of macrovascular risk equations, the results were close to the intercept and slope coefficients corresponding to a perfect match, resulting in high R2and failure to reject concordance using an F test. The results were similar for sub-groups of dependent and independent validation, with some degree of under-prediction of macrovascular events.
  
  
  
  ECHO-T2DM continues to match health outcomes in clinical trials in T2DM, with prediction accuracy similar to other leading models of T2DM.
authors.affiliation: The Swedish Institute for Health Economics, Box 2127, SE-220 02, Lund, Sweden. michael.willis@ihe.se.
authors.forename: Michael
authors.identifier: 0000-0002-6565-7025
authors.initials: M
authors.lastname: Willis
authors.affiliation: The Swedish Institute for Health Economics, Box 2127, SE-220 02, Lund, Sweden.
authors.forename: Pierre
authors.identifier: 0000-0002-6287-7508
authors.initials: P
authors.lastname: Johansen
authors.affiliation: The Swedish Institute for Health Economics, Box 2127, SE-220 02, Lund, Sweden.
authors.forename: Andreas
authors.identifier: 0000-0002-4604-7766
authors.initials: A
authors.lastname: Nilsson
authors.affiliation: The Swedish Institute for Health Economics, Box 2127, SE-220 02, Lund, Sweden.
authors.forename: Christian
authors.identifier: 0000-0001-7196-3363
authors.initials: C
authors.lastname: Asseburg
mesh_terms: D002986:Clinical Trials as Topic; D003362:Cost-Benefit Analysis; D003924:Diabetes Mellitus, Type 2 / Q000191:economics / Q000628:therapy*; D017408:Guidelines as Topic; D006801:Humans; D018803:Models, Economic; D017063:Outcome Assessment, Health Care / Q000379:methods*; D015203:Reproducibility of Results