title: Lessons Learned from a Cross-Model Validation between a Discrete Event Simulation Model and a Cohort State-Transition Model for Personalized Breast Cancer Treatment.
abstract: Breast cancer is the most common malignancy among women in developed countries. We developed a model (the Oncotyrol breast cancer outcomes model) to evaluate the cost-effectiveness of a 21-gene assay when used in combination with Adjuvant! Online to support personalized decisions about the use of adjuvant chemotherapy. The goal of this study was to perform a cross-model validation.
  
  
  
  The Oncotyrol model evaluates the 21-gene assay by simulating a hypothetical cohort of 50-year-old women over a lifetime horizon using discrete event simulation. Primary model outcomes were life-years, quality-adjusted life-years (QALYs), costs, and incremental cost-effectiveness ratios (ICERs). We followed the International Society for Pharmacoeconomics and Outcomes Research-Society for Medical Decision Making (ISPOR-SMDM) best practice recommendations for validation and compared modeling results of the Oncotyrol model with the state-transition model developed by the Toronto Health Economics and Technology Assessment (THETA) Collaborative. Both models were populated with Canadian THETA model parameters, and outputs were compared.
  
  
  
  The differences between the models varied among the different validation end points. The smallest relative differences were in costs, and the greatest were in QALYs. All relative differences were less than 1.2%. The cost-effectiveness plane showed that small differences in the model structure can lead to different sets of nondominated test-treatment strategies with different efficiency frontiers. We faced several challenges: distinguishing between differences in outcomes due to different modeling techniques and initial coding errors, defining meaningful differences, and selecting measures and statistics for comparison (means, distributions, multivariate outcomes).
  
  
  
  Cross-model validation was crucial to identify and correct coding errors and to explain differences in model outcomes. In our comparison, small differences in either QALYs or costs led to changes in ICERs because of changes in the set of dominated and nondominated strategies.
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)|Division of Public Health Decision Modelling, Health Technology Assessment and Health Economics, ONCOTYROL-Center for Personalized Cancer Medicine, Innsbruck, Austria (BJ, UR, CK, MK, MS, FK, US)
authors.forename: Beate
authors.initials: B
authors.lastname: Jahn
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)|Division of Public Health Decision Modelling, Health Technology Assessment and Health Economics, ONCOTYROL-Center for Personalized Cancer Medicine, Innsbruck, Austria (BJ, UR, CK, MK, MS, FK, US)
authors.forename: Ursula
authors.initials: U
authors.lastname: Rochau
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)|Division of Public Health Decision Modelling, Health Technology Assessment and Health Economics, ONCOTYROL-Center for Personalized Cancer Medicine, Innsbruck, Austria (BJ, UR, CK, MK, MS, FK, US)
authors.forename: Christina
authors.initials: C
authors.lastname: Kurzthaler
authors.affiliation: Toronto Health Economics and Technology Assessment (THETA) Collaborative, University of Toronto, ON, Canada (MP, MK)|Department of Emergency Medicine; University of Alberta, Edmonton, AB, Canada (MP)
authors.forename: Mike
authors.initials: M
authors.lastname: Paulden
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)|Division of Public Health Decision Modelling, Health Technology Assessment and Health Economics, ONCOTYROL-Center for Personalized Cancer Medicine, Innsbruck, Austria (BJ, UR, CK, MK, MS, FK, US)|Toronto Health Economics and Technology Assessment (THETA) Collaborative, University of Toronto, ON, Canada (MP, MK)
authors.forename: Martina
authors.initials: M
authors.lastname: Kluibenschädl
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)
authors.forename: Marjan
authors.initials: M
authors.lastname: Arvandi
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)|Division of Public Health Decision Modelling, Health Technology Assessment and Health Economics, ONCOTYROL-Center for Personalized Cancer Medicine, Innsbruck, Austria (BJ, UR, CK, MK, MS, FK, US)
authors.forename: Felicitas
authors.initials: F
authors.lastname: Kühne
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)|Department of Radiology, Yale University, New Haven, CT, USA (AG)|Institute for Technology Assessment, Massachusetts General Hospital, Harvard Medical School, Boston, MA USA (AG)|Alfried Krupp von Bohlen und Halbach Foundation-Institute for Health Systems Management, University of Duisburg-Essen, Essen, Germany (AG)
authors.forename: Alexander
authors.initials: A
authors.lastname: Goehler
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)|Division of Public Health Decision Modelling, Health Technology Assessment and Health Economics, ONCOTYROL-Center for Personalized Cancer Medicine, Innsbruck, Austria (BJ, UR, CK, MK, MS, FK, US)
authors.forename: Murray D
authors.initials: MD
authors.lastname: Krahn
authors.affiliation: Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health and Health Technology Assessment, UMIT-University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria (BJ, UR, CK, MK, MA, MS, FK, AG, US)|Division of Public Health Decision Modelling, Health Technology Assessment and Health Economics, ONCOTYROL-Center for Personalized Cancer Medicine, Innsbruck, Austria (BJ, UR, CK, MK, MS, FK, US)|Center for Health Decision Science, Department of Health Policy and Management, Harvard School of Public Health, Boston, MA, USA (US)|Institute for Technology Assessment and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA (US)
authors.forename: Uwe
authors.initials: U
authors.lastname: Siebert
mesh_terms: D000328:Adult; D018931:Antineoplastic Agents, Hormonal / Q000191:economics* / Q000627:therapeutic use*; D001943:Breast Neoplasms / Q000188:drug therapy* / Q000235:genetics*; D002170:Canada; D003362:Cost-Benefit Analysis / Q000706:statistics & numerical data; D003657:Decision Making; D005260:Female; D006801:Humans; D008390:Markov Chains; D008875:Middle Aged; D008962:Models, Theoretical; D057285:Precision Medicine; D019057:Quality-Adjusted Life Years
chemical_list: D018931:Antineoplastic Agents, Hormonal
keywords: Markov models; cost-effectiveness analysis; cross-model validation, decision analysis; discrete event simulation; state-transition model