title: Evaluation of Adjusted and Unadjusted Indirect Comparison Methods in Benefit Assessment. A Simulation Study for Time-to-event Endpoints.
abstract: BACKGROUND
  With the Act on the Reform of the Market for Medicinal Products (AMNOG) in Germany, pharmaceutical manufacturers are obliged to submit a dossier demonstrating added benefit of a new drug compared to an appropriate comparator. Underlying evidence was planned for registration purposes and therefore often does not meet the appropriate comparator as defined by the Federal Joint Committee (G-BA). For this reason AMNOG allows indirect comparisons to assess the extent of added benefit.
  
  
  OBJECTIVE
  The aim of this study is to evaluate the characteristics and applicability of adjusted indirect comparison described by Bucher and Matching-Adjusted Indirect Comparison (MAIC) in various situations within the early benefit assessment according to §35a Social Code Book 5. In particular, we consider time-to-event endpoints.
  
  
  METHODS
  We conduct a simulation study where we consider three different scenarios: I) similar study populations, II) dissimilar study populations without interactions and III) dissimilar study populations with interactions between treatment effect and effect modifiers. We simulate data from a Cox model with Weibull distributed survival times. Desired are unbiased effect estimates. We compare the power and the proportion of type 1 errors of the methods.
  
  
  RESULTS
  I) Bucher and MAIC perform equivalently well and yield unbiased effect estimates as well as proportions of type 1 errors below the significance level of 5 %. II) Both Bucher and MAIC yield unbiased effect estimates, but Bucher shows a higher power for detection of true added benefit than MAIC. III) Only MAIC, but not Bucher yields unbiased effect estimates. When using robust variance estimation MAIC yields a proportion of type 1 error close to 5 %. In general, power of all methods for indirect comparisons is low. An increasing loss of power for the indirect comparisons can be observed as the true treatment effects decrease.
  
  
  CONCLUSIONS
  Due to the great loss of power and the potential bias for indirect comparisons, head-to-head trials using the appropriate comparator as defined by the Federal Joint Committee should be conducted whenever possible. However, indirect comparisons are needed if no such direct evidence is available. To conduct indirect comparisons in case of a present common comparator and similar study populations in the trials to be compared, both Bucher and MAIC can be recommended. In case of using adjusted effect measures (such as Hazard Ratio), the violation of the similarity assumption has no relevant effect on the Bucher approach as long as interactions between treatment effect and effect modifiers are absent. Therefore Bucher can still be considered appropriate in this specific situation. In the authors' opinion, MAIC can be considered as an option (at least as sensitivity analysis to Bucher) if such interactions are present or cannot be ruled out. Nevertheless, in practice MAIC is potentially biased and should always be considered with utmost care.
authors.affiliation: Sarah Kühnast, Pfizer Deutschland GmbH, Linkstr. 10, 10785 Berlin, Germany, E-Mail: sarah.kuehnast@pfizer.com.
authors.forename: Sarah
authors.initials: S
authors.lastname: Kühnast
authors.forename: Julia
authors.initials: J
authors.lastname: Schiffner-Rohe
authors.forename: Jörg
authors.initials: J
authors.lastname: Rahnenführer
authors.forename: Friedhelm
authors.initials: F
authors.lastname: Leverkus
mesh_terms: D000970:Antineoplastic Agents / Q000627:therapeutic use; D015982:Bias; D015415:Biomarkers; D057186:Comparative Effectiveness Research / Q000379:methods*; D003198:Computer Simulation; D023381:Endpoint Determination / Q000379:methods*; D005858:Germany / Q000453:epidemiology; D006801:Humans; D015233:Models, Statistical; D009369:Neoplasms / Q000175:diagnosis / Q000188:drug therapy* / Q000453:epidemiology*; D017063:Outcome Assessment, Health Care / Q000379:methods*; D015203:Reproducibility of Results; D018570:Risk Assessment / Q000379:methods; D012680:Sensitivity and Specificity
chemical_list: D000970:Antineoplastic Agents; D015415:Biomarkers
keywords: AMNOG; Bucher; Indirect comparisons; MAIC; dissimilarity