title: Comparison of three instruments assessing the quality of economic evaluations: a practical exercise on economic evaluations of the surgical treatment of obesity.
abstract: OBJECTIVE
  The increasing use of full economic evaluations has led to the development of various instruments to assess their quality. The purpose of this study was to compare the frequently used British Medical Journal (BMJ) check-list and two new instruments: the Consensus Health Economic Criteria (CHEC) list and the Quality of Health Economic Studies (QHES) instrument. The analysis was based on a practical exercise on economic evaluations of the surgical treatment of obesity.
  
  
  METHODS
  The quality of nine selected studies was assessed independently by two health economists. To compare instruments, the Spearman rank correlation coefficient was calculated for each assessor. Moreover, the test-retest reliability for each instrument was assessed with the intraclass correlation coefficient (ICC) (3,1). Finally, the inter-rater agreement for each instrument was estimated at two levels: comparison of the total score of each article by the ICC(2,1) and comparison of results per item by kappa values.
  
  
  RESULTS
  The Spearman's rank correlation coefficient between instruments was usually high (rho > 0.70). Furthermore, test-retest reliability was good for every instruments, that is, 0.98 (95 percent CI, 0.86-0.99) for the BMJ check-list, 0.97 (95 percent CI, 0.73-0.98) for the CHEC list, and 0.95 (95 percent CI, 0.75-0.99) for the QHES instrument. However, inter-rater agreement was poor (kappa < 0.40 for most items and ICC(2,1) < or = 0.5).
  
  
  CONCLUSIONS
  The study shows that the results of the quality assessment of economic evaluations are not so much influenced by the instrument used but more by the assessor. Therefore, quality assessments should be performed by at least two independent experts and final scoring based on consensus.
authors.affiliation: School of Public Health-Unité de Socioéconomie de la santé (SESA), Université Catholique de Louvain, Brussels, Belgium. sophie.gerkens@uclouvain.be
authors.forename: Sophie
authors.initials: S
authors.lastname: Gerkens
authors.forename: Ralph
authors.initials: R
authors.lastname: Crott
authors.forename: Irina
authors.initials: I
authors.lastname: Cleemput
authors.forename: Jean-Paul
authors.initials: JP
authors.lastname: Thissen
authors.forename: Marie-Christine
authors.initials: MC
authors.lastname: Closon
authors.forename: Yves
authors.initials: Y
authors.lastname: Horsmans
authors.forename: Claire
authors.initials: C
authors.lastname: Beguin
mesh_terms: D049088:Bariatrics / Q000191:economics*; D003362:Cost-Benefit Analysis / Q000379:methods* / Q000592:standards*; D005069:Evaluation Studies as Topic; D006801:Humans; D011795:Surveys and Questionnaires