title: Testing the construct validity of hospital care quality indicators: a case study on hip replacement.
abstract: Quality indicators are increasingly used to measure the quality of care and compare quality across hospitals. In the Netherlands over the past few years numerous hospital quality indicators have been developed and reported. Dutch indicators are mainly based on expert consensus and face validity and little is known about their construct validity. Therefore, we aim to study the construct validity of a set of national hospital quality indicators for hip replacements.
  
  
  
  We used the scores of 100 Dutch hospitals on national hospital quality indicators looking at care delivered over a two year period. We assessed construct validity by relating structure, process and outcome indicators using chi-square statistics, bootstrapped Spearman correlations, and independent sample t-tests. We studied indicators that are expected to associate as they measure the same clinical construct.
  
  
  
  Among the 28 hypothesized correlations, three associations were significant in the direction hypothesized. Hospitals with low scores on wound infections had high scores on scheduling postoperative appointments (p-value = 0.001) and high scores on not transfusing homologous blood (correlation coefficient = -0.28; p-value = 0.05). Hospitals with high scores on scheduling complication meetings, also had high scores on providing thrombosis prophylaxis (correlation coefficient = 0.21; p-value = 0.04).
  
  
  
  Despite the face validity of hospital quality indicators for hip replacement, construct validity seems to be limited. Although the individual indicators might be valid and actionable, drawing overall conclusions based on the whole indicator set should be done carefully, as construct validity could not be established. The factors that may explain the lack of construct validity are poor data quality, no adjustment for case-mix and statistical uncertainty.
authors.affiliation: Department of Public Health, Centre for Medical Decision Making, Erasmus MC, Dr. Molewaterplein 50, NA-2217, 3000 CA, Rotterdam, The Netherlands. c.fischer@erasmusmc.nl.|Department of Public Health, Amsterdam Medical Centre, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands. c.fischer@erasmusmc.nl.
authors.forename: Claudia
authors.initials: C
authors.lastname: Fischer
authors.affiliation: Department of Public Health, Centre for Medical Decision Making, Erasmus MC, Dr. Molewaterplein 50, NA-2217, 3000 CA, Rotterdam, The Netherlands.
authors.forename: Hester F
authors.initials: HF
authors.lastname: Lingsma
authors.affiliation: Department of Public Health, Amsterdam Medical Centre, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands.
authors.forename: Helen A
authors.initials: HA
authors.lastname: Anema
authors.affiliation: Department of Medical Decision Making, Leiden University Medical Centre, LUMC-J10-S, PO Box 9600, 2300 RC, Leiden, The Netherlands.
authors.forename: Job
authors.initials: J
authors.lastname: Kievit
authors.affiliation: Department of Public Health, Centre for Medical Decision Making, Erasmus MC, Dr. Molewaterplein 50, NA-2217, 3000 CA, Rotterdam, The Netherlands.
authors.forename: Ewout W
authors.initials: EW
authors.lastname: Steyerberg
authors.affiliation: Department of Public Health, Amsterdam Medical Centre, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands.
authors.forename: Niek
authors.initials: N
authors.lastname: Klazinga
mesh_terms: D019644:Arthroplasty, Replacement, Hip / Q000592:standards*; D006760:Hospitalization; D006761:Hospitals / Q000592:standards; D006801:Humans; D009426:Netherlands / Q000453:epidemiology; D010043:Outcome and Process Assessment, Health Care; D019984:Quality Indicators, Health Care / Q000592:standards*; D015203:Reproducibility of Results; D013530:Surgical Wound Infection / Q000453:epidemiology
keywords: Database; Health care quality; Hip replacement; Quality indicators; Validity