Does voluntary reporting bias hospital quality rankings?
BACKGROUND
The Leapfrog Group, a large consortium of private payers, collects voluntary, self-reported quality information from hospitals for several different procedures. Critics argue that voluntary reporting leads to over-representation of high quality hospitals.


METHODS
We used data from 1169 hospitals participating in the 2008 Leapfrog Survey for six high-risk procedures. We first ranked the hospitals who voluntarily submitted data to Leapfrog using a composite quality measure (a combination of mortality and hospital volume). We then created a similar set of rankings for a national sample of hospitals from the 2006 Nationwide Inpatient Sample (NIS). Next, we created four groups (quartiles) of hospital performance based on the national sample of hospitals. Finally, we compared the Leapfrog hospitals with the national sample of hospitals. If voluntary reporting resulted in bias, we would expect the Leapfrog hospitals to cluster in the top quartiles for each procedure.


RESULTS
The distribution of the Leapfrog hospitals within the NIS performance quartiles varied extensively across procedures. For abdominal aortic aneurysm repair and coronary artery bypass grafting, there was a clustering of hospitals in the two worst quartiles (64% and 65%, respectively). Conversely, for aortic valve replacement, pancreatectomy, percutaneous coronary interventions, and esophagectomy, there were 52%, 56%, 57%, and 61% of hospitals in the top half, respectively.


CONCLUSIONS
In the Leapfrog Survey, voluntary reporting did not lead to a systematic over-representation of high quality hospitals. There should be little concern that hospital quality rankings would have significant bias introduced by voluntary participation of hospitals.
Ghaferi
Amir A
AA
Michigan Surgical Collaborative for Outcomes Research and Evaluation, Department of Surgery, University of Michigan, Ann Arbor, Michigan 48109-0432, USA. aghaferi@umich.edu
Osborne
Nicholas H
NH
Dimick
Justin B
JB
D015982:Bias; D001026:Coronary Artery Bypass / Q000401:mortality / Q000706:statistics & numerical data; D003625:Data Collection; D016629:Esophagectomy / Q000401:mortality / Q000706:statistics & numerical data; D019918:Heart Valve Prosthesis Implantation / Q000401:mortality / Q000706:statistics & numerical data; D017052:Hospital Mortality; D006761:Hospitals / Q000592:standards*; D006801:Humans; D007297:Inpatients; D017063:Outcome Assessment, Health Care; D010180:Pancreatectomy / Q000401:mortality / Q000706:statistics & numerical data; D012107:Research Design; D014481:United States; D035141:Voluntary Programs