title: Classification accuracy of claims-based methods for identifying providers failing to meet performance targets.
abstract: Quality assessment is critical for healthcare reform, but data sources are lacking for measurement of many important healthcare outcomes. With over 49 million people covered by Medicare as of 2010, Medicare claims data offer a potentially valuable source that could be used in targeted health care quality improvement efforts. However, little is known about the operating characteristics of provider profiling methods using claims-based outcome measures that may estimate provider performance with error. Motivated by the example of screening mammography performance, we compared approaches to identifying providers failing to meet guideline targets using Medicare claims data. We used data from the Breast Cancer Surveillance Consortium and linked Medicare claims to compare claims-based and clinical estimates of cancer detection rate. We then demonstrated the performance of claim-based estimates across a broad range of operating characteristics using simulation studies. We found that identification of poor performing providers was extremely sensitive to algorithm specificity, with no approach identifying more than 65% of poor performing providers when claims-based measures had specificity of 0.995 or less. We conclude that claims have the potential to contribute important information on healthcare outcomes to quality improvement efforts. However, to achieve this potential, development of highly accurate claims-based outcome measures should remain a priority.
authors.affiliation: Group Health Research Institute, Seattle, WA, U.S.A.; Department of Biostatistics, University of Washington, Seattle, WA, U.S.A.
authors.forename: Rebecca A
authors.initials: RA
authors.lastname: Hubbard
authors.forename: Rhondee
authors.initials: R
authors.lastname: Benjamin-Johnson
authors.forename: Tracy
authors.initials: T
authors.lastname: Onega
authors.forename: Rebecca
authors.initials: R
authors.lastname: Smith-Bindman
authors.forename: Weiwei
authors.initials: W
authors.lastname: Zhu
authors.forename: Joshua J
authors.initials: JJ
authors.lastname: Fenton
mesh_terms: D001499:Bayes Theorem; D001943:Breast Neoplasms / Q000145:classification / Q000175:diagnosis* / Q000453:epidemiology; D003198:Computer Simulation; D005069:Evaluation Studies as Topic; D005260:Female; D019983:Guideline Adherence / Q000592:standards / Q000706:statistics & numerical data; D006282:Health Personnel / Q000592:standards* / Q000706:statistics & numerical data; D006801:Humans; D007345:Insurance Claim Review; D008327:Mammography / Q000145:classification / Q000592:standards* / Q000706:statistics & numerical data; D008390:Markov Chains; D006278:Medicare / Q000191:economics / Q000592:standards*; D009010:Monte Carlo Method; D058991:Patient Protection and Affordable Care Act; D011159:Population Surveillance; D011785:Quality Assurance, Health Care / Q000331:legislation & jurisprudence / Q000379:methods* / Q000706:statistics & numerical data; D058996:Quality Improvement / Q000331:legislation & jurisprudence; D014481:United States / Q000453:epidemiology
keywords: Medicare; breast cancer; hierarchical models; provider profiling