title: Misclassification Risk of Tier-Based Physician Quality Performance Systems.
abstract: There is increasing interest in identifying high-quality physicians, such as whether physicians perform above or below a threshold level. To evaluate whether current methods accurately distinguish above- versus below-threshold physicians, we estimate misclassification rates for two-category identification systems.
  
  
  
  Claims data for Medicare fee-for-service beneficiaries residing in Florida or New York in 2010.
  
  
  
  Estimate colorectal cancer, glaucoma, and diabetes quality scores for 23,085 physicians. Use a beta-binomial model to estimate physician score reliabilities. Compute the proportion of physicians whose performance tier would be misclassified under three scoring systems.
  
  
  
  In the three scoring systems, misclassification ranges were 8.6-25.7 percent, 6.4-22.8 percent, and 4.5-21.7%. True positive rate ranges were 72.9-97.0 percent, 83.4-100.0 percent, and 34.7-88.2 percent. True negative rate ranges were 68.5-91.6 percent, 10.5-92.4 percent, and 81.1-99.9 percent. Positive predictive value ranges were 70.5-91.6 percent, 77.0-97.3 percent, and 55.2-99.1 percent.
  
  
  
  Current methods for profiling physicians on quality may produce misleading results, as the number of eligible events is typically small. Misclassification is a policy-relevant measure of the potential impact of tiering on providers, payers, and patients. Quantifying misclassification rates should inform the construction of high-performance networks and quality improvement initiatives.
authors.affiliation: Center for Effectiveness & Safety Research, Kaiser Permanente, Pasadena, CA.
authors.forename: John L
authors.initials: JL
authors.lastname: Adams
authors.affiliation: RAND Corporation, Santa Monica, CA.
authors.forename: Susan M
authors.initials: SM
authors.lastname: Paddock
mesh_terms: D000465:Algorithms; D018588:Fee-for-Service Plans; D005431:Florida; D006801:Humans; D007345:Insurance Claim Review; D058007:Physicians, Primary Care / Q000592:standards*; D011787:Quality of Health Care / Q000592:standards*; D018570:Risk Assessment / Q000145:classification*
keywords: High-performance networks; misclassification; provider profiling; quality of care; reliability