title: Profiling provider outcome quality for pay-for-performance in the presence of missing data: a simulation approach.
abstract: OBJECTIVE
  Provider profiling of outcome performance has become increasingly common in pay-for-performance programs. For chronic conditions, a substantial proportion of patients eligible for outcome measures may be lost to follow-up, potentially compromising outcome profiling. In the context of primary care depression treatment, we assess the implications of missing data for the accuracy of alternative approaches to provider outcome profiling.
  
  
  METHODS
  We used data from the Improving Mood-Promoting Access to Collaborative Treatment trial and the Depression Improvement across Minnesota, Offering a New Direction initiative to generate parameters for a Monte Carlo simulation experiment.
  
  
  METHODS
  The patient outcome of interest is the rate of remission of depressive symptoms at 6 months among a panel of patients with major depression at baseline. We considered two alternative approaches to profiling this outcome: (1) a relative, or tournament style threshold, set at the 80th percentile of remission rate among all providers, and (2) an absolute threshold, evaluating whether providers exceed a specified remission rate (30 percent). We performed a Monte Carlo simulation experiment to evaluate the total error rate (proportion of providers who were incorrectly classified) under each profiling approach. The total error rate was partitioned into error from random sampling variability and error resulting from missing data. We then evaluated the accuracy of alternative profiling approaches under different assumptions about the relationship between missing data and depression remission.
  
  
  RESULTS
  Over a range of scenarios, relative profiling approaches had total error rates that were approximately 20 percent lower than absolute profiling approaches, and error due to missing data was approximately 50 percent lower for relative profiling. Most of the profiling error in the simulations was a result of random sampling variability, not missing data: between 11 and 21 percent of total error was attributable to missing data for relative profiling, while between 16 and 33 percent of total error was attributable to missing data for absolute profiling. Finally, compared with relative profiling, absolute profiling was much more sensitive to missing data that was correlated with the remission outcome.
  
  
  CONCLUSIONS
  Relative profiling approaches for pay-for-performance were more accurate and more robust to missing data than absolute profiling approaches.
authors: Department of Public Health, Weill Cornell Medical College, New York, NY 10065, USA. amr2015@med.cornell.edu
authors: Andrew M
authors: AM
authors: Ryan
authors: Yuhua
authors: Y
authors: Bao
mesh_terms: D003863:Depression / Q000191:economics* / Q000628:therapy*; D006801:Humans; D008605:Mental Health Services / Q000191:economics; D008910:Minnesota; D018803:Models, Economic; D009010:Monte Carlo Method; D017063:Outcome Assessment, Health Care; D010816:Physician Incentive Plans / Q000191:economics*; D058996:Quality Improvement / Q000191:economics*; D011787:Quality of Health Care / Q000191:economics; D012052:Reimbursement, Incentive / Q000191:economics*; D014481:United States
references: N Engl J Med. 2007 Feb 1;356(5):486-96
references: 17259444
references: Psychiatr Ann. 2008 Dec 1;38(12):793-801
references: 19668352
references: Circ Cardiovasc Qual Outcomes. 2010 Jan;3(1):2-3
references: 20123663
references: JAMA. 2009 Dec 9;302(22):2444-50
references: 19996399
references: Health Serv Res. 2012 Aug;47(4):1699-718
references: 22352894
references: Med Care. 2010 Jul;48(7):576-82
references: 20508531
references: Stat Med. 1988 Jan-Feb;7(1-2):305-15
references: 3353609
references: Fed Regist. 2011 May 6;76(88):26490-547
references: 21548401
references: JAMA. 2002 Dec 11;288(22):2836-45
references: 12472325
references: N Engl J Med. 2008 Jul 17;359(3):274-84
references: 18635432
references: N Engl J Med. 2010 Mar 18;362(11):1014-21
references: 20237347
references: Med Decis Making. 2007 Jan-Feb;27(1):9-20
references: 17237448
references: Annu Rev Clin Psychol. 2010;6:79-107
references: 20192796