Improving the reliability of physician performance assessment: identifying the "physician effect" on quality and creating composite measures.
BACKGROUND
The proliferation of efforts to assess physician performance underscore the need to improve the reliability of physician-level quality measures.


OBJECTIVE
Using diabetes care as a model, to address 2 key issues in creating reliable physician-level quality performance scores: estimating the physician effect on quality and creating composite measures.


METHODS
Retrospective longitudinal observational study.


METHODS
A national sample of physicians (n = 210) their patients with diabetes (n = 7574) participating in the National Committee on Quality Assurance-American Diabetes Association's Diabetes Provider Recognition Program.


METHODS
Using 11 diabetes process and intermediate outcome quality measures abstracted from the medical records of participants, we tested each measure for the magnitude of physician-level variation (the physician effect or "thumbprint"). We then combined measures with a substantial physician effect into a composite, physician-level diabetes quality score and tested its reliability.


RESULTS
We identified the lowest target values for each outcome measure for which there was a recognizable "physician thumbprint" (ie, intraclass correlation coefficient > or =0.30) to create a composite performance score. The internal consistency reliability (Cronbach's alpha) of the composite score, created by combining the process and outcome measures with an intraclass correlation coefficient > or =0.30, exceeded 0.80. The standard errors of the composite case-mix adjusted score were sufficiently small to discriminate those physicians scoring in the highest from those scoring in the lowest quartiles of the quality of care distribution with no overlap.


CONCLUSIONS
We conclude that the aggregation of well-tested quality measures that maximize the "physician effect" into a composite measure yields reliable physician-level quality of care scores for patients with diabetes.
Kaplan
Sherrie H
SH
Center for Health Policy Research and Department of Medicine, School of Medicine, University of California, Irvine, California 92697, USA. skaplan@uci.edu
Griffith
John L
JL
Price
Lori L
LL
Pawlson
L Gregory
LG
Greenfield
Sheldon
S
D000328:Adult; D000368:Aged; D003920:Diabetes Mellitus / Q000503:physiopathology / Q000628:therapy; D005260:Female; D006801:Humans; D008137:Longitudinal Studies; D008297:Male; D008485:Medical Audit; D008875:Middle Aged; D017063:Outcome Assessment, Health Care / Q000592:standards; D010818:Practice Patterns, Physicians' / Q000592:standards*; D011785:Quality Assurance, Health Care / Q000458:organization & administration*; D019984:Quality Indicators, Health Care / Q000592:standards*; D015203:Reproducibility of Results; D012189:Retrospective Studies; D017749:Total Quality Management / Q000379:methods