Measuring clinical performance: comparison and validity of telephone survey and administrative data.
OBJECTIVE
To compare and validate self-reported telephone survey and administrative data for two Health Plan Employer Data and Information Set (HEDIS) performance measures: mammography and diabetic retinal exams.


METHODS
A telephone survey was administered to approximately 700 women and 600 persons with diabetes randomly chosen from each of two health maintenance organizations (HMOs).


METHODS
Agreement of survey and administrative data was assessed by using kappa coefficients. Validity measures were assessed by comparing survey and administrative data results to a standard: when the two sources agreed, that was accepted as the standard; when they differed, confirmatory information was sought from medical records to establish the standard. When confirmatory information was not available ranges of estimates consistent with the data were constructed by first assuming that all persons for whom no information was available had received the service and alternately that they had not received the service.


RESULTS
The kappas for mammography were .65 at both HMOs; for retinal exam they were .38 and .40. Sensitivity for both data sources was consistently high. However, specificity was lower for survey (range .44 to .66) than administrative data (.99 to 1.00). The positive predictive value was high for mammography using either data source but differed for retinal exam (survey .69 to .78; administrative data .99 to 1.00).


CONCLUSIONS
Administrative and survey data performed consistently in both HMOs. Although administrative data appeared to have greater specificity than survey data the validity and utility of different data sources for performance measurement have only begun to be explored.
Thompson
B L
BL
Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Division of Adult and Community Health, Health Care and Aging Studies Branch, USA. betsyt@vanion.com
O'Connor
P
P
Boyle
R
R
Hindmarsh
M
M
Salem
N
N
Simmons
K W
KW
Wagner
E
E
Oswald
J
J
Smith
S M
SM
D000328:Adult; D001943:Breast Neoplasms / Q000175:diagnosis / Q000517:prevention & control*; D003930:Diabetic Retinopathy / Q000175:diagnosis / Q000517:prevention & control*; D017598:Efficiency, Organizational; D005260:Female; D006264:Health Benefit Plans, Employee / Q000592:standards; D019538:Health Care Surveys / Q000379:methods*; D006279:Health Maintenance Organizations / Q000592:standards* / Q000706:statistics & numerical data; D006801:Humans; D008297:Male; D008327:Mammography / Q000706:statistics & numerical data; D008875:Middle Aged; D008910:Minnesota; D011314:Preventive Health Services / Q000706:statistics & numerical data*; D019984:Quality Indicators, Health Care; D015203:Reproducibility of Results; D013689:Telephone; D015351:Vision Screening / Q000706:statistics & numerical data; D014861:Washington