Reproducibility of measures of overuse of cataract surgery by three physician panels.
BACKGROUND
Assess the reproducibility of methods to measure overuse of cataract surgery.


OBJECTIVE
The objectives of this study are: (1) To determine the extent of agreement about clinical scenarios among, between, and within three physician panels; (2) to apply ratings of clinical scenarios from three panels to actual surgeries; and (3) to assess reproducibility of rates of appropriate use and overuse.


METHODS
Three physician panels scored 2,894 clinical scenarios for the appropriate use of cataract surgery. One thousand and twenty charts were abstracted and assigned to the clinical scenario that best corresponded to the patient's clinical situation. Two hundred and fifty nine clinical scenarios were required to assign the cases. Weighted kappa values, confidence intervals, and percentages of agreement were used to measure agreement among, between, and within panels.


RESULTS
The all ophthalmologist panel (OP) and the convened multispecialty panel (CM) each rate 92% of the cases as appropriate use, compared with 70% by the mail-in multispecialty panel (MM). The MM have higher uncertain (26% vs. 8% and 7%) and higher inappropriate use (3.5% vs. 0.1% and 1.9%). For the clinical scenarios, the CM and the MM have similar percentages of overuse (6.6%, 7.3%), in contrast to the OP (0.4%). The weighted kappa value for the overall level of agreement about the clinical scenarios among the three panels is 0.53, consistent with moderate agreement.


CONCLUSIONS
Study results demonstrate reproducibility for assessment of appropriate use of surgery between the OP and CM. However, both multispecialty panels rate more clinical scenarios as inappropriate use than the ophthalmologist panel. Thus, reproducibility between the CM and the OP may be attributable to the low percentage of overuse of cataract surgery in the study population. The overall level of agreement about the clinical scenarios among the panels is moderate.
Tobacman
J K
JK
Department of Internal Medicine, University of Iowa, Iowa City, 52242-1081, USA.
Scott
I U
IU
Cyphert
S
S
Zimmerman
B
B
D000368:Aged; D000369:Aged, 80 and over; D001291:Attitude of Health Personnel; D002387:Cataract Extraction / Q000706:statistics & numerical data*; D005260:Female; D006300:Health Services Misuse / Q000706:statistics & numerical data*; D006302:Health Services Research / Q000379:methods*; D006801:Humans; D008017:Life Expectancy; D008297:Male; D008875:Middle Aged; D009885:Ophthalmology; D018579:Patient Selection; D010820:Physicians / Q000523:psychology*; D015203:Reproducibility of Results; D012720:Severity of Illness Index; D013043:Specialties, Surgical; D014792:Visual Acuity