Setting standards for effectiveness: a comparison of expert panels and decision analysis.
OBJECTIVE
To compare criteria for coronary revascularization developed by the expert panel process and by decision analysis.


METHODS
We reviewed the medical records of 3080 chronic stable angina patients who either underwent coronary artery bypass graft surgery (CABG) or percutaneous transluminal coronary angioplasty (PTCA) and determined the agreement between appropriateness ratings made by two expert physician panels, one from the United States and the second from The Netherlands. We also evaluated the agreement between these panels' appropriateness ratings and a decision analytic model's effectiveness categories.


RESULTS
There was poor agreement between U.S. and Dutch panel appropriateness ratings for PTCA (kappa = 0.03) and slight agreement for bypass surgery (kappa = 0.18). Dutch ratings had substantial agreement with the decision analytic models effectiveness categories for both PTCA and CABG (kappa = 0.83 and 0.79, respectively) whereas there was no systematic agreement between U.S. ratings and the decision analytic model for PTCA and poor agreement for CABG (kappa = 0.00 and 0.18, respectively).


CONCLUSIONS
Although the level of agreement between expert panels and decision analysis on when a procedure is appropriate or effective may vary by procedure and the strength of the scientific evidence, we found that Dutch physicians agree much more strongly with decision analysis than U.S. physicians.
Bernstein
S J
SJ
Department of Medicine, University of Michigan, Ann Arbor 48109-0376, USA.
Hofer
T P
TP
Meijler
A P
AP
Rigter
H
H
D015906:Angioplasty, Balloon, Coronary / Q000592:standards; D001026:Coronary Artery Bypass / Q000592:standards; D003661:Decision Support Techniques; D006801:Humans; D009204:Myocardial Revascularization / Q000592:standards*; D009426:Netherlands; D009518:New York; D017063:Outcome Assessment, Health Care / Q000379:methods*; D017410:Practice Guidelines as Topic; D011367:Professional Staff Committees; D011446:Prospective Studies; D015203:Reproducibility of Results; D012189:Retrospective Studies