Agreement of expert judgment in causality assessment of adverse drug reactions.
BACKGROUND
Global introspection is, with operational algorithms and Bayes' theorem, one of the three main approaches used to assess the causal relationship between a drug treatment and the occurrence of an adverse event.


OBJECTIVE
To analyze and compare the judgments of five senior experts using global introspection about drug causation on a random set of putative adverse drug reactions.


METHODS
A random sample of 150 drug-effect pairs was constituted. For each pair, five senior experts had to independently assess the probability of drug causation from 0 to 1 by using a 100 mm visual analog scale (VAS). For analysis, those probabilities were secondarily split into seven levels of causality: excluded (0-0.05); unlikely (0.06-0.25); doubtful (0.26-0.45); unassessable/unclassifiable (0.46-0.55); plausible (0.56-0.75); likely (0.75-0.95); and certain (0.95-1). Agreement among the five experts was assessed using kappa coefficients (kappa).


RESULTS
The overall agreement between experts was poor (kappa=0.20), although significantly different from chance, and varied according to the level of causality. It was lower for the unlikely, doubtful, unassessable/unclassifiable, and plausible categories (kappa=0.03, 0.03, -0.01, and 0.13, respectively) than for VAS extremes: excluded, likely, and certain (kappa=0.40, 0.32, and 0.30, respectively).


CONCLUSIONS
This study confirms that experts express marked disagreements when assessing drug causality independently. The agreement rate was lower for intermediate levels of causality, especially when strong evidence was lacking for confirming or ruling out drug causality. Therefore, in a decision-making context, a step-by-step consensual approach such as the Delphi method seems necessary to make the assessment of such cases more reliable.
Arimone
Yannick
Y
INSERM, U 657, 33076, Bordeaux Cedex, France. yannick.arimone@etud.u-bordeaux2.fr
Bégaud
Bernard
B
Miremont-Salamé
Ghada
G
Fourrier-Réglat
Annie
A
Moore
Nicholas
N
Molimard
Mathieu
M
Haramburu
Françoise
F
D016907:Adverse Drug Reaction Reporting Systems / Q000706:statistics & numerical data; D015984:Causality; D032921:Consensus; D003627:Data Interpretation, Statistical; D064420:Drug-Related Side Effects and Adverse Reactions; D005104:Expert Testimony; D006801:Humans; D007600:Judgment; D018570:Risk Assessment / Q000379:methods / Q000706:statistics & numerical data*; D011795:Surveys and Questionnaires