title: Agreement of expert judgment in causality assessment of adverse drug reactions.
abstract: BACKGROUND
  Global introspection is, with operational algorithms and Bayes' theorem, one of the three main approaches used to assess the causal relationship between a drug treatment and the occurrence of an adverse event.
  
  
  OBJECTIVE
  To analyze and compare the judgments of five senior experts using global introspection about drug causation on a random set of putative adverse drug reactions.
  
  
  METHODS
  A random sample of 150 drug-effect pairs was constituted. For each pair, five senior experts had to independently assess the probability of drug causation from 0 to 1 by using a 100 mm visual analog scale (VAS). For analysis, those probabilities were secondarily split into seven levels of causality: excluded (0-0.05); unlikely (0.06-0.25); doubtful (0.26-0.45); unassessable/unclassifiable (0.46-0.55); plausible (0.56-0.75); likely (0.75-0.95); and certain (0.95-1). Agreement among the five experts was assessed using kappa coefficients (kappa).
  
  
  RESULTS
  The overall agreement between experts was poor (kappa=0.20), although significantly different from chance, and varied according to the level of causality. It was lower for the unlikely, doubtful, unassessable/unclassifiable, and plausible categories (kappa=0.03, 0.03, -0.01, and 0.13, respectively) than for VAS extremes: excluded, likely, and certain (kappa=0.40, 0.32, and 0.30, respectively).
  
  
  CONCLUSIONS
  This study confirms that experts express marked disagreements when assessing drug causality independently. The agreement rate was lower for intermediate levels of causality, especially when strong evidence was lacking for confirming or ruling out drug causality. Therefore, in a decision-making context, a step-by-step consensual approach such as the Delphi method seems necessary to make the assessment of such cases more reliable.
authors.affiliation: INSERM, U 657, 33076, Bordeaux Cedex, France. yannick.arimone@etud.u-bordeaux2.fr
authors.forename: Yannick
authors.initials: Y
authors.lastname: Arimone
authors.forename: Bernard
authors.initials: B
authors.lastname: Bégaud
authors.forename: Ghada
authors.initials: G
authors.lastname: Miremont-Salamé
authors.forename: Annie
authors.initials: A
authors.lastname: Fourrier-Réglat
authors.forename: Nicholas
authors.initials: N
authors.lastname: Moore
authors.forename: Mathieu
authors.initials: M
authors.lastname: Molimard
authors.forename: Françoise
authors.initials: F
authors.lastname: Haramburu
mesh_terms: D016907:Adverse Drug Reaction Reporting Systems / Q000706:statistics & numerical data; D015984:Causality; D032921:Consensus; D003627:Data Interpretation, Statistical; D064420:Drug-Related Side Effects and Adverse Reactions; D005104:Expert Testimony; D006801:Humans; D007600:Judgment; D018570:Risk Assessment / Q000379:methods / Q000706:statistics & numerical data*; D011795:Surveys and Questionnaires