title: Interrater agreement of two adverse drug reaction causality assessment methods: A randomised comparison of the Liverpool Adverse Drug Reaction Causality Assessment Tool and the World Health Organization-Uppsala Monitoring Centre system.
abstract: BACKGROUND
  A new method to assess causality of suspected adverse drug reactions, the Liverpool Adverse Drug Reaction Causality Assessment Tool (LCAT), showed high interrater agreement when used by its developers. Our aim was to compare the interrater agreement achieved by LCAT to that achieved by another causality assessment method, the World Health Organization-Uppsala Monitoring Centre system for standardised case causality assessment (WHO-UMC system), in our setting.
  
  
  METHODS
  Four raters independently assessed adverse drug reaction causality of 48 drug-event pairs, identified during a hospital-based survey. A randomised design ensured that no washout period was required between assessments with the two methods. We compared the methods' interrater agreement by calculating agreement proportions, kappa statistics, and the intraclass correlation coefficient. We identified potentially problematic questions in the LCAT by comparing raters' responses to individual questions.
  
  
  RESULTS
  Overall unweighted kappa was 0.61 (95% CI 0.43 to 0.80) on the WHO-UMC system and 0.27 (95% CI 0.074 to 0.46) on the LCAT. Pairwise unweighted Cohen kappa ranged from 0.33 to 1.0 on the WHO-UMC system and from 0.094 to 0.71 on the LCAT. The intraclass correlation coefficient was 0.86 (95% CI 0.74 to 0.92) on the WHO-UMC system and 0.61 (95% CI 0.39 to 0.77) on the LCAT. Two LCAT questions were identified as significant points of disagreement.
  
  
  CONCLUSIONS
  We were unable to replicate the high interrater agreement achieved by the LCAT developers and instead found its interrater agreement to be lower than that achieved when using the WHO-UMC system. We identified potential reasons for this and recommend priority areas for improving the LCAT.
authors.affiliation: Division of Clinical Pharmacology, Department of Medicine, University of Cape Town, Cape Town, South Africa.
authors.forename: Johannes P
authors.identifier: http://orcid.org/0000-0002-3511-1259
authors.initials: JP
authors.lastname: Mouton
authors.affiliation: Division of Clinical Pharmacology, Department of Medicine, University of Cape Town, Cape Town, South Africa.
authors.forename: Ushma
authors.initials: U
authors.lastname: Mehta
authors.affiliation: Division of Clinical Pharmacology, Department of Medicine, University of Cape Town, Cape Town, South Africa.
authors.forename: Dawn P
authors.initials: DP
authors.lastname: Rossiter
authors.affiliation: Division of Clinical Pharmacology, Department of Medicine, University of Cape Town, Cape Town, South Africa.
authors.forename: Gary
authors.initials: G
authors.lastname: Maartens
authors.affiliation: Division of Clinical Pharmacology, Department of Medicine, University of Cape Town, Cape Town, South Africa.
authors.forename: Karen
authors.initials: K
authors.lastname: Cohen
mesh_terms: D000328:Adult; D016907:Adverse Drug Reaction Reporting Systems / Q000706:statistics & numerical data*; D000465:Algorithms; D019380:Anti-HIV Agents / Q000009:adverse effects; D000995:Antitubercular Agents / Q000009:adverse effects; D064420:Drug-Related Side Effects and Adverse Reactions / Q000175:diagnosis*; D005260:Female; D006761:Hospitals; D006801:Humans; D007004:Hypoglycemic Agents / Q000009:adverse effects; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D010343:Patient Admission; D011897:Random Allocation; D015203:Reproducibility of Results; D013019:South Africa; D011795:Surveys and Questionnaires; D014665:Vasodilator Agents / Q000009:adverse effects; D014944:World Health Organization
chemical_list: D019380:Anti-HIV Agents; D000995:Antitubercular Agents; D007004:Hypoglycemic Agents; D014665:Vasodilator Agents