title: Agreement studies in radiology research.
abstract: OBJECTIVE
  The goal of this study was to estimate the frequency and the quality of agreement studies published in diagnostic imaging journals.
  
  
  METHODS
  All studies published between January 2011 and December 2012 in four radiology journals were reviewed. Four trained readers evaluated agreement studies using a 24-item form that included the 15 items of the Guidelines for Reporting Reliability and Agreement Studies criteria.
  
  
  RESULTS
  Of 2229 source titles, 280 studies (13%) reported agreement. The mean number of patients per study was 81±99 (SD) (range, 0-180). Justification for sample size was found in 9 studies (3%). The number of raters was≤2 in 226 studies (81%). No intra-observer study was performed in 212 (76%) articles. Confidence intervals and interpretation of statistical estimates were provided in 98 (35%) and 147 (53%) of the studies, respectively. In 168 studies (60%), the agreement study was not mentioned in the discussion section. In 8 studies (3%), reporting of the agreement study was judged to be adequate. Twenty studies (7%) were dedicated to agreement.
  
  
  CONCLUSIONS
  Agreement studies are preliminary and not adequately reported. Studies dedicated to agreement are infrequent. They are research opportunities that should be promoted.
authors.affiliation: Interventional Neuroradiology Laboratory, centre hospitalier de l'université de Montréal (CHUM), Research Centre, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada.
authors.forename: B
authors.initials: B
authors.lastname: Farzin
authors.affiliation: Service of Neuroradiology, Department of Radiology, centre hospitalier de l'université de Montréal (CHUM), Notre-Dame Hospital, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada.
authors.forename: J-C
authors.initials: JC
authors.lastname: Gentric
authors.affiliation: Service of Neuroradiology, Department of Radiology, centre hospitalier de l'université de Montréal (CHUM), Notre-Dame Hospital, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada.
authors.forename: M
authors.initials: M
authors.lastname: Pham
authors.affiliation: Service of Neuroradiology, Department of Radiology, centre hospitalier de l'université de Montréal (CHUM), Notre-Dame Hospital, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada.
authors.forename: S
authors.initials: S
authors.lastname: Tremblay-Paquet
authors.affiliation: Service of Neuroradiology, Department of Radiology, centre hospitalier de l'université de Montréal (CHUM), Notre-Dame Hospital, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada.
authors.forename: L
authors.initials: L
authors.lastname: Brosseau
authors.affiliation: Service of Neuroradiology, Department of Radiology, centre hospitalier de l'université de Montréal (CHUM), Notre-Dame Hospital, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada.
authors.forename: C
authors.initials: C
authors.lastname: Roy
authors.affiliation: Interventional Neuroradiology Laboratory, centre hospitalier de l'université de Montréal (CHUM), Research Centre, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada.
authors.forename: S
authors.initials: S
authors.lastname: Jamali
authors.affiliation: Department of Mathematics and Statistics, université de Montréal, Faculty of Science, Pavillon André-Aisenstadt, 2920, chemin de la tour, local AA-5190, H3T 1J4 Montréal, Quebec, Canada.
authors.forename: M
authors.initials: M
authors.lastname: Chagnon
authors.affiliation: Department of Surgery, Division of Neurosurgery, University of Alberta Hospital, Mackenzie Health Sciences Centre, 8440-112 Street, T6G 2B7 Edmonton, Alberta, Canada.
authors.forename: T E
authors.initials: TE
authors.lastname: Darsaut
authors.affiliation: Service of Neuroradiology, Department of Radiology, centre hospitalier de l'université de Montréal (CHUM), Notre-Dame Hospital, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada.
authors.forename: F
authors.initials: F
authors.lastname: Guilbert
authors.affiliation: Inserm S894, Department of Radiology, Université Paris Descartes Sorbonne Paris Cité, centre hospitalier Sainte-Anne, 1, rue Cabanis, 75014 Paris, France.
authors.forename: O
authors.initials: O
authors.lastname: Naggara
authors.affiliation: Interventional Neuroradiology Laboratory, centre hospitalier de l'université de Montréal (CHUM), Research Centre, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada; Service of Neuroradiology, Department of Radiology, centre hospitalier de l'université de Montréal (CHUM), Notre-Dame Hospital, 1560 Sherbrooke east, Simard Z-12909, H2L4M1 Montreal, Quebec, Canada. Electronic address: jean.raymond@umontreal.ca.
authors.forename: J
authors.initials: J
authors.lastname: Raymond
mesh_terms: D032921:Consensus; D006801:Humans; D015588:Observer Variation; D011859:Radiography; D012107:Research Design
keywords: Agreement; Inter-observer; Intra-observer; Reliability; Reproducibility