title: Information-based measure of disagreement for more than two observers: a useful tool to compare the degree of observer disagreement.
abstract: BACKGROUND
  Assessment of disagreement among multiple measurements for the same subject by different observers remains an important problem in medicine. Several measures have been applied to assess observer agreement. However, problems arise when comparing the degree of observer agreement among different methods, populations or circumstances.
  
  
  METHODS
  The recently introduced information-based measure of disagreement (IBMD) is a useful tool for comparing the degree of observer disagreement. Since the proposed IBMD assesses disagreement between two observers only, we generalized this measure to include more than two observers.
  
  
  RESULTS
  Two examples (one with real data and the other with hypothetical data) were employed to illustrate the utility of the proposed measure in comparing the degree of disagreement.
  
  
  CONCLUSIONS
  The IBMD allows comparison of the disagreement in non-negative ratio scales across different populations and the generalization presents a solution to evaluate data with different number of observers for different cases, an important issue in real situations.A website for online calculation of IBMD and respective 95% confidence interval was additionally developed. The website is widely available to mathematicians, epidemiologists and physicians to facilitate easy application of this statistical strategy to their own data.
authors.affiliation: Health Information and Decision Sciences Department, Faculty of Medicine, University of Porto, Al. Prof. Hernâni Monteiro, 4200-319 Porto, Portugal. teresasarhen@gmail.com
authors.forename: Teresa
authors.initials: T
authors.lastname: Henriques
authors.forename: Luis
authors.initials: L
authors.lastname: Antunes
authors.forename: João
authors.initials: J
authors.lastname: Bernardes
authors.forename: Mara
authors.initials: M
authors.lastname: Matias
authors.forename: Diogo
authors.initials: D
authors.lastname: Sato
authors.forename: Cristina
authors.initials: C
authors.lastname: Costa-Santos
mesh_terms: D049628:Body Size; D015148:Cardiotocography; D005260:Female; D006339:Heart Rate; D006340:Heart Rate, Fetal; D006801:Humans; D020407:Internet; D007747:Labor Stage, First; D007749:Labor Stage, Third; D015233:Models, Statistical; D015588:Observer Variation; D011247:Pregnancy