title: Quality assessment of ordinal scale reproducibility: log-linear models provided useful information on scale structure.
abstract: OBJECTIVE
  In health research, ordinal scales are extensively used. Reproducibility of ratings using these scales is important to assess their quality. This study aimed to compare two methods analyzing reproducibility: weighted Kappa statistic and log-linear models.
  
  
  METHODS
  Contributions of each method to the reproducibility assessment of ratings using ordinal scales were compared using intra- and interobserver data chosen in three different fields: Crow's feet scale in dermatology, dysplasia scale in oncology, updated Sydney scale in gastroenterology.
  
  
  RESULTS
  Both methods provided an agreement level. In addition, log-linear models allowed evaluation of the structure of agreement. For the Crow's feet scale, both methods gave equivalent high agreement levels. For the dysplasia scale, log-linear models highlighted scale defects and Kappa statistic showed a moderate agreement. For the updated Sydney scale, log-linear models underlined a null distinguishability between two adjacent categories, whereas Kappa statistic gave a high global agreement level.
  
  
  CONCLUSIONS
  Methods that can investigate level and structure of agreement between ordinal ratings are valuable tools, since they may highlight heterogeneities within the scales structure and suggest modifications to improve their reproducibility.
authors.affiliation: Inserm U717, Département de Biostatistique et Informatique Médicale, Saint-Louis Hospital, 1 Avenue Claude Vellefaux, F-75010 Paris, University Paris 7, France. fabien.valet@paris7.jussieu.fr
authors.forename: Fabien
authors.initials: F
authors.lastname: Valet
authors.forename: Christiane
authors.initials: C
authors.lastname: Guinot
authors.forename: Khaled
authors.initials: K
authors.lastname: Ezzedine
authors.forename: Jean-Yves
authors.initials: JY
authors.lastname: Mary
mesh_terms: D000236:Adenoma / Q000473:pathology; D001706:Biopsy; D015179:Colorectal Neoplasms / Q000473:pathology; D003627:Data Interpretation, Statistical; D005260:Female; D016481:Helicobacter Infections / Q000473:pathology; D016480:Helicobacter pylori; D006801:Humans; D016014:Linear Models; D015588:Observer Variation; D015203:Reproducibility of Results; D015595:Skin Aging