title: Graphical aids for visualizing and interpreting patterns in departures from agreement in ordinal categorical observer agreement data.
abstract: When studying the agreement between two observers rating the same n units into the same k discrete ordinal categories, Bangdiwala (1985) proposed using the "agreement chart" to visually assess agreement. This article proposes that often it is more interesting to focus on the patterns of disagreement and visually understanding the departures from perfect agreement. The article reviews the use of graphical techniques for descriptively assessing agreement and disagreements, and also reviews some of the available summary statistics that quantify such relationships.
authors.affiliation: a Department of Biostatistics, Gillings School of Global Public Health , University of North Carolina at Chapel Hill , Chapel Hill , North Carolina , USA.
authors.forename: Shrikant I
authors.initials: SI
authors.lastname: Bangdiwala
mesh_terms: D003627:Data Interpretation, Statistical; D006801:Humans; D008488:Medical Illustration; D009103:Multiple Sclerosis / Q000175:diagnosis / Q000453:epidemiology; D015588:Observer Variation; D010363:Pattern Recognition, Automated / Q000592:standards / Q000706:statistics & numerical data*; D015203:Reproducibility of Results; D013223:Statistics as Topic / Q000592:standards
keywords: Agreement chart; disagreement; interobserver agreement; kappa