title: Investigating the disagreement between clinicians' ratings of patients in ICUs.
abstract: We present a Bayesian analysis of ordinal annotations made by clinicians of patients in intensive care. In particular, we investigate the different ways in which clinicians can disagree and how their disagreement is reduced once they take part in a recently proposed procedure (INSIGHT) that aims at improving consistency. The model combines a nonparametric function (loosely interpretable as the health of the patient) with clinician-specific generative procedures for producing the observed ordinal values. Our analysis provides valuable details of the rating behavior of the individual clinicians and shows that the INSIGHT procedure is particularly effective at removing (some) clinician-specific inconsistencies and biases.
authors: Simon
authors: S
authors: Rogers
authors: Derek
authors: D
authors: Sleeman
authors: John
authors: J
authors: Kinsella
mesh_terms: D001185:Artificial Intelligence; D003198:Computer Simulation; D006801:Humans; D007362:Intensive Care Units / Q000706:statistics & numerical data*; D008499:Medical Records / Q000592:standards*; D015233:Models, Statistical; D010820:Physicians / Q000706:statistics & numerical data*