A computer program for assessing interexaminer agreement when multiple ratings are made on a single subject.
This report describes a computer program for applying a new statistical method for determining levels of agreement, or reliability, when multiple examiners evaluate a single subject. The statistics that are performed include the following: an overall level of agreement, expressed as a percentage, that takes into account all possible levels of partial agreement; the same statistical approach for deriving a separate level of agreement of every examiner with every other examiner; and tests of the extent to which a giver examiner's rating (say a symptom score of three on a five-category ordinal rating scale) deviates from the group or overall average rating. These deviation scores are interpreted as standard Z statistics. Finally, both statistical and clinical criteria are provided to evaluate levels of interexaminer agreement.
Cicchetti
D V
DV
Yale Child Study Center, New Haven, CT 06510, USA. dom.cicchetti@yale.edu
Showalter
D
D
D003936:Diagnosis, Computer-Assisted; D006801:Humans; D019965:Neurocognitive Disorders / Q000145:classification / Q000175:diagnosis* / Q000523:psychology; D009483:Neuropsychological Tests / Q000706:statistics & numerical data*; D015588:Observer Variation; D011594:Psychometrics; D015203:Reproducibility of Results; D012984:Software