A simple computational alternative to analysis of variance formulas for estimating the k-judge reliability.
Formulas requiring the computation of only three standard deviations are presented for computing the interjudge reliability coefficient for any number of judges. These formulas yield coefficients identical to those obtained from a one-way repeated-measures analysis of variance. Even researchers with small handheld calculators can use this simple approach.
Feldt
Leonard S
LS
University of Iowa, USA.
Charter
Richard A
RA
D000465:Algorithms; D000704:Analysis of Variance; D006801:Humans; D015233:Models, Statistical; D015588:Observer Variation; D011594:Psychometrics; D015203:Reproducibility of Results