Clinical trials in psychosocial medicine: methodologic and statistical considerations. Part II. Assessing reliability with equal-appearing interval scales.
The usual statistical procedures for assessing reliability among several raters do not apply to scales of the equal-appearing interval type. This is so primarily because the set of possible responses on such scales, namely the nonnegative integers 0,1,...(M-1), contains so few points that normal distribution theory cannot be invoked. Traditional analyses of such data, which include the one-way analysis of variance intraclass correlation, produce a reliability coefficient based on an assumed normally distributed response variable, and must therefore be interpreted with great caution. This paper discusses a statistical approach which exploits the discrete nature of the response variable and hence is more appropriate for such data.
D002986:Clinical Trials as Topic / Q000379:methods*; D006801:Humans; D008511:Medicine; D008960:Models, Psychological; D011569:Psychiatric Status Rating Scales; D013038:Specialization; D013223:Statistics as Topic