title: Validity of reliability: comparison of interrater reliabilities of psychopathological symptoms.
abstract: We have tested the stability of interrater reliability of psychiatric symptoms over a quarter of century using 2 rating scales. Interrater reliabilities of items of 2 psychiatric rating scales employed by 2 consecutive follow-ups were compared. Interrater reliabilites proved to be by and large stable. Interrater reliability depends on the standard deviation of the items scores. In addition to the traditional approach, a new statistical method for unifying the assessments from multiple raters is also presented. Using this method, we demonstrated that probabilities of correct ratings are higher in the absence of manifest symptoms, or in the presence of symptoms, as compared with cases characterized by middle scores. To interpret the relationships revealed in the setting of the experiment, we introduce for its theoretical designation the term "validity of reliability." It is recommended for evaluation of results of rating scales in the context of psychiatric nosology.
authors.affiliation: Society of Research of Endogenous Psychoses "2000", Budapest, Hungary. bpetho@enternet.hu
authors.forename: Bertalan
authors.initials: B
authors.lastname: Pethõ
authors.forename: Gábor
authors.initials: G
authors.lastname: Tusnády
authors.forename: András
authors.initials: A
authors.lastname: Vargha
authors.forename: Judit
authors.initials: J
authors.lastname: Tolna
authors.forename: Márta
authors.initials: M
authors.lastname: Farkas
authors.forename: Györgyi
authors.initials: G
authors.lastname: Vizkeleti
authors.forename: Agoston
authors.initials: A
authors.lastname: Tóth
authors.forename: András
authors.initials: A
authors.lastname: Szilágyi
authors.forename: István
authors.initials: I
authors.lastname: Bitter
authors.forename: András
authors.initials: A
authors.lastname: Kelemen
authors.forename: Pál
authors.initials: P
authors.lastname: Czobor
mesh_terms: D000367:Age Factors; D000553:Ambulatory Care; D001499:Bayes Theorem; D005260:Female; D005500:Follow-Up Studies; D006801:Humans; D008137:Longitudinal Studies; D001523:Mental Disorders / Q000145:classification / Q000175:diagnosis* / Q000523:psychology; D015233:Models, Statistical; D011336:Probability; D011569:Psychiatric Status Rating Scales / Q000706:statistics & numerical data*; D011594:Psychometrics; D015203:Reproducibility of Results; D012720:Severity of Illness Index; D016008:Statistical Distributions