title: SEQUENTIAL TESTING OF MEASUREMENT ERRORS IN INTER-RATER RELIABILITY STUDIES.
abstract: Inter-rater reliability is usually assessed by means of the intraclass correlation coefficient. Using two-way analysis of variance to model raters and subjects as random effects, we derive group sequential testing procedures for the design and analysis of reliability studies in which multiple raters evaluate multiple subjects. Compared with the conventional fixed sample procedures, the group sequential test has smaller average sample number. The performance of the proposed technique is examined using simulation studies and critical values are tabulated for a range of two-stage design parameters. The methods are exemplified using data from the Physician Reliability Study for diagnosis of endometriosis.
authors.affiliation: George Washington University ; Capital One.
authors.forename: Mei
authors.initials: M
authors.lastname: Jin
authors.affiliation: National Institute of Child Health and Human Development.
authors.forename: Aiyi
authors.initials: A
authors.lastname: Liu
authors.affiliation: National Institute of Child Health and Human Development.
authors.forename: Zhen
authors.initials: Z
authors.lastname: Chen
authors.affiliation: George Washington University.
authors.forename: Zhaohai
authors.initials: Z
authors.lastname: Li
keywords: Interim analysis; inter-rater reliability; intraclass correlation coefficient; measurement errors; sample size and power; two-way ANOVA