title: Optimal two-stage reliability studies.
abstract: The intraclass correlation is often used to assess the reliability of a measurement system. There is a considerable literature devoted to optimizing the standard assessment plan in which a number of subjects are measured repeatedly. We propose a two-stage investigation, here called a leveraged plan (LP), where in Stage I, we measure a number of subjects once. Then in Stage II, we select a subset of subjects with extreme initial measurements for repeated measurement. For a fixed total number of measurements, we show that the optimal LP provides a more precise estimate of the intraclass correlation coefficient than does the optimal standard plan (SP). We provide a table for finding the optimal LP given the true intraclass correlation and a specified precision for the estimate. For a fixed total number of measurements N, a nearly optimal LP makes roughly N/2 measurements in Stage I and then selects roughly N/6 extreme subjects to re-measure thrice each in Stage II. We also compare optimal leveraged with optimal SPs when there is a limit on the number of times each subject can be re-measured.
authors.affiliation: Business and Industrial Statistics Research Group (BISRG), Department of Statistics and Actuarial Sciences, University of Waterloo, Waterloo, Canada N2L 3G1.
authors.forename: Ryan
authors.initials: R
authors.lastname: Browne
authors.forename: Stefan H
authors.initials: SH
authors.lastname: Steiner
authors.forename: R Jock
authors.initials: RJ
authors.lastname: MacKay
mesh_terms: D000465:Algorithms; D016001:Confidence Intervals; D015340:Epidemiologic Research Design; D006801:Humans; D016013:Likelihood Functions; D015233:Models, Statistical; D015203:Reproducibility of Results