Penny-wise and pound-foolish: the impact of measurement error on sample size requirements in clinical trials.
BACKGROUND
Clinical research studies must compensate for measurement error by increasing the number of subjects that are studied, thereby increasing the financial costs of research and exposing greater numbers of subjects to study risks. In this article, we model the relationship between reliability and sample-size requirements and consider the potential tangible cost savings resulting from the decreased number of subjects needed when reliability of raters is improved or multiple ratings are used.


METHODS
Standard methods are used to model reliability based on the intraclass correlation coefficient (R) and to perform power calculations. The impact of multiple raters on reliability for a given baseline level of reliability is modeled according to the Spearman Brown formula.


RESULTS
Our models demonstrate that meaningful reductions in sample size requirements are gained from improvements in reliability. For example, improving reliability from R = .7 to R = .9 will decreases sample size requirements by 22%. Reliability is improved by training and by the use of the mean of multiple ratings. For example, if the reliability of a single rating is 0.7, the reliability of the mean of two ratings will be 0.8.


CONCLUSIONS
The costs to improve reliability either through rater training efforts or use of the mean of multiple ratings is cost effective because of the consequent reduction in number of subjects needed. Efforts to improve reliability and thus reduce subject requirements in a study also may lead to fewer patients bearing the burden of research participation and to a shortening of the duration of studies.
Perkins
D O
DO
Department of Psychiatry, University of North Carolina, Chapel Hill, North Carolina, USA.
Wyatt
R J
RJ
Bartko
J J
JJ
D002986:Clinical Trials as Topic / Q000191:economics / Q000706:statistics & numerical data; D006801:Humans; D001523:Mental Disorders / Q000188:drug therapy*; D010919:Placebos / Q000627:therapeutic use*; D015203:Reproducibility of Results
D010919:Placebos