title: On the evaluation of reliability, repeatability, and reproducibility of instrumental evaluation methods and measurement systems.
abstract: Validation of instrumental evaluation methods or measurement systems plays an important role in both pharmaceutical and cosmetic research and development. In practice, it is suggested that validation should be performed according to performance characteristics as described in the United States Pharmacopedia and National Formulary (USP/NF, 2000) for analytical methods validation. A validated method or measurement system is expected to achieve a certain degree of accuracy and reliability. However, it is a concern whether the test results obtained are repeatable (with similar test samples) and/or reproducible (under similar but slightly different experimental conditions). In this article, reliability and repeatability/reproducibility of a measurement system estimated within a mixed-effects nested design are monitored by relevant variability acceptance limits. A method based on the concept of empirical power (reproducibility) is used to determine these acceptance limits and thus ensure that there is a high probability of repeatability/reproducibility of the tests results. Formulas or procedures for sample size requirements for comparing the variabilities between products are derived. An example is presented to illustrate the use of the proposed method.
authors.affiliation: a L'OREAL Research & Innovation , Clichy , France.
authors.forename: Samir
authors.initials: S
authors.lastname: Salah
authors.affiliation: b Duke University School of Medicine , Durham , North Carolina , USA.
authors.forename: Shein-Chung
authors.initials: SC
authors.lastname: Chow
authors.affiliation: c Peking University Clinical Research Institute , Beijing , China.
authors.forename: Fuyu
authors.initials: F
authors.lastname: Song
mesh_terms: D005069:Evaluation Studies as Topic; D011336:Probability; D015203:Reproducibility of Results; D012107:Research Design; D014481:United States
keywords: Acceptance limit; empirical power; measurement systems; reproducibility probability; sample size for comparing variabilities; variability monitoring