title: How many testers are needed to assure the usability of medical devices?
abstract: Before releasing a product, manufacturers have to follow a regulatory framework and meet standards, producing reliable evidence that the device presents low levels of risk in use. There is, though, a gap between the needs of the manufacturers to conduct usability testing while managing their costs, and the requirements of authorities for representative evaluation data. A key issue here is the number of users that should complete this evaluation to provide confidence in a product's safety. This paper reviews the US FDA's indication that a sample composed of 15 participants per major group (or a minimum of 25 users) should be enough to identify 90-97% of the usability problems and argues that a more nuanced approach to determining sample size (which would also fit well with the FDA's own concerns) would be beneficial. The paper will show that there is no a priori cohort size that can guarantee a reliable assessment, a point stressed by the FDA in the appendices to its guidance, but that manufacturers can terminate the assessment when appropriate by using a specific approach - illustrated in this paper through a case study - called the 'Grounded Procedure'.
authors.affiliation: Human Factors Research Group, Faculty of Engineering, The University of Nottingham, University Park, NG7 2RD, Nottingham, UK.
authors.forename: Simone
authors.initials: S
authors.lastname: Borsci
authors.forename: Robert D
authors.initials: RD
authors.lastname: Macredie
authors.forename: Jennifer L
authors.initials: JL
authors.lastname: Martin
authors.forename: Terry
authors.initials: T
authors.lastname: Young
mesh_terms: D004869:Equipment Safety / Q000592:standards / Q000639:trends; D004864:Equipment and Supplies; D006801:Humans; D062307:Medical Device Legislation / Q000592:standards* / Q000639:trends; D014481:United States; D014486:United States Food and Drug Administration
keywords: evaluation cohort; five-user assumption; medical devices; product safety; usability testing