title: The statistical interpretation of pilot trials: should significance thresholds be reconsidered?
abstract: BACKGROUND
  In an evaluation of a new health technology, a pilot trial may be undertaken prior to a trial that makes a definitive assessment of benefit. The objective of pilot studies is to provide sufficient evidence that a larger definitive trial can be undertaken and, at times, to provide a preliminary assessment of benefit.
  
  
  METHODS
  We describe significance thresholds, confidence intervals and surrogate markers in the context of pilot studies and how Bayesian methods can be used in pilot trials. We use a worked example to illustrate the issues raised.
  
  
  RESULTS
  We show how significance levels other than the traditional 5% should be considered to provide preliminary evidence for efficacy and how estimation and confidence intervals should be the focus to provide an estimated range of possible treatment effects. We also illustrate how Bayesian methods could also assist in the early assessment of a health technology.
  
  
  CONCLUSIONS
  We recommend that in pilot trials the focus should be on descriptive statistics and estimation, using confidence intervals, rather than formal hypothesis testing and that confidence intervals other than 95% confidence intervals, such as 85% or 75%, be used for the estimation. The confidence interval should then be interpreted with regards to the minimum clinically important difference. We also recommend that Bayesian methods be used to assist in the interpretation of pilot trials. Surrogate endpoints can also be used in pilot trials but they must reliably predict the overall effect on the clinical outcome.
authors.forename: Ellen C
authors.initials: EC
authors.lastname: Lee
authors.forename: Amy L
authors.initials: AL
authors.lastname: Whitehead
authors.forename: Richard M
authors.initials: RM
authors.lastname: Jacques
authors.affiliation: Medical Statistics Group, School of Health and Related Research (ScHARR), University of Sheffield, 30 Regent Street, Sheffield S1 4DA, UK. s.a.julious@sheffield.ac.uk.
authors.forename: Steven A
authors.initials: SA
authors.lastname: Julious
mesh_terms: D001499:Bayes Theorem; D015415:Biomarkers; D016001:Confidence Intervals; D003627:Data Interpretation, Statistical; D006801:Humans; D007871:Leg Ulcer / Q000628:therapy; D010865:Pilot Projects; D012107:Research Design; D016896:Treatment Outcome
chemical_list: D015415:Biomarkers