title: Statistical evaluation of the use of concurrent controls in treatment screening studies.
abstract: A well-designed pilot study can advance science by providing essential preliminary data to support or motivate further research, refine study logistics, and demonstrate proof of concept. Often, the outcomes of such studies can be quantified by a success/failure dichotomy. For example, a novel compound may show activation of a neural pathway, or it may not. When an intervention's efficacy is quantified using a dichotomous outcome, probability mass functions can be enumerated to determine the probability that the observed result from a pilot study supports further evaluation of the intervention since there is only a finite, and often small, number of sample configurations possible. The purpose of this research was to determine the probability of an "efficacy signal" for pilot studies using one- and two-sample pilot study designs. Efficacy signal was defined as the probability of observing a more favorable response proportion relative to a historical control (one-sample setting) or to a concurrent control (two-sample setting). An enumeration study (exact simulation) was conducted to calculate the efficacy signal probability. One-sample study designs yielded higher probability of determining an efficacy signal than the two-sample setting; however, sampling variation must be accounted for. A 68% score confidence interval is recommended for this purpose.
authors.affiliation: Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA. carter.rickey@mayo.edu
authors.forename: Rickey E
authors.initials: RE
authors.lastname: Carter
authors.forename: Qian
authors.initials: Q
authors.lastname: Shi
mesh_terms: D002986:Clinical Trials as Topic; D015331:Cohort Studies; D010865:Pilot Projects; D011336:Probability; D018401:Sample Size; D013223:Statistics as Topic
keywords: confidence interval; pilot study; randomization; study design
references.citation: Sci Transl Med. 2011 Jul 6;3(90):90cm17
references.pmid: 21734173
references.citation: Clin Transl Sci. 2011 Oct;4(5):332-7
references.pmid: 22029804
references.citation: BMJ. 2005 Oct 15;331(7521):903
references.pmid: 16223828
references.citation: Arch Gen Psychiatry. 2006 May;63(5):484-9
references.pmid: 16651505
references.citation: J Transl Med. 2004 Oct 28;2(1):37
references.pmid: 15511289
references.citation: JAMA. 1999 Jun 2;281(21):1983; author reply 1984-5
references.pmid: 10359376
references.citation: J Clin Oncol. 2005 Mar 20;23(9):1794-5
references.pmid: 15699476
references.citation: Drug Alcohol Depend. 2009 Mar 1;100(3):187-93
references.pmid: 19058924
references.citation: J Clin Oncol. 2005 Oct 1;23(28):7199-206
references.pmid: 16192604
references.citation: J Thorac Oncol. 2010 Jul;5(7):932-4
references.pmid: 20581575
references.citation: Int J Radiat Oncol Biol Phys. 1980 Mar;6(3):371-4
references.pmid: 7390914
references.citation: Stat Med. 1995 Sep 15;14(17):1933-40
references.pmid: 8532986
references.citation: Schizophr Res. 2011 Jun;129(1):94-5
references.pmid: 21376537
references.citation: Crit Care Med. 2009 Jan;37(1 Suppl):S69-74
references.pmid: 19104228
references.citation: Appl Nurs Res. 2000 May;13(2):92-6
references.pmid: 10842905
references.citation: Dialogues Clin Neurosci. 2012 Mar;14(1):29-37
references.pmid: 22577302
references.citation: Clin Trials. 2006;3(4):335-48
references.pmid: 17060208
references.citation: Cancer Nurs. 1991 Jun;14(3):132-5
references.pmid: 2059956
references.citation: J Clin Oncol. 2010 May 20;28(15):2641-7
references.pmid: 20406933