title: From hot hands to declining effects: the risks of small numbers.
abstract: About 25 years ago, a group of researchers demonstrated that there is no such thing as the "hot hand" in professional basketball. When a player hits 5 or 7 shots in a row (or misses 10 in a row), what's at work is random variation, nothing more. However, random causes do not stop players, coaches, fans, and media from talking about and acting on "hot hands," telling stories and making choices that ultimately are based on randomness. The same phenomenon is true in medicine. Some clinical trials with small numbers of events yielded positive findings, which in turn led clinicians, academics, and government officials to talk, telling stories and sometimes making choices that were later shown to be based on randomness. I provide some cardiovascular examples, such as the use of angiotensin receptor blockers for chronic heart failure, nesiritide for acute heart failure, and cytochrome P-450 (CYP) 2C19 genotyping for the acute coronary syndromes. I also review the more general "decline effect," by which drugs appear to yield a lower effect size over time. The decline effect is due at least in part to over interpretation of small studies, which are more likely to be noticed because of publication bias. As funders of research, we at the National Heart, Lung, and Blood Institute seek to support projects that will yield robust, credible evidence that will affect practice and policy in the right way. We must be alert to the risks of small numbers.
authors: Michael S
authors: MS
authors: Lauer
mesh_terms: D054058:Acute Coronary Syndrome / Q000201:enzymology / Q000235:genetics; D057911:Angiotensin Receptor Antagonists / Q000627:therapeutic use; D001189:Aryl Hydrocarbon Hydroxylases / Q000235:genetics; D002986:Clinical Trials as Topic / Q000379:methods / Q000592:standards; D065731:Cytochrome P-450 CYP2C19; D005838:Genotype; D006333:Heart Failure / Q000188:drug therapy; D006801:Humans; D045283:Natriuretic Agents / Q000627:therapeutic use; D020097:Natriuretic Peptide, Brain / Q000627:therapeutic use; D017063:Outcome Assessment, Health Care; D011336:Probability; D011338:Probability Theory; D017594:Publication Bias; D016032:Randomized Controlled Trials as Topic / Q000379:methods / Q000592:standards; D018401:Sample Size
chemical_list: D057911:Angiotensin Receptor Antagonists; D045283:Natriuretic Agents; D020097:Natriuretic Peptide, Brain; D001189:Aryl Hydrocarbon Hydroxylases; C045793:CYP2C19 protein, human; D065731:Cytochrome P-450 CYP2C19