title: Four hundred or more participants needed for stable contingency table estimates of clinical prediction rule performance.
abstract: OBJECTIVE
  To quantify variability in the results of statistical analyses based on contingency tables and discuss the implications for the choice of sample size for studies that derive clinical prediction rules.
  
  
  METHODS
  An analysis of three pre-existing sets of large cohort data (n = 4,062-8,674) was performed. In each data set, repeated random sampling of various sample sizes, from n = 100 up to n = 2,000, was performed 100 times at each sample size and the variability in estimates of sensitivity, specificity, positive and negative likelihood ratios, posttest probabilities, odds ratios, and risk/prevalence ratios for each sample size was calculated.
  
  
  RESULTS
  There were very wide, and statistically significant, differences in estimates derived from contingency tables from the same data set when calculated in sample sizes below 400 people, and typically, this variability stabilized in samples of 400-600 people. Although estimates of prevalence also varied significantly in samples below 600 people, that relationship only explains a small component of the variability in these statistical parameters.
  
  
  CONCLUSIONS
  To reduce sample-specific variability, contingency tables should consist of 400 participants or more when used to derive clinical prediction rules or test their performance.
authors.affiliation: School of Physiotherapy and Exercise Science, Curtin University, Kent Street, Bently, Perth, Western Australia 6102, Australia; Clinical Biomechanics Research Unit, Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Campusvej 55, Odense M 5230, Denmark. Electronic address: peter.kent@curtin.edu.au.
authors.forename: Peter
authors.initials: P
authors.lastname: Kent
authors.affiliation: Clinical Biomechanics Research Unit, Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Campusvej 55, Odense M 5230, Denmark; Division of Epidemiology, Dalla Lana School of Public Health, University of Toronto, 155 College St, Toronto, Ontario, ON M5T 3M7, Canada.
authors.forename: Eleanor
authors.initials: E
authors.lastname: Boyle
authors.affiliation: Department of Physiotherapy, Faculty of Medicine Nursing and Health Sciences, Monash University, McMahons Road Frankston, Melbourne, VIC 3199, Australia.
authors.forename: Jennifer L
authors.initials: JL
authors.lastname: Keating
authors.affiliation: The Modic Clinic, Jernbanegade 4, Odense 5000, Denmark.
authors.forename: Hanne B
authors.initials: HB
authors.lastname: Albert
authors.affiliation: Clinical Biomechanics Research Unit, Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Campusvej 55, Odense M 5230, Denmark; Nordic Institute of Chiropractic and Clinical Biomechanics, Campusvej 55, University of Southern Denmark, Odense M 5230, Denmark.
authors.forename: Jan
authors.initials: J
authors.lastname: Hartvigsen
mesh_terms: D001699:Biometry / Q000379:methods*; D015331:Cohort Studies; D003661:Decision Support Techniques; D006801:Humans; D009426:Netherlands; D016017:Odds Ratio; D011336:Probability; D018401:Sample Size; D012680:Sensitivity and Specificity
keywords: Clinical prediction rule; Decision support techniques; Epidemiologic research design; Predictive value of tests; Reproducibility of results; Sample size