Confidence intervals for effect sizes: compliance and clinical significance in the Journal of Consulting and clinical Psychology.
OBJECTIVE
In 2005, the Journal of Consulting and Clinical Psychology (JCCP) became the first American Psychological Association (APA) journal to require statistical measures of clinical significance, plus effect sizes (ESs) and associated confidence intervals (CIs), for primary outcomes (La Greca, 2005). As this represents the single largest editorial effort to improve statistical reporting practices in any APA journal in at least a decade, in this article we investigate the efficacy of that change.


METHODS
All intervention studies published in JCCP in 2003, 2004, 2007, and 2008 were reviewed. Each article was coded for method of clinical significance, type of ES, and type of associated CI, broken down by statistical test (F, t, chi-square, r/R(2), and multivariate modeling).


RESULTS
By 2008, clinical significance compliance was 75% (up from 31%), with 94% of studies reporting some measure of ES (reporting improved for individual statistical tests ranging from eta(2) = .05 to .17, with reasonable CIs). Reporting of CIs for ESs also improved, although only to 40%. Also, the vast majority of reported CIs used approximations, which become progressively less accurate for smaller sample sizes and larger ESs (cf. Algina & Kessleman, 2003).


CONCLUSIONS
Changes are near asymptote for ESs and clinical significance, but CIs lag behind. As CIs for ESs are required for primary outcomes, we show how to compute CIs for the vast majority of ESs reported in JCCP, with an example of how to use CIs for ESs as a method to assess clinical significance.
Odgaard
Eric C
EC
Department of Psychology, University of South Florida, St. Petersburg, FL 33701, USA. eodgaard@mail.usf.edu
Fowler
Robert L
RL
D016001:Confidence Intervals; D003625:Data Collection / Q000706:statistics & numerical data*; D035081:Editorial Policies; D019983:Guideline Adherence; D006801:Humans; D010506:Periodicals as Topic; D011586:Psychology, Clinical / Q000706:statistics & numerical data*; D012017:Referral and Consultation / Q000706:statistics & numerical data*; D015203:Reproducibility of Results; D012106:Research / Q000706:statistics & numerical data*