title: Simpson's paradox in psychological science: a practical guide.
abstract: The direction of an association at the population-level may be reversed within the subgroups comprising that population-a striking observation called Simpson's paradox. When facing this pattern, psychologists often view it as anomalous. Here, we argue that Simpson's paradox is more common than conventionally thought, and typically results in incorrect interpretations-potentially with harmful consequences. We support this claim by reviewing results from cognitive neuroscience, behavior genetics, clinical psychology, personality psychology, educational psychology, intelligence research, and simulation studies. We show that Simpson's paradox is most likely to occur when inferences are drawn across different levels of explanation (e.g., from populations to subgroups, or subgroups to individuals). We propose a set of statistical markers indicative of the paradox, and offer psychometric solutions for dealing with the paradox when encountered-including a toolbox in R for detecting Simpson's paradox. We show that explicit modeling of situations in which the paradox might occur not only prevents incorrect interpretations of data, but also results in a deeper understanding of what data tell us about the world.
authors: Department of Psychological Methods, University of Amsterdam Amsterdam, Netherlands ; Medical Research Council - Cognition and Brain Sciences Unit Cambridge, UK.
authors: Rogier A
authors: RA
authors: Kievit
authors: Willem E
authors: WE
authors: Frankenhuis
authors: Lourens J
authors: LJ
authors: Waldorp
authors: Denny
authors: D
authors: Borsboom
keywords: Simpson's paradox; ecological fallacy; measurement; paradox; reductionism; statistical inference
references: Pers Soc Psychol Bull. 2003 Jan;29(1):14-27
references: 15272956
references: Mem Cognit. 2001 Mar;29(2):193-208
references: 11352202
references: Prev Sci. 2000 Dec;1(4):173-81
references: 11523746
references: Int J Epidemiol. 2006 Jun;35(3):520-5
references: 16645033
references: J Pers. 1992 Jun;60(2):175-215
references: 1635039
references: Annu Rev Psychol. 2007;58:593-614
references: 16968208
references: Int J Epidemiol. 2011 Jun;40(3):780-5
references: 21454324
references: Science. 1975 Feb 7;187(4175):398-404
references: 17835295
references: Psychol Bull. 1952 Sep;49(5):499-520
references: 12993927
references: PLoS One. 2011;6(11):e26828
references: 22073203
references: Arch Gen Psychiatry. 2006 Jun;63(6):604-8
references: 16754833
references: BMC Med Res Methodol. 2008 May 30;8:34
references: 18513392
references: Psychol Methods. 2005 Mar;10(1):21-39
references: 15810867
references: Am Psychol. 2011 Dec;66(9):916-7
references: 22121992
references: Emerg Themes Epidemiol. 2008 Jan 22;5:2
references: 18211676
references: Proc Natl Acad Sci U S A. 2010 Apr 6;107(14):6192-7
references: 20308553
references: Ann Intern Med. 1997 Oct 15;127(8 Pt 2):757-63
references: 9382394
references: Perspect Psychol Sci. 2011 May;6(3):274-90
references: 26168518
references: Cogn Sci. 2011 Mar;35(2):211-50
references: 21428999
references: Organ Behav Hum Decis Process. 2001 Mar;84(2):308-333
references: 11277674
references: Am Psychol. 2006 Oct;61(7):726-8
references: 17032082
references: Am J Public Health. 2008 Dec;98(12):2237-43
references: 18235070
references: J Exp Psychol Learn Mem Cogn. 2008 Jan;34(1):186-203
references: 18194062
references: Behav Brain Sci. 1998 Jun;21(3):353-63; discussion 363-97
references: 10097017
references: Psychol Rev. 2000 Oct;107(4):659-76
references: 11089402
references: Cogn Neurosci. 2012;3(2):89-97
references: 24168689
references: Am Psychol. 2010 May-Jun;65(4):279-89
references: 20455621
references: Science. 2009 Jan 9;323(5911):272-5
references: 19131632
references: BMJ. 1994 Dec 3;309(6967):1480-1
references: 7804052
references: Alcohol Alcohol. 2000 Mar-Apr;35(2):197-201
references: 10787397
references: Perspect Psychol Sci. 2012 May;7(3):221-37
references: 26168460
references: Nature. 2006 Mar 30;440(7084):676-9
references: 16572172
references: Psychometrika. 2011 Oct;76(4):564-83
references: 27519681
references: Psychol Methods. 2003 Sep;8(3):338-63
references: 14596495
references: Psychol Rev. 2009 Jan;116(1):187-206
references: 19159153
references: Neuropsychologia. 2011 Nov;49(13):3605-11
references: 21939677
references: J Exp Psychol. 1954 Jun;47(6):381-91
references: 13174710
references: J Exp Psychol Gen. 2008 Aug;137(3):403-21
references: 18729707
references: PLoS One. 2011;6(10):e24532
references: 21998630
references: Cognition. 1995 Aug;56(2):99-128
references: 7554794
references: Int Psychogeriatr. 2009 Aug;21(4):616-21
references: 19243653
references: Psychol Rev. 1993 Jan;100(1):143-8; discussion 149-53
references: 8426879