title: Assessing individual differences in categorical data.
abstract: In cognitive modeling, data are often categorical observations taken over participants and items. Usually subsets of these observations are pooled and analyzed by a cognitive model assuming the category counts come from a multinomial distribution with the same model parameters underlying all observations. It is well known that if there are individual differences in participants and/or items, a model analysis of the pooled data may be quite misleading, and in such cases it may be appropriate to augment the cognitive model with parametric random effects assumptions. On the other hand, if random effects are incorporated into a cognitive model that is not needed, the resulting model may be more flexible than the multinomial model that assumes no heterogeneity, and this may lead to overfitting. This article presents Monte Carlo statistical tests for directly detecting individual participant and/or item heterogeneity that depend only on the data structure itself. These tests are based on the fact that heterogeneity in participants and/or items results in overdispersion of certain category count statistics. It is argued that the methods developed in the article should be applied to any set of participant x item categorical data prior to cognitive model-based analyses.
authors.affiliation: University of California, Irvine, California 92697-5100, USA.
authors.forename: Jared B
authors.initials: JB
authors.lastname: Smith
authors.forename: William H
authors.initials: WH
authors.lastname: Batchelder
mesh_terms: D001288:Attention; D016009:Chi-Square Distribution; D003071:Cognition; D003198:Computer Simulation; D003625:Data Collection / Q000706:statistics & numerical data*; D003627:Data Interpretation, Statistical; D006801:Humans; D007206:Individuality; D011939:Mental Recall; D009010:Monte Carlo Method; D010153:Paired-Associate Learning; D011932:Reading; D021641:Recognition, Psychology; D012559:Schizophrenia / Q000175:diagnosis; D012565:Schizophrenic Psychology; D012691:Serial Learning; D014706:Verbal Learning
references.citation: Biometrics. 1973 Dec;29(4):637-48
references.pmid: 4785230
references.citation: Psychon Bull Rev. 2004 Aug;11(4):579-615
references.pmid: 15581115
references.citation: J Exp Psychol Learn Mem Cogn. 2006 Mar;32(2):316-32
references.pmid: 16569149
references.citation: J Exp Psychol Learn Mem Cogn. 2002 Mar;28(2):392-406
references.pmid: 11911396
references.citation: Psychon Bull Rev. 2005 Aug;12(4):605-21
references.pmid: 16447375
references.citation: Psychon Bull Rev. 2005 Aug;12(4):573-604
references.pmid: 16447374
references.citation: Percept Psychophys. 2001 Nov;63(8):1421-55
references.pmid: 11800466
references.citation: Psychol Rep. 1995 Dec;77(3 Pt 2):1379-90
references.pmid: 8643807
references.citation: Psychol Rev. 2001 Jan;108(1):33-56
references.pmid: 11212631
references.citation: Psychol Bull. 1956 Mar;53(2):134-40
references.pmid: 13297917
references.citation: Psychon Bull Rev. 2005 Jun;12(3):409-41
references.pmid: 16235626
references.citation: Stat Med. 1999 Feb 15;18(3):287-305
references.pmid: 10070675
references.citation: Psychol Rev. 1993 Jan;100(1):143-8; discussion 149-53
references.pmid: 8426879
references.citation: Psychon Bull Rev. 2000 Jun;7(2):185-207
references.pmid: 10909131
references.citation: J Vis. 2005 May 27;5(5):478-92
references.pmid: 16097878
references.citation: Psychol Assess. 2002 Jun;14(2):184-201
references.pmid: 12056081
references.citation: J Exp Psychol Learn Mem Cogn. 1995 May;21(3):531-47
references.pmid: 7602260
references.citation: Psychon Bull Rev. 1999 Mar;6(1):57-86
references.pmid: 12199315
references.citation: J Gerontol B Psychol Sci Soc Sci. 1997 Sep;52(5):P206-15
references.pmid: 9310089
references.citation: Psychol Rev. 2002 Oct;109(4):710-21
references.pmid: 12374325