Model choice and sample size in item response theory analysis of aphasia tests.
OBJECTIVE
The purpose of this study was to identify the most appropriate item response theory (IRT) measurement model for aphasia tests requiring 2-choice responses and to determine whether small samples are adequate for estimating such models.


METHODS
Pyramids and Palm Trees (Howard & Patterson, 1992) test data that had been collected from individuals with aphasia were analyzed, and the resulting item and person estimates were used to develop simulated test data for 3 sample size conditions. The simulated data were analyzed using a standard 1-parameter logistic (1-PL) model and 3 models that accounted for the influence of guessing: augmented 1-PL and 2-PL models and a 3-PL model. The model estimates obtained from the simulated data were compared to their known true values.


RESULTS
With small and medium sample sizes, an augmented 1-PL model was the most accurate at recovering the known item and person parameters; however, no model performed well at any sample size. Follow-up simulations confirmed that the large influence of guessing and the extreme easiness of the items contributed substantially to the poor estimation of item difficulty and person ability.


CONCLUSIONS
Incorporating the assumption of guessing into IRT models improves parameter estimation accuracy, even for small samples. However, caution should be exercised in interpreting scores obtained from easy 2-choice tests, regardless of whether IRT modeling or percentage correct scoring is used.
VA Pittsburgh Healthcare System and University of Pittsburgh, PA, USA. william.hula@va.gov
William D
WD
Hula
Gerasimos
G
Fergadiotis
Nadine
N
Martin
D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D001037:Aphasia / Q000503:physiopathology* / Q000523:psychology*; D002755:Choice Behavior / Q000502:physiology; D004192:Discrimination, Psychological / Q000502:physiology; D006801:Humans; D016015:Logistic Models; D008875:Middle Aged; D008960:Models, Psychological; D011594:Psychometrics / Q000379:methods* / Q000706:statistics & numerical data; D013066:Speech-Language Pathology / Q000379:methods; D055815:Young Adult