title: Is Rasch model analysis applicable in small sample size pilot studies for assessing item characteristics? An example using PROMIS pain behavior item bank data.
abstract: OBJECTIVE
  Large samples are generally considered necessary for Rasch model to obtain robust item parameter estimates. Recently, small sample Rasch analysis was suggested as preliminary assessment of items' psychometric properties. This study is to evaluate the Rasch analysis results using small sample sizes.
  
  
  METHODS
  Ten PROMIS pain behavior items were used. Random samples of 30, 50, 100, and 250, and a targeted sample of 30 were drawn 10 times each from a total of 800 subjects. Rasch analysis was conducted for each of these samples and the full sample.
  
  
  RESULTS
  In the full sample, there were 104 cases of extreme scores, no null categories, two incorrectly ordered items, and four misfit items. For samples of 250, 100, 50, 30, and targeted 30, the average numbers of extreme scores were 42.2, 17.1, 9.6, 6.1, and 1.2; the average numbers of null categories were 1.0, 3.2, 8.7, 13.4, and 8.3; the average numbers of items with incorrectly ordered item parameters were 0.1, 0.8, 2.9, 4.7, and 3.7; and the average numbers of items with fit residuals exceeding ± 2.5 were 0.8, 0.3, 0.1, 0.2, and 0.3, respectively.
  
  
  CONCLUSIONS
  Rasch analysis based on small samples (≤ 50) identified a greater number of items with incorrectly ordered parameters than larger samples (≥ 100). However, fewer items were identified as misfitting. Results from small samples led to opposite conclusions from those based on larger samples. Rasch analysis based on small samples should be used for exploratory purposes with extreme caution.
authors.affiliation: Center for Health Outcomes Research, United BioSource Corporation, 7101 Wisconsin Ave., Suite 600, Bethesda, MD, 20814, USA, wen-hung.chen@unitedbiosource.com.
authors.forename: Wen-Hung
authors.initials: WH
authors.lastname: Chen
authors.forename: William
authors.initials: W
authors.lastname: Lenderking
authors.forename: Ying
authors.initials: Y
authors.lastname: Jin
authors.forename: Kathleen W
authors.initials: KW
authors.lastname: Wyrwich
authors.forename: Heather
authors.initials: H
authors.lastname: Gelhorn
authors.forename: Dennis A
authors.initials: DA
authors.lastname: Revicki
mesh_terms: D006306:Health Surveys / Q000379:methods* / Q000592:standards; D006801:Humans; D015233:Models, Statistical; D010146:Pain / Q000503:physiopathology / Q000523:psychology; D010147:Pain Measurement / Q000379:methods*; D010865:Pilot Projects; D011594:Psychometrics / Q000379:methods* / Q000592:standards; D011788:Quality of Life; D012016:Reference Values; D018401:Sample Size; D011795:Surveys and Questionnaires
references.citation: Qual Life Res. 2010 Jun;19(5):613-5
references.pmid: 20422295
references.citation: Med Care. 2004 Jan;42(1 Suppl):I7-16
references.pmid: 14707751
references.citation: Pain. 1995 May;61(2):277-284
references.pmid: 7659438
references.citation: J Appl Meas. 2002;3(1):85-106
references.pmid: 11997586
references.citation: Pain. 2009 Nov;146(1-2):158-69
references.pmid: 19683873
references.citation: Qual Life Res. 2012 Apr;21(3):377-80
references.pmid: 22311251