title: Rater effect comparability in local independence and rater bundle models.
abstract: A large body of literature exists describing how rater effects may be detected in rating data. In this study, we compared the flag and agreement rates for several rater effects based on calibration of a real data under two psychometric models-the Rasch rating scale model (RSM) and the Rasch testlet-based rater bundle model (RBM). The results show that the RBM provided more accurate diagnoses of rater severity and leniency than do the RSM which is based on the local independence assumption. However, the statistical indicators associated with rater centrality and inaccuracy remain consistent between these two models.
authors.affiliation: Pearson, 3974 Roberts Ridge NE, Iowa City, IA 52240, USA, ed.wolfe@pearson.com.
authors.forename: Edward W
authors.initials: EW
authors.lastname: Wolfe
authors.forename: Tian
authors.initials: T
authors.lastname: Song
mesh_terms: D015982:Bias; D003625:Data Collection / Q000706:statistics & numerical data; D003627:Data Interpretation, Statistical; D004521:Educational Measurement / Q000706:statistics & numerical data*; D006801:Humans; D007318:Inservice Training; D015233:Models, Statistical; D015588:Observer Variation; D011594:Psychometrics / Q000706:statistics & numerical data*; D015203:Reproducibility of Results