title: Multi-faceted Rasch measurement and bias patterns in EFL writing performance assessment.
abstract: This study applied multi-faceted Rasch measurement to examine rater bias in the assessment of essays written by college students learning English as a foreign language. Four raters who had received different academic training from four distinctive disciplines applied a six-category rating scale to analytically rate essays on an argumentative topic and on a descriptive topic. FACETS, a Rasch computer program, was utilized to pinpoint bias patterns by analyzing the rater-topic, rater-category, and topic-category interactions. Results showed: argumentative essays were rated more severely than were descriptive essays; the linguistics-major rater was the most lenient rater, while the literature-major rater was the severest one; and the category of language use received the severest ratings, whereas content was given the most lenient ratings. The severity hierarchies for raters, essay topics, and rating categories suggested that raters' academic training and their perceptions of the importance of categories were associated with their bias patterns. Implications for rater training are discussed.
authors.affiliation: Department of Children's English Education, National Taipei University of Education, Taiwan. the@tea.ntue.edu.tw
authors.forename: Tung-Hsien
authors.initials: TH
authors.lastname: He
authors.forename: Wen Johnny
authors.initials: WJ
authors.lastname: Gou
authors.forename: Ya-Chen
authors.initials: YC
authors.lastname: Chien
authors.forename: I-Shan Jenny
authors.initials: IS
authors.lastname: Chen
authors.forename: Shan-Mao
authors.initials: SM
authors.lastname: Chang
mesh_terms: D003431:Cross-Cultural Comparison; D005178:Faculty; D006801:Humans; D008091:Literature; D019303:Multilingualism; D015588:Observer Variation; D011594:Psychometrics / Q000706:statistics & numerical data*; D013334:Students / Q000523:psychology*; D013624:Taiwan; D014956:Writing