What is behind a summary-evaluation decision?
Research in psychology has reported that, among the variety of possibilities for assessment methodologies, summary evaluation offers a particularly adequate context for inferring text comprehension and topic understanding. However, grades obtained in this methodology are hard to quantify objectively. Therefore, we carried out an empirical study to analyze the decisions underlying human summary-grading behavior. The task consisted of expert evaluation of summaries produced in critically relevant contexts of summarization development, and the resulting data were modeled by means of Bayesian networks using an application called Elvira, which allows for graphically observing the predictive power (if any) of the resultant variables. Thus, in this article, we analyzed summary-evaluation decision making in a computational framework.
Zipitria
Iraide
I
Department of Social Psychology and Behavioral Science Methodology, University of the Basque Country, Donostia, Spain. iraide.zipitria@ehu.es
Larrañaga
Pedro
P
Armañanzas
Ruben
R
Arruarte
Ana
A
Elorriaga
Jon A
JA
D000465:Algorithms; D001499:Bayes Theorem; D032882:Comprehension; D003657:Decision Making; D004521:Educational Measurement / Q000379:methods; D006801:Humans; D007807:Language Tests; D008960:Models, Psychological; D011589:Psychology, Experimental; D013647:Task Performance and Analysis