title: Towards new measures of information retrieval evaluation.
abstract: All of the methods currently used to assess information retrieval (IR) systems have limitations in their ability to measure how well users are able to acquire information. We utilized a new approach to assessing information obtained, based on a short-answer test given to senior medical students. Students took the ten-question test and then searched one of two IR systems on the five questions for which they were least certain of their answer Our results showed that pre-searching scores on the test were low but that searching yielded a high proportion of answers with both systems. These methods are able to measure information obtained, and will be used in subsequent studies to assess differences among IR systems.
authors.affiliation: Oregon Health Sciences University, Portland.
authors.forename: W R
authors.initials: WR
authors.lastname: Hersh
authors.forename: D L
authors.initials: DL
authors.lastname: Elliot
authors.forename: D H
authors.initials: DH
authors.lastname: Hickam
authors.forename: S L
authors.initials: SL
authors.lastname: Wolf
authors.forename: A
authors.initials: A
authors.lastname: Molnar
authors.forename: C
authors.initials: C
authors.lastname: Leichtenstien
mesh_terms: D005069:Evaluation Studies as Topic; D006801:Humans; D016247:Information Storage and Retrieval / Q000706:statistics & numerical data; D007256:Information Systems / Q000706:statistics & numerical data; D009922:Oregon; D010865:Pilot Projects; D013337:Students, Medical