A validated search assessment tool: assessing practice-based learning and improvement in a residency program.
OBJECTIVE
The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center.


METHODS
Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies.


RESULTS
Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts.


CONCLUSIONS
The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.
University of Michigan, Ann Arbor, MI, USA. preet@umich.edu
Gurpreet K
GK
Rana
Doreen R
DR
Bradley
Stanley J
SJ
Hamstra
Paula T
PT
Ross
Robert E
RE
Schumacher
John G
JG
Frohna
Hilary M
HM
Haftel
Monica L
ML
Lypson
D019317:Evidence-Based Medicine; D006801:Humans; D016247:Information Storage and Retrieval / Q000379:methods*; D007396:Internship and Residency; D016239:MEDLINE; D010372:Pediatrics / Q000193:education; D015203:Reproducibility of Results; D057386:Search Engine
JAMA. 2006 Sep 6;296(9):1116-27
16954491
Intensive Care Med. 2003 Dec;29(12):2119-2127
12955188
J Med Libr Assoc. 2005 Jan;93(1):53-60
15685275
Health Info Libr J. 2007 Dec;24 Suppl 1:18-37
18005292
Jt Comm J Qual Patient Saf. 2008 Aug;34(8):453-9
18714746
J Med Libr Assoc. 2002 Apr;90(2):194-201
11999177