The promise and challenge of including multimedia items in medical licensure examinations: some insights from an empirical trial.
BACKGROUND
The Comprehensive Osteopathic Medical Licensing Examination conducted a trial of multimedia items in the 2008-2009 Level 3 testing cycle to determine (1) if multimedia items were able to test additional elements of medical knowledge and skills and (2) how to develop effective multimedia items.


METHODS
Forty-four content-matched multimedia and text multiple-choice items were randomly delivered to Level 3 candidates. Logistic regression and paired-samples t tests were used for pairwise and group-level comparisons, respectively.


RESULTS
Nine pairs showed significant differences in either difficulty or/and discrimination. Content analysis found that, if text narrations were less direct, multimedia materials could make items easier. When textbook terminologies were replaced by multimedia presentations, multimedia items could become more difficult. Moreover, a multimedia item was found not uniformly difficult for candidates at different ability levels, possibly because multimedia and text items tested different elements of a same concept.


CONCLUSIONS
Multimedia items may be capable of measuring some constructs different from what text items can measure. Effective multimedia items with reasonable psychometric properties can be intentionally developed.
Shen
Linjun
L
National Board of Osteopathic Medical Examiners, Chicago, IL 60631, USA. lshen@nbome.org
Li
Feiming
F
Wattleworth
Roberta
R
Filipetto
Frank
F
D000328:Adult; D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006801:Humans; D008007:Licensure, Medical; D016015:Logistic Models; D008297:Male; D019212:Multimedia; D010021:Osteopathic Medicine / Q000193:education*; D014481:United States