Assessment of critical appraisal skills.
BACKGROUND
Studies have provided little evidence that critical appraisal skills improve with focused courses. However, outcome measures in these studies have been questionable. The goal of this study was to develop a feasible, reliable, and valid assessment of critical appraisal skills.


METHODS
Forty-four surgery residents read three articles and then responded to short answer questions and provided 7-point ratings regarding various methodological aspects of each article. Reliability and validity of the examination were assessed.


RESULTS
The mean score was 52.4% (SD 8.6%). Internal consistency of the 55-question examination was 0.77. Interrater reliability of clinician markers was 0.91. Mean score for residents with more intensive critical appraisal training was significantly higher than for those with little or no training (56.6% versus 49.3%, t(35) = 2.31, P = 0.02), suggesting construct validity.


CONCLUSIONS
This examination has promising psychometric properties, and may be useful in evaluating critical appraisal curricula.
MacRae
Helen M
HM
Department of Surgery, University of Toronto, 451 Mt. Sinai Hospital, 600 University Ave., Toronto, ON, Canada M5G 1X5. hmacrae@mtsinai.on.ca
Regehr
Glenn
G
Brenneman
Fred
F
McKenzie
Marg
M
McLeod
Robin S
RS
D002983:Clinical Competence; D007396:Internship and Residency; D010506:Periodicals as Topic; D011795:Surveys and Questionnaires