[Correcting the critical reading test of the national ranking examination].
BACKGROUND
In 2008 the national ranking examination (NRE) will include a test on critical reading of scientific articles. This decision has provoked controversy about whether reproducible correction is possible. The aim of our study was to assess the consistency of grading between this two-part test (critical analysis and summarizing, analyzed separately), and the more classic tests.


METHODS
Eight graders, all instructors at the Tours Medical School, corrected papers for each of the 3 tests. Papers for the critical reading test came from medical school final examinations and for the standard test from a practice examination. The instructors worked in pairs: each pair read 30 papers for each test, and both members separately graded each paper. The final grade was the mean of the two grades. The consistency of grading between the 4 pairs was estimated by Kendall's coefficient of concordance.


RESULTS
Kendall's coefficients of concordance were estimated at 0.94 (95% CI=[0.86; 0.97]) for the standard test, at 0.92 (95% CI=[0.81;0.97]) for the critical analysis test, and at 0.75 (95% CI=[0.62; 0.84]) for the summaries. Pairwise comparisons estimated the difference in concordance between the standard test and the summary at 0.18 (95% CI=[0.08; 0.32]) and that between the standard and critical reading test at 0.01 (95% CI= [-0.07; 0.12]). Finally the difference in concordance between the 2 new tests - summary and critical reading - was estimated at -0.17 (95% CI=[-0.32; -0.04]).


CONCLUSIONS
The focus should be on the difficulty of reproducible correction of the summaries, to set up methods for appropriate correction and adequate grading. The elaboration of detailed scoring templates, including numerous items and specifying in which part of the summary each item must be placed, should help to improve the reproducibility of this test's correction.
Maruani
Annabel
A
Faculté de Médecine, Université François Rabelais, Tours.
Giraudeau
Bruno
B
Alison
Daniel
D
Bertrand
Philippe
P
Bourlier
Pascal
P
Fauchier
Laurent
L
Valat
Jean-Pierre
JP
Lorette
Gérard
G
D004504:Education, Medical, Undergraduate; D004521:Educational Measurement; D019317:Evidence-Based Medicine; D005602:France; D006801:Humans