title: [Correcting the critical reading test of the national ranking examination].
abstract: BACKGROUND
  In 2008 the national ranking examination (NRE) will include a test on critical reading of scientific articles. This decision has provoked controversy about whether reproducible correction is possible. The aim of our study was to assess the consistency of grading between this two-part test (critical analysis and summarizing, analyzed separately), and the more classic tests.
  
  
  METHODS
  Eight graders, all instructors at the Tours Medical School, corrected papers for each of the 3 tests. Papers for the critical reading test came from medical school final examinations and for the standard test from a practice examination. The instructors worked in pairs: each pair read 30 papers for each test, and both members separately graded each paper. The final grade was the mean of the two grades. The consistency of grading between the 4 pairs was estimated by Kendall's coefficient of concordance.
  
  
  RESULTS
  Kendall's coefficients of concordance were estimated at 0.94 (95% CI=[0.86; 0.97]) for the standard test, at 0.92 (95% CI=[0.81;0.97]) for the critical analysis test, and at 0.75 (95% CI=[0.62; 0.84]) for the summaries. Pairwise comparisons estimated the difference in concordance between the standard test and the summary at 0.18 (95% CI=[0.08; 0.32]) and that between the standard and critical reading test at 0.01 (95% CI= [-0.07; 0.12]). Finally the difference in concordance between the 2 new tests - summary and critical reading - was estimated at -0.17 (95% CI=[-0.32; -0.04]).
  
  
  CONCLUSIONS
  The focus should be on the difficulty of reproducible correction of the summaries, to set up methods for appropriate correction and adequate grading. The elaboration of detailed scoring templates, including numerous items and specifying in which part of the summary each item must be placed, should help to improve the reproducibility of this test's correction.
authors.affiliation: Faculté de Médecine, Université François Rabelais, Tours.
authors.forename: Annabel
authors.initials: A
authors.lastname: Maruani
authors.forename: Bruno
authors.initials: B
authors.lastname: Giraudeau
authors.forename: Daniel
authors.initials: D
authors.lastname: Alison
authors.forename: Philippe
authors.initials: P
authors.lastname: Bertrand
authors.forename: Pascal
authors.initials: P
authors.lastname: Bourlier
authors.forename: Laurent
authors.initials: L
authors.lastname: Fauchier
authors.forename: Jean-Pierre
authors.initials: JP
authors.lastname: Valat
authors.forename: Gérard
authors.initials: G
authors.lastname: Lorette
mesh_terms: D004504:Education, Medical, Undergraduate; D004521:Educational Measurement; D019317:Evidence-Based Medicine; D005602:France; D006801:Humans