Peer review in a small and a big medical journal: case study of the Croatian Medical Journal and the Lancet.
OBJECTIVE
To compare reviewers recommendations and editorial decisions in The Lancet, a high-impact journal, and Croatian Medical Journal (CMJ), a small general medical journal.


METHODS
Case study of research manuscripts submitted to the CMJ (n=140 manuscripts; 308 review forms) and a sample of similar manuscripts submitted to The Lancet (n=141; 348 review forms) in 1999 and the first half of 2000. Reviewers recommendations and their influence on editorial decisions on manuscripts were analyzed by logistic regression. Agreement between reviewers was assessed by the kappa statistic.


RESULTS
Although reviewers scores were identical in the two journals (median=3 for both journals, range 0 to 5), Lancet reviewers more often recommended rejection than CMJ reviewers (44% vs 17%; chi-square=52.1, p=0.029), and agreed best on rejection (kappa=0.29 [95% CI=0.04 to 0.53] vs kappa=-0.04 [-0.45 to 0.36] for CMJ). Lancet editors were even stricter than their reviewers and accepted for publication only 53% [95% CI=37 to 68] of manuscripts graded acceptable by the reviewers, compared with 85% [73 to 91] for CMJ editors (chi-square=10.0, p=0.001). For nine questions about manuscript quality in the review form, multiple regression analysis showed significant association between editorial decision and reviewers scores for the suitability of research design (B=0.70, OR=2.01, 95% CI=1.40 to 2.89, p<0.001) and discussion of systematic/random error (B=0.32, OR=1.38 [1.03 to 1.85], p=0.031) for The Lancet, and scores for novelty of information (B=0.56, OR=1.75 [1.35 to 2.27], p<0.001) for the CMJ.


CONCLUSIONS
Reviewers of big journals, such as The Lancet, are stricter in their recommendations reviewers of a smaller journal, such as the CMJ. The Lancet editors rely on reviewers to identify methodologically superior studies, whereas CMJ editors look for the novelty of information in a manuscript.
Marusić
Ana
A
Croatian Medical Journal, Zagreb University School of Medicine, Salata 3b, 10000 Zagreb, Croatia. marusica@mef.hr
Lukić
Ivan Kresimir
IK
Marusić
Matko
M
McNamee
David
D
Sharp
David
D
Horton
Richard
R
D017523:Croatia; D018864:Cultural Diversity; D003657:Decision Making; D006801:Humans; D017712:Peer Review, Research / Q000592:standards*; D010506:Periodicals as Topic / Q000706:statistics & numerical data*; D011786:Quality Control; D006113:United Kingdom