Implementation of a journal peer reviewer stratification system based on quality and reliability.
OBJECTIVE
Before starting this study, Annals of Emergency Medicine had a large and unwieldy reviewer pool that demonstrated substantial variability in quality and reliability. We hypothesize that a tiered reviewer stratification system might enable our journal editors to target the bulk of their review invitations to our better reviewers and thus improve our efficiency.


METHODS
In 2003, we instituted a 3-tiered hierarchic classification for our reviewers and stratified them within these categories according to predefined criteria for reviewer quality and reliability. Our approximately 50 editors then targeted the bulk of their review invitations to the top performance tier.


RESULTS
Comparing 2009 data with 2002 (the year before the system), we found fewer late reviews (13% versus 32%) and fewer reviewers not used in a given year (28% versus 59%). More top-tier reviewer invitations led to an on-time review (48% versus 37%) in 2009 compared with 2002. Editors have found the system to be simple and easy to use. No serious problems have been identified.


CONCLUSIONS
Implementation of a tiered system stratifying journal peer reviewers by quality and reliability was readily accomplished by Annals and has appeared to improve the efficiency of our peer review.
Department of Emergency Medicine, Loma Linda University, CA 92354, USA. steve@viridissimo.com
Steven M
SM
Green
Michael L
ML
Callaham
D035081:Editorial Policies; D004635:Emergency Medicine / Q000592:standards; D015588:Observer Variation; D010380:Peer Review / Q000379:methods / Q000592:standards*; D010506:Periodicals as Topic / Q000592:standards; D011446:Prospective Studies; D015203:Reproducibility of Results