Reviewing the reviewers: comparison of review quality and reviewer characteristics at the American Journal of Roentgenology.
OBJECTIVE
The purpose of our study was to determine which manuscript reviewer characteristics are most strongly associated with reviewer performance as judged by editors of the American Journal of Roentgenology (AJR).


METHODS
At the AJR, manuscript reviews are rated by the journal editors on a subjective scale from 1 (lowest) to 4, on the basis of the value, thoroughness, and punctuality of the critique. We obtained all scores for AJR reviewers and determined the average score for each reviewer. We also sent a questionnaire to 989 reviewers requesting specific information regarding the age, sex, radiology subspecialty, number of years serving as a reviewer, academic rank, and practice type of the reviewer. The demographic profiles were correlated with the average quality score for each reviewer. Statistical analysis included correlation analysis and analysis of variance modeling. Reviewer quality scores were also correlated with the scoring of individual reviews and ultimate disposition of 196 manuscripts sent to the AJR during the same period.


RESULTS
Responses to the questionnaire were obtained from 821 reviewers (83.0%), for whom quality scores were available for 714 (87.0%). Correlation analysis shows that the quality score of reviewers strongly correlated with younger age (p = 0.001). A statistically significant correlation between quality score and practice type was seen (p = 0.008), with reviewers from academic institutions receiving higher scores. No significant correlation was found between quality score and sex (p = 0.72), years of reviewing (p = 0.26), academic rank (p = 0.10), or the ultimate disposition of the manuscript (p = 0.40). The quality score of the reviewers showed no variation by subspecialty (p = 0.99).


CONCLUSIONS
The highest-rated AJR reviewers tended to be young and from academic institutions. The quality of peer review did not correlate with the sex, academic rank, or subspecialty of the reviewer.
Kliewer
Mark A
MA
Department of Radiology, University of Wisconsin, 600 Highland Ave., Madison, WI 53792, USA.
Freed
Kelly S
KS
DeLong
David M
DM
Pickhardt
Perry J
PJ
Provenzale
James M
JM
D005260:Female; D006801:Humans; D008297:Male; D008374:Manuscripts, Medical as Topic; D008875:Middle Aged; D017712:Peer Review, Research; D010506:Periodicals as Topic / Q000592:standards*; D011643:Publishing / Q000592:standards*; D011871:Radiology; D011795:Surveys and Questionnaires