title: Peer Review Quality and Transparency of the Peer-Review Process in Open Access and Subscription Journals.
abstract: BACKGROUND
  Recent controversies highlighting substandard peer review in Open Access (OA) and traditional (subscription) journals have increased the need for authors, funders, publishers, and institutions to assure quality of peer-review in academic journals. I propose that transparency of the peer-review process may be seen as an indicator of the quality of peer-review, and develop and validate a tool enabling different stakeholders to assess transparency of the peer-review process.
  
  
  RESULTS
  Based on editorial guidelines and best practices, I developed a 14-item tool to rate transparency of the peer-review process on the basis of journals' websites. In Study 1, a random sample of 231 authors of papers in 92 subscription journals in different fields rated transparency of the journals that published their work. Authors' ratings of the transparency were positively associated with quality of the peer-review process but unrelated to journal's impact factors. In Study 2, 20 experts on OA publishing assessed the transparency of established (non-OA) journals, OA journals categorized as being published by potential predatory publishers, and journals from the Directory of Open Access Journals (DOAJ). Results show high reliability across items (α = .91) and sufficient reliability across raters. Ratings differentiated the three types of journals well. In Study 3, academic librarians rated a random sample of 140 DOAJ journals and another 54 journals that had received a hoax paper written by Bohannon to test peer-review quality. Journals with higher transparency ratings were less likely to accept the flawed paper and showed higher impact as measured by the h5 index from Google Scholar.
  
  
  CONCLUSIONS
  The tool to assess transparency of the peer-review process at academic journals shows promising reliability and validity. The transparency of the peer-review process can be seen as an indicator of peer-review quality allowing the tool to be used to predict academic quality in new journals.
authors.affiliation: Department of Methodology and Statistics, Tilburg School of Behavioral Sciences, Tilburg University, Tilburg, The Netherlands.
authors.forename: Jelte M
authors.initials: JM
authors.lastname: Wicherts
mesh_terms: D022126:Access to Information; D015706:Bibliometrics; D035843:Biomedical Research / Q000941:ethics*; D035081:Editorial Policies; D006801:Humans; D017712:Peer Review, Research / Q000941:ethics*; D010506:Periodicals as Topic / Q000941:ethics*; D011786:Quality Control
references.citation: BMC Med. 2012;10:124
references.pmid: 23088823
references.citation: PLoS One. 2014;9(9):e106474
references.pmid: 25184367
references.citation: JAMA. 1998 Jul 15;280(3):237-40
references.pmid: 9676667
references.citation: Front Comput Neurosci. 2012 Apr 03;6:20
references.pmid: 22536180
references.citation: JAMA. 1998 Jul 15;280(3):231-3
references.pmid: 9676665
references.citation: Nature. 2012 Sep 13;489(7415):179
references.pmid: 22972258
references.citation: JAMA. 2002 Jun 5;287(21):2790-3
references.pmid: 12038913
references.citation: JAMA. 1998 Jul 15;280(3):229-31
references.pmid: 9676664
references.citation: Science. 2012 Nov 23;338(6110):1018
references.pmid: 23180837
references.citation: Science. 2013 Oct 4;342(6154):60-5
references.pmid: 24092725
references.citation: Nature. 2013 Mar 28;495(7442):426-9
references.pmid: 23538808
references.citation: JAMA. 2002 Jun 5;287(21):2786-90
references.pmid: 12038912
references.citation: PLoS One. 2010;5(12):e14331
references.pmid: 21179459
references.citation: BMC Med Res Methodol. 2013;13:74
references.pmid: 23758823