title: Developing and evaluating criteria to help reviewers of biomedical informatics manuscripts.
abstract: Peer-reviewed publication of scientific research results represents the most important means of their communication. The authors have annually reviewed a large heterogeneous set of papers to produce the International Medical Informatics Association (IMIA) Yearbook of Medical Informatics. To support an objective and high-quality review process, the authors attempted to provide reviewers with a set of refined quality criteria, comprised of 80 general criteria and an additional 60 criteria for specific types of manuscripts. Authors conducted a randomized controlled trial, with 18 reviewers, to evaluate application of the refined criteria on review outcomes. Whereas the trial found that reviewers applying the criteria graded papers more strictly (lower overall scores), and that junior reviewers appreciated the availability of the criteria, there was no overall change in the interrater variability in reviewing the manuscripts. The authors describe their experience as a "case report" and provide a reference to the refined quality review criteria without claiming that the criteria represent a validated instrument for quantitative quality measurement.
authors.affiliation: Research Group Assessment of Health Information Systems, University for Health Informatics and Technology Tyrol (UMIT), Innrain 98, 6020 Innsbruck, Austria. elske.ammenwerth@umit.at
authors.forename: Elske
authors.initials: E
authors.lastname: Ammenwerth
authors.forename: Astrid C
authors.initials: AC
authors.lastname: Wolff
authors.forename: Petra
authors.initials: P
authors.lastname: Knaup
authors.forename: Hanno
authors.initials: H
authors.lastname: Ulmer
authors.forename: Stefan
authors.initials: S
authors.lastname: Skonetzki
authors.forename: Jan H
authors.initials: JH
authors.lastname: van Bemmel
authors.forename: Alexa T
authors.initials: AT
authors.lastname: McCray
authors.forename: Reinhold
authors.initials: R
authors.lastname: Haux
authors.forename: Casimir
authors.initials: C
authors.lastname: Kulikowski
mesh_terms: D006801:Humans; D008490:Medical Informatics / Q000592:standards*; D017712:Peer Review, Research / Q000592:standards*