title: Peer review for improving the quality of grant applications.
abstract: BACKGROUND
  Grant giving relies heavily on peer review for the assessment of the quality of proposals but the evidence of effects of these procedures is scarce.
  
  
  OBJECTIVE
  To estimate the effect of grant giving peer review processes on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research.
  
  
  METHODS
  Electronic database searches and citation searches; researchers in the field were contacted.
  
  
  METHODS
  Prospective or retrospective comparative studies with two or more comparison groups assessing different interventions or one intervention against doing nothing. Interventions may regard different ways of screening, assigning or masking submissions, different ways of eliciting opinions or different decision making procedures. Only original research proposals and quality outcome measures were considered.
  
  
  METHODS
  Studies were read, classified and described according to their design and study question. No quantitative analysis was performed.
  
  
  RESULTS
  Ten studies were included. Two studies assessed the effect of different ways of screening submissions, one study compared open versus blinded peer review and three studies assessed the effect of different decision making procedures. Four studies considered agreement of the results of peer review processes as the outcome measure. Screening procedures appear to have little effect on the result of the peer review process. Open peer reviewers behave differently from blinded ones. Studies on decision-making procedures gave conflicting results. Agreement among reviewers and between different ways of assigning proposals or eliciting opinions was usually high.
  
  
  CONCLUSIONS
  There is little empirical evidence on the effects of grant giving peer review. No studies assessing the impact of peer review on the quality of funded research are presently available. Experimental studies assessing the effects of grant giving peer review on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research are urgently needed. Practices aimed to control and evaluate the potentially negative effects of peer review should be implemented meanwhile.
authors: Servizo Sovrazonale di Epidemiologia, ASL 20, Via Venezia 6, Alessandria, Piemonte, Italy, 15100. Vittorio.DeMicheli@regione.piemonte.it
authors: V
authors: V
authors: Demicheli
authors: C
authors: C
authors: Di Pietrantonj
mesh_terms: D005381:Financing, Organized; D017712:Peer Review, Research / Q000592:standards*; D011786:Quality Control