title: Quality control of epidemiological lectures online: scientific evaluation of peer review.
abstract: OBJECTIVE
  To examine the feasibility of using peer review for the quality control of online materials.
  
  
  METHODS
  We analyzed the inter-rater agreement on the quality of epidemiological lectures online, based on the Global Health Network Supercourse lecture library. We examined the agreement among reviewers by looking at kappa statistics and intraclass correlations. Seven expert reviewers examined and rated a random sample of 100 Supercourse lectures. Their reviews were compared with the reviews of the lay Supercourse reviewers.
  
  
  RESULTS
  Both expert and non-expert reviewers rated lectures very highly, with a mean overall score of 4 out of 5. Kappa (Kappa) statistic and intraclass correlations indicated that inter-rater agreement for experts and non-experts was surprisingly low (below 0.4).
  
  
  CONCLUSIONS
  To our knowledge, this was the first time that poor inter-rater agreement was demonstrated for the Internet lectures. Future research studies need to evaluate the alternatives to the peer review system, especially for online materials.
authors: Department of Epidemiology, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15261, USA. fyL1@pitt.edu
authors: Faina
authors: F
authors: Linkov
authors: Mita
authors: M
authors: Lovalekar
authors: Ronald
authors: R
authors: LaPorte
mesh_terms: D003194:Computer-Assisted Instruction / Q000592:standards / Q000706:statistics & numerical data*; D004502:Education, Medical, Continuing / Q000592:standards / Q000706:statistics & numerical data*; D004813:Epidemiology / Q000193:education*; D006801:Humans; D020407:Internet; D015588:Observer Variation; D010380:Peer Review / Q000379:methods*; D011786:Quality Control