Quality control of epidemiological lectures online: scientific evaluation of peer review.
OBJECTIVE
To examine the feasibility of using peer review for the quality control of online materials.


METHODS
We analyzed the inter-rater agreement on the quality of epidemiological lectures online, based on the Global Health Network Supercourse lecture library. We examined the agreement among reviewers by looking at kappa statistics and intraclass correlations. Seven expert reviewers examined and rated a random sample of 100 Supercourse lectures. Their reviews were compared with the reviews of the lay Supercourse reviewers.


RESULTS
Both expert and non-expert reviewers rated lectures very highly, with a mean overall score of 4 out of 5. Kappa (Kappa) statistic and intraclass correlations indicated that inter-rater agreement for experts and non-experts was surprisingly low (below 0.4).


CONCLUSIONS
To our knowledge, this was the first time that poor inter-rater agreement was demonstrated for the Internet lectures. Future research studies need to evaluate the alternatives to the peer review system, especially for online materials.
Linkov
Faina
F
Department of Epidemiology, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15261, USA. fyL1@pitt.edu
Lovalekar
Mita
M
LaPorte
Ronald
R
D003194:Computer-Assisted Instruction / Q000592:standards / Q000706:statistics & numerical data*; D004502:Education, Medical, Continuing / Q000592:standards / Q000706:statistics & numerical data*; D004813:Epidemiology / Q000193:education*; D006801:Humans; D020407:Internet; D015588:Observer Variation; D010380:Peer Review / Q000379:methods*; D011786:Quality Control