title: Publication bias in medical informatics evaluation research: is it an issue or not?
abstract: The phenomenon of publication bias has probably existed since results of scientific research are being published. Positive and/or statistically significant results seem more likely to be published than negative and/or insignificant results. However, it is unclear if there is a remarkable impact of publication bias in medical informatics evaluation literature and how aware researchers are of its effect. We conducted a small-scale study in order to find out what the ratio of papers describing positive results vs. negative results is, tried to find enough studies to a certain subject to carry out a meta-analysis and assess publication bias by statistical methods, and finally examined reviews and meta-analyses for their results and their quality. A random sample of 86 studies showed a remarkably high percentage of descriptions of positive results (69.8%). 19 (36.6%) of the analyzed 54 reviews and meta-analyses came to a positive conclusion with regard to the overall effect of the analyzed system, 32 (62.5%) were inconclusive, and only one review came to a negative conclusion. Quantitative assessment of publication bias for health informatics studies was found difficult due to the low number of comparable studies. Although there is no clear evidence for a great impact of publication bias in medical informatics evaluation literature, further research should carried out.
authors.affiliation: Institute for Health Information Systems, UMIT - University for Health Sciences, Medical Informatics and Technology, Hall in Tyrol, Austria.
authors.forename: Christof
authors.initials: C
authors.lastname: Machan
authors.forename: Elske
authors.initials: E
authors.lastname: Ammenwerth
authors.forename: Thomas
authors.initials: T
authors.lastname: Bodner
mesh_terms: D005069:Evaluation Studies as Topic; D008490:Medical Informatics; D015201:Meta-Analysis as Topic; D017594:Publication Bias