title: THE VALIDITY OF USING ROC SOFTWARE FOR ANALYSING VISUAL GRADING CHARACTERISTICS DATA: AN INVESTIGATION BASED ON THE NOVEL SOFTWARE VGC ANALYZER.
abstract: The purpose of the present work was to investigate the validity of using single-reader-adapted receiver operating characteristics (ROC) software for analysis of visual grading characteristics (VGC) data. VGC data from four published VGC studies on optimisation of X-ray examinations, previously analysed using ROCFIT, were reanalysed using a recently developed software dedicated to VGC analysis (VGC Analyzer), and the outcomes [the mean and 95 % confidence interval (CI) of the area under the VGC curve (AUCVGC) and the p-value] were compared. The studies included both paired and non-paired data and were reanalysed both for the fixed-reader and the random-reader situations. The results showed good agreement between the softwares for the mean AUCVGC For non-paired data, wider CIs were obtained with VGC Analyzer than previously reported, whereas for paired data, the previously reported CIs were similar or even broader. Similar observations were made for the p-values. The results indicate that the use of single-reader-adapted ROC software such as ROCFIT for analysing non-paired VGC data may lead to an increased risk of committing Type I errors, especially in the random-reader situation. On the other hand, the use of ROC software for analysis of paired VGC data may lead to an increased risk of committing Type II errors, especially in the fixed-reader situation.
authors.affiliation: Department of Medical Physics and Biomedical Engineering, Sahlgrenska University Hospital, SE-413 45 Gothenburg, Sweden Department of Radiation Physics, Institute of Clinical Sciences, the Sahlgrenska Academy at University of Gothenburg, SE-413 45 Gothenburg, Sweden jonny.hansson@vgregion.se.
authors.forename: Jonny
authors.initials: J
authors.lastname: Hansson
authors.affiliation: Department of Medical Physics and Biomedical Engineering, Sahlgrenska University Hospital, SE-413 45 Gothenburg, Sweden Department of Radiation Physics, Institute of Clinical Sciences, the Sahlgrenska Academy at University of Gothenburg, SE-413 45 Gothenburg, Sweden.
authors.forename: Lars Gunnar
authors.initials: LG
authors.lastname: Månsson
authors.affiliation: Department of Medical Physics and Biomedical Engineering, Sahlgrenska University Hospital, SE-413 45 Gothenburg, Sweden Department of Radiation Physics, Institute of Clinical Sciences, the Sahlgrenska Academy at University of Gothenburg, SE-413 45 Gothenburg, Sweden.
authors.forename: Magnus
authors.initials: M
authors.lastname: Båth
mesh_terms: D000465:Algorithms; D003626:Data Display; D003627:Data Interpretation, Statistical; D057225:Data Mining / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D020407:Internet / Q000458:organization & administration; D012372:ROC Curve; D011873:Radiology Information Systems / Q000458:organization & administration; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012984:Software; D014584:User-Computer Interface