title: Automated Diabetic Retinopathy Image Assessment Software: Diagnostic Accuracy and Cost-Effectiveness Compared with Human Graders.
abstract: With the increasing prevalence of diabetes, annual screening for diabetic retinopathy (DR) by expert human grading of retinal images is challenging. Automated DR image assessment systems (ARIAS) may provide clinically effective and cost-effective detection of retinopathy. We aimed to determine whether ARIAS can be safely introduced into DR screening pathways to replace human graders.
  
  
  
  Observational measurement comparison study of human graders following a national screening program for DR versus ARIAS.
  
  
  
  Retinal images from 20 258 consecutive patients attending routine annual diabetic eye screening between June 1, 2012, and November 4, 2013.
  
  
  
  Retinal images were manually graded following a standard national protocol for DR screening and were processed by 3 ARIAS: iGradingM, Retmarker, and EyeArt. Discrepancies between manual grades and ARIAS results were sent to a reading center for arbitration.
  
  
  
  Screening performance (sensitivity, false-positive rate) and diagnostic accuracy (95% confidence intervals of screening-performance measures) were determined. Economic analysis estimated the cost per appropriate screening outcome.
  
  
  
  Sensitivity point estimates (95% confidence intervals) of the ARIAS were as follows: EyeArt 94.7% (94.2%-95.2%) for any retinopathy, 93.8% (92.9%-94.6%) for referable retinopathy (human graded as either ungradable, maculopathy, preproliferative, or proliferative), 99.6% (97.0%-99.9%) for proliferative retinopathy; Retmarker 73.0% (72.0 %-74.0%) for any retinopathy, 85.0% (83.6%-86.2%) for referable retinopathy, 97.9% (94.9%-99.1%) for proliferative retinopathy. iGradingM classified all images as either having disease or being ungradable. EyeArt and Retmarker saved costs compared with manual grading both as a replacement for initial human grading and as a filter prior to primary human grading, although the latter approach was less cost-effective.
  
  
  
  Retmarker and EyeArt systems achieved acceptable sensitivity for referable retinopathy when compared with that of human graders and had sufficient specificity to make them cost-effective alternatives to manual grading alone. ARIAS have the potential to reduce costs in developed-world health care economies and to aid delivery of DR screening in developing or remote health care settings.
authors.affiliation: Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, United Kingdom. Electronic address: Adnan.tufail@moorfields.nhs.uk.
authors.forename: Adnan
authors.initials: A
authors.lastname: Tufail
authors.affiliation: Department of Social Policy, LSE Health, London School of Economics and Political Science, London, United Kingdom.
authors.forename: Caroline
authors.initials: C
authors.lastname: Rudisill
authors.affiliation: Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, United Kingdom.
authors.forename: Catherine
authors.initials: C
authors.lastname: Egan
authors.affiliation: Population Health Research Institute, St George's, University of London, Cranmer Terrace, London, United Kingdom.
authors.forename: Venediktos V
authors.initials: VV
authors.lastname: Kapetanakis
authors.affiliation: Department of Social Policy, LSE Health, London School of Economics and Political Science, London, United Kingdom.
authors.forename: Sebastian
authors.initials: S
authors.lastname: Salas-Vega
authors.affiliation: Population Health Research Institute, St George's, University of London, Cranmer Terrace, London, United Kingdom.
authors.forename: Christopher G
authors.initials: CG
authors.lastname: Owen
authors.affiliation: Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, United Kingdom; University of Washington, Department of Ophthalmology, Seattle, Washington.
authors.forename: Aaron
authors.initials: A
authors.lastname: Lee
authors.affiliation: Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, United Kingdom.
authors.forename: Vern
authors.initials: V
authors.lastname: Louw
authors.affiliation: Homerton University Hospital, Homerton Row, London, United Kingdom.
authors.forename: John
authors.initials: J
authors.lastname: Anderson
authors.affiliation: Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, United Kingdom.
authors.forename: Gerald
authors.initials: G
authors.lastname: Liew
authors.affiliation: Homerton University Hospital, Homerton Row, London, United Kingdom.
authors.forename: Louis
authors.initials: L
authors.lastname: Bolter
authors.affiliation: Doheny Eye Institute, Los Angeles, California.
authors.forename: Sowmya
authors.initials: S
authors.lastname: Srinivas
authors.affiliation: Doheny Eye Institute, Los Angeles, California.
authors.forename: Muneeswar
authors.initials: M
authors.lastname: Nittala
authors.affiliation: Doheny Eye Institute, Los Angeles, California.
authors.forename: SriniVas
authors.initials: S
authors.lastname: Sadda
authors.affiliation: Centre for Health Informatics and Multiprofessional Education, Institute of Health Informatics, University College London, London, United Kingdom.
authors.forename: Paul
authors.initials: P
authors.lastname: Taylor
authors.affiliation: Population Health Research Institute, St George's, University of London, Cranmer Terrace, London, United Kingdom.
authors.forename: Alicja R
authors.initials: AR
authors.lastname: Rudnicka
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D002648:Child; D003362:Cost-Benefit Analysis; D003663:Decision Trees; D003930:Diabetic Retinopathy / Q000175:diagnosis* / Q000191:economics*; D004470:Economics, Medical; D005188:False Negative Reactions; D005260:Female; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000379:methods; D008297:Male; D008403:Mass Screening / Q000379:methods; D008875:Middle Aged; D010808:Physical Examination / Q000379:methods; D011237:Predictive Value of Tests; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012984:Software