Assessment of image-based technology: impact of referral cutoff on accuracy and reliability of remote retinopathy of prematurity diagnosis.
Telemedicine has potential to improve the delivery, quality, and accessibility of ophthalmic care for infants with Retinopathy of Prematurity (ROP). Using a telemedicine screening strategy, three potential diagnostic cutoffs may be used to define disease that warrants ophthalmologic referral: presence of any ROP, presence of moderate ("type-2 prethreshold") ROP, or presence of severe ROP requiring treatment. This study examines the relationship between accuracy and reliability of diagnosis by three masked ophthalmologist graders at each of these diagnostic cutoffs. The sensitivity, specificity, inter-grader reliability, and intra-grader reliability showed significant variation depending on the diagnostic cutoff, with best results at cutoffs of type-2 prethreshold ROP or treatment-requiring ROP. Before the large-scale adoption of telemedicine for image-based screening of diseases such as ROP, standards defining clinically-relevant referral cutoffs must be established, and diagnostic accuracy and reliability at these cutoffs must be characterized.
Chiang
Michael F
MF
Department of Biomedical Informatics, Columbia University, New York, NY, USA.
Keenan
Jeremy D
JD
Du
Yunling E
YE
Schiff
William
W
Barile
Gaetano
G
Li
Joan
J
Hess
Ditte J
DJ
Johnson
Rose Anne
RA
Flynn
John T
JT
Starren
Justin
J
D001271:Atlases as Topic; D003663:Decision Trees; D005240:Feasibility Studies; D006801:Humans; D007231:Infant, Newborn; D008488:Medical Illustration; D015588:Observer Variation; D009887:Ophthalmoscopy; D010781:Photography; D012017:Referral and Consultation; D019114:Remote Consultation; D015203:Reproducibility of Results; D012178:Retinopathy of Prematurity / Q000145:classification / Q000175:diagnosis*; D012680:Sensitivity and Specificity