title: Dealing with inter-expert variability in retinopathy of prematurity: A machine learning approach.
abstract: OBJECTIVE
  Understanding the causes of disagreement among experts in clinical decision making has been a challenge for decades. In particular, a high amount of variability exists in diagnosis of retinopathy of prematurity (ROP), which is a disease affecting low birth weight infants and a major cause of childhood blindness. A possible cause of variability, that has been mostly neglected in the literature, is related to discrepancies in the sets of important features considered by different experts. In this paper we propose a methodology which makes use of machine learning techniques to understand the underlying causes of inter-expert variability.
  
  
  METHODS
  The experiments are carried out on a dataset consisting of 34 retinal images, each with diagnoses provided by 22 independent experts. Feature selection techniques are applied to discover the most important features considered by a given expert. Those features selected by each expert are then compared to the features selected by other experts by applying similarity measures. Finally, an automated diagnosis system is built in order to check if this approach can be helpful in solving the problem of understanding high inter-rater variability.
  
  
  RESULTS
  The experimental results reveal that some features are mostly selected by the feature selection methods regardless the considered expert. Moreover, for pairs of experts with high percentage agreement among them, the feature selection algorithms also select similar features. By using the relevant selected features, the classification performance of the automatic system was improved or maintained.
  
  
  CONCLUSIONS
  The proposed methodology provides a handy framework to identify important features for experts and check whether the selected features reflect the pairwise agreements/disagreements. These findings may lead to improved diagnostic accuracy and standardization among clinicians, and pave the way for the application of this methodology to other problems which present inter-expert variability.
authors.affiliation: Department of Computer Science, Universidade da Coruña, A Coruña, Spain. Electronic address: vbolon@udc.es.
authors.forename: V
authors.initials: V
authors.lastname: Bolón-Canedo
authors.affiliation: Cognitive Systems Laboratory, Northeastern University, Boston, MA, USA.
authors.forename: E
authors.initials: E
authors.lastname: Ataer-Cansizoglu
authors.affiliation: Cognitive Systems Laboratory, Northeastern University, Boston, MA, USA.
authors.forename: D
authors.initials: D
authors.lastname: Erdogmus
authors.affiliation: Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, USA.
authors.forename: J
authors.initials: J
authors.lastname: Kalpathy-Cramer
authors.affiliation: Department of Computer Science, Universidade da Coruña, A Coruña, Spain.
authors.forename: O
authors.initials: O
authors.lastname: Fontenla-Romero
authors.affiliation: Department of Computer Science, Universidade da Coruña, A Coruña, Spain.
authors.forename: A
authors.initials: A
authors.lastname: Alonso-Betanzos
authors.affiliation: Departments of Ophthalmology & Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA.
authors.forename: M F
authors.initials: MF
authors.lastname: Chiang
mesh_terms: D006801:Humans; D007223:Infant; D000069550:Machine Learning; D012178:Retinopathy of Prematurity / Q000473:pathology*
keywords: Classification; Clinical decision-making; Feature selection; Inter-expert variability; Machine learning; Retinopathy of prematurity