Supervised machine learning and active learning in classification of radiology reports.
OBJECTIVE
This paper presents an automated system for classifying the results of imaging examinations (CT, MRI, positron emission tomography) into reportable and non-reportable cancer cases. This system is part of an industrial-strength processing pipeline built to extract content from radiology reports for use in the Victorian Cancer Registry.


METHODS
In addition to traditional supervised learning methods such as conditional random fields and support vector machines, active learning (AL) approaches were investigated to optimize training production and further improve classification performance. The project involved two pilot sites in Victoria, Australia (Lake Imaging (Ballarat) and Peter MacCallum Cancer Centre (Melbourne)) and, in collaboration with the NSW Central Registry, one pilot site at Westmead Hospital (Sydney).


RESULTS
The reportability classifier performance achieved 98.25% sensitivity and 96.14% specificity on the cancer registry's held-out test set. Up to 92% of training data needed for supervised machine learning can be saved by AL.


CONCLUSIONS
AL is a promising method for optimizing the supervised training production used in classification of radiology reports. When an AL strategy is applied during the data selection process, the cost of manual classification can be reduced significantly.


CONCLUSIONS
The most important practical application of the reportability classifier is that it can dramatically reduce human effort in identifying relevant reports from the large imaging pool for further investigation of cancer. The classifier is built on a large real-world dataset and can achieve high performance in filtering relevant reports to support cancer registries.
School of Information Technologies, University of Sydney, Sydney, New South Wales, Australia.
Dung H M
DH
Nguyen
School of Information Technologies, University of Sydney, Sydney, New South Wales, Australia.
Jon D
JD
Patrick
D000465:Algorithms; D001185:Artificial Intelligence; D003952:Diagnostic Imaging / Q000145:classification*; D006801:Humans; D008279:Magnetic Resonance Imaging / Q000145:classification; D009369:Neoplasms / Q000175:diagnosis*; D049268:Positron-Emission Tomography / Q000145:classification; D011873:Radiology Information Systems; D012680:Sensitivity and Specificity; D060388:Support Vector Machine; D014057:Tomography, X-Ray Computed / Q000145:classification; D018875:Vocabulary, Controlled
Classification; Radiology Information Systems; active learning; machine learning