Automated computer-assisted categorization of radiology reports.
OBJECTIVE
The objective of our study was to create and validate an automated computerized method for the categorization of narrative text radiograph reports.


METHODS
Using commercially available software with embedded Boolean logic, we created a text search algorithm to categorize reports of radiography examinations into "fracture,"normal," and "neither normal nor fracture." The algorithm was refined and optimized through repeated testing on 512 consecutive ankle radiography reports from a single clinical imaging center. The final algorithm was applied on a different set of 750 consecutive radiography reports of the spine and extremities produced at three different clinical imaging sites and interpreted by 44 different radiologists. Expert reviewers assessed the accuracy of the final classification. The chi-square test or Fisher's exact test was performed to determine the reproducibility of results across different clinical imaging sites.


RESULTS
The computerized classification was highly accurate for the classification of radiography reports into "normal" (specificity, 91.6%; sensitivity, 91.3%), "neither normal nor fracture"(sensitivity, 87.8%; specificity, 94.9%), and "fracture"(sensitivity, 94.1%; specificity, 98.1%) categories. This performance showed no significant difference across the three sites (p >0.05).


CONCLUSIONS
Computerized categorization of narrative-text radiography reports is highly sensitive and specific and can be used to classify reports from different imaging sites generated by different radiologists. This method can be an extremely powerful tool in future cost-effectiveness studies, health care policy studies, operations assessments, and quality control.
Thomas
Bijoy J
BJ
Division of Musculoskeletal Radiology, Department of Radiology, Massachusetts General Hospital, 32 Fruit St., YAW 6E, Boston, MA 02114, USA.
Ouellette
Hugue
H
Halpern
Elkan F
EF
Rosenthal
Daniel I
DI
D000465:Algorithms; D016009:Chi-Square Distribution; D003658:Decision Making, Computer-Assisted; D005188:False Negative Reactions; D005189:False Positive Reactions; D006801:Humans; D011873:Radiology Information Systems; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012984:Software