Fig 1.
Flowchart showing supervised approach to radiology report classification.
Fig 2.
Flowchart demonstrating unsupervised approach to radiology report classification.
Fig 3.
Performance assessment of non-neural classifiers on internal and external testing sets.
A-E. ROC curves displaying performance metrics on an expert-labelled internal testing set (n = 329 Normal, n = 601 Abnormal, n = 35 Uncertain). F-G. ROC curves demonstrating classifier performance on external MIMIC-CXR free-text reports (n = 272 Normal, n = 184 Abnormal, n = 9 Uncertain).
Fig 4.
Performance assessment of BiLSTM classifiers on internal and external testing sets.
A-D. ROC curves displaying performance metrics on an expert-labelled internal testing set (n = 329 Normal, n = 601 Abnormal, n = 35 Uncertain). E-H. ROC curves demonstrating classifier performance on the external MIMIC-CXR expert-labelled free-text reports (n = 272 Normal, n = 184 Abnormal, n = 9 Uncertain).
Table 1.
Performance comparison of supervised multi-class classifiers on internal and external testing sets.
Class-weighted values are reported.
Fig 5.
Performance assessment of Transformer-based classifiers on internal and external testing sets.
A-D. ROC curves displaying performance metrics on an expert-labelled internal testing set (n = 329 Normal, n = 601 Abnormal, n = 35 Uncertain). E-H. ROC curves demonstrating classifier performance on the external MIMIC-CXR expert-labelled free-text reports (n = 272 Normal, n = 184 Abnormal, n = 9 Uncertain).
Fig 6.
Unsupervised report classification using fastText, ivis, and Gaussian Mixture Model clustering.
A. Two-dimensional ivis representation of 50-dimensional fastText embeddings of n = 495,179 unlabelled radiological reports from NHS GGC. Colour gradient reflects posterior probability of Normal and Abnormal report cluster. B. Scatterplot of predicted ivis embeddings for n = 3,715 expert-labelled reports in the internal testing set. Blue and red points represent manually-labelled Normal and Abnormal reports respectively. Colour gradient reflects contours of posterior probability distributions obtained from GMM model trained on two-dimensional ivis representations of n = 495,179 unlabelled radiological reports. C. Scatterplot of predicted ivis embeddings for n = 456 expert-labelled reports in the MIMIC-CXR testing set. Blue and red points represent manually-labelled Normal and Abnormal reports respectively. Colour gradient reflects contours of posterior probability distributions obtained from GMM model trained on two-dimensional ivis representations of n = 495,179 unlabelled radiological reports. D. ROC curves of unsupervised GMM classifier applied to 50-dimensional fastText embeddings of internal (n = 3,715) and external (n = 456) manually-labelled reports. E-F. ROC curves of unsupervised GMM classifier applied to two- and ten-dimensional ivis embeddings of manually labelled internal (n = 3,715) and external (n = 456) reports.
Table 2.
Performance comparison of unsupervised classifiers on internal and external radiological reports.
Average performance values are reported.