Dataset

In what follows, different datasets used in this study are described individually, followed by supplementary information about the demographic data, imaging protocols, acquisition settings, de-identification, and the labeling process. The utilized dataset consists of a training and a test set, where the training dataset is the COVID-CT-MD [16] we introduced previously and is acquired from one imaging center using similar scanning parameters. The test dataset, the so-called SPGC-COVID, is comprised of four different sets each with specific characteristics to evaluate the robustness and generalizability of the DL model from different aspects.

An overview of different datasets and imaging centers is visualized in Fig 1. Different components of the utilized dataset are as follows:

• Train Set: We used our in-house and publicly available dataset [16], referred to as the “COVID-CT-MD”, as the training dataset which contains CT scans of COVID-19, CAP, and normal cases acquired by the “SIEMENS, SOMATOM Scope” scanner using the standard radiation dose from Babak Imaging Center, Tehran, Iran. A subset of 55 COVID-19 and 25 CAP cases are analyzed by one radiologist (M.J.R.) to identify slices demonstrating infection. The labeled subset of the data contains 4, 993 slices demonstrating infection and 18, 416 slices without evidence of infection. 30% of the cases in this set are randomly selected as the validation set.
• The SPGC-COVID Test Set: This dataset, which is released through this manuscript, comprises the following four different subsets:
  • Test Set 1: Low and Ultra-Low dose CT scans of COVID-19 and normal cases acquired from the same imaging center as that of the train set. This dataset is a subset of our in-house dataset of Low-Dose CT scans [17] and is publicly available.
  • Test Set 2: CT scans of COVID-19, CAP, and normal cases acquired in a different imaging center (Tehran Heart Center, Iran) using the “SIEMENS SOMATOM Emotion 16” scanner and different scanning parameters. Some cases in this dataset have an additional history of cardiovascular disease/surgeries with specific CT imaging findings, which are not available in the train set.
  • Test Set 3: CT scans of COVID-19, CAP, and normal cases obtained by the same scanner and scanning protocol used to acquire the train set. Cases in this test set are not included in the COVID-CT-MD public dataset.
  • Test Set 4: A combination of new CT scans of all three categories (i.e., COVID-19, CAP, Normal) obtained from the same centers as those of Test set 1 and 2, using the same acquisition settings and scanners.

Additional statistical and demographic information about different train and test sets used in this study are provided in Table 1. In Table 1, Center 1 represents the Babak Imaging Center and Center 2 is the Tehran Heart Center. Both imaging centers are located in Tehran, Iran and use the Filtered Back Projection reconstruction method [18] to obtain the CT images. Sample CT slices from the first three test sets are shown in Fig 2. Various scanning protocols and settings have been used to obtain the train and test datasets used in this study. The important parameters that contribute the most to the image quality and characteristics are presented in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Number of cases, demographic data, and additional acquisition information for each dataset.

https://doi.org/10.1371/journal.pone.0282121.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Acquisition parameters used to obtain each dataset.

https://doi.org/10.1371/journal.pone.0282121.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Sample CT slices from the first three test sets.

In Test set 1, the noise level is high. In Test 2, some cases reveal cardiovascular-related complications. In Test 3, the image quality and contrast are higher compared to other test sets.

https://doi.org/10.1371/journal.pone.0282121.g002

Summary of the 2021 SPGC-COVID challenge

In the first phase of this SPGC-COVID challenge, participants had access to the same train and validation sets as those used in this study to develop and evaluate their models. In the second phase, they have been provided with the first three test sets and had two weeks to submit their final models. Finally, the best-performing models based on the first three test sets have been evaluated on the fourth test set to determine the overall performances. In what follows, the main components of the six best-performing models in the SPGC-COVID challenge [7–12] are briefly described. As stated previously, an external model [20] developed based on the same dataset, but not as part of the challenge, is also investigated and used for comparison in this study. This model is also summarized below.

• Ref. [7]: In this model, slice-level predictions are acquired from an EfficientNet-based classifier [21] and a weighted majority voting is proposed to obtain the final patient-level labels. To train this classifier, the authors first trained two separate binary classifiers to detect slices demonstrating infection from COVID-19 and CAP cases. Then, they fed these models with unlabelled cases to provide the training set for the main classifier. Additionally, they only considered the middle slices (e.g., 80 middle slices) in a volumetric CT scan during the training phase.
• Ref. [8]: This model aggregates the output of six classifiers developed based on the 3D ResNet101 model [22]. One model in this proposed framework is a three-way classifier trained over all of the cases while the other five models are binary classifiers independently trained over COVID-19 and CAP cases using different combinations of train and validation sets.
• Ref. [9]: This model presents a feature extraction-based approach in which a modified pre-trained ResNet50 model classifies each slice into the target classes and the penultimate fully connected layer is extracted as the feature map. Next, a max-pooling layer followed by two fully connected layers is used to generate patient-level prediction from slice-level feature maps. The output of this model is then aggregated with two BiLSTM patient-level classifiers, which are fed by the same slice-level feature maps to provide the final patient-level labels.
• Ref. [10]: The pre-trained 3D Resnet50 [23] is the backbone of this model. The authors first doubled the number of slices for each case using a 3D cubic interpolation method. Then, they extracted the lung area using a pixel-based segmentation approach, followed by classical image processing techniques such as pixel filling and border cleaning. Finally, a subset of slices is selected from each volumetric CT scan based on their lung area and an experimentally-set threshold, which are then resized into a (224, 224, 224) data, using a 3D cubic interpolation method, providing the patient-level input for training and evaluation purposes.
• Ref. [11]: This model utilizes a two-stage framework in which the first stage is responsible for performing a multi-task classification to classify 2D slices into one of the target groups and identify the location of the slice in the sequence of CT images at the same time. The model at the first stage uses an ensemble of four popular CNN-based classifiers (i.e., ResneXt50 [24], DenseNet161 [25], Inception-V3 [26], and Wide-Resnet [27]), followed by an aggregation mechanism that divides the whole volumetric CT scan into 20 groups of slices and calculates the percentage of infected slices related to COVID-19 and CAP classes in each group. The values obtained for all groups are then concatenated and fed into an XG-boost classifier [28] in the second stage to generate patient-level predictions.
• Ref. [12]: The model proposed in this work initiates with a slice-level EfficientNet-B1 classifier [21] aiming to classify slices and generate feature maps (intermediate layers) to be used in the subsequent sequence classifier. In the sequence classifier, several weak classifiers are trained and the outputs are aggregated using an adaptive weighting mechanism to obtain the final patient-level results. To further enhance the performance of the model and cope with the imbalanced training set, a combination of weak and strong data augmentations are applied to the training cases, forcing the model to produce similar labels for both types of augmented images. Furthermore, to improve the robustness of the model when being tested on varied datasets, a K-Means clustering method (K = 3) [29] is adopted to develop a single classifier for each cluster of the data and aggregate the results via a majority voting approach.

The following provides a brief description of the model, which is not proposed in the SPGC-COVID challenge, but is used for comparison in this study as similar datasets are used for the development and evaluation of the model:

• Ref. [20]: This model aims to introduce a robust training algorithm and classification framework, which is capable of being updated upon receiving new datasets to deal with the characteristic shifts in different test sets. First, it adopts a two-stage architecture similar to the COVID-FACT model proposed in reference [14] and trains the benchmark model in a self-supervised fashion [30], and then the majority voting is adopted to obtain patient-level labels. The backbone model used in this study is DenseNet169 [25] and strict slice preprocessing and sampling methods are applied to the training set. Such methods contain pixel-based approaches with some fixed thresholds used to extract lung areas and select the slices with the most visible lung area. Next, each test set is divided into four quarters, which are then used in an unsupervised updating process, in which quarters are passed to the model sequentially and confident predictions are selected to fine-tune the slice-level classifiers. A slice-level prediction is considered confident in this study if it achieves the probability of at least 0.9 in agreement with the patient-level label.

The key components of our proposed model and those used for comparison are summarized in Table 3. All models resized and normalized the input data to be compatible with the utilized architectures. In addition, most frameworks (except those using a 3D model as their backbone) adopted a multi-stage framework transferring information from the slice-level domain to the patient-level one, some of which also utilized an ensemble architecture to aggregate the extracted information.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Underlying key features of the proposed framework and seven models used for comparison in this study (i.e., top six models developed following the SPGC-COVID challenge, and one model developed outside of the scope of this challenge).

https://doi.org/10.1371/journal.pone.0282121.t003

Proposed Capsule Network-based framework

In this study, we have developed a two-stage framework similar to the model proposed in our previous study [14], referred to as the “COVID-FACT”, as our benchmark model to classify volumetric CT scans into three target classes of COVID-19, CAP, and normal. We then use the unlabeled data from the test sets to boost the performance and robustness of the framework on the unseen cases. The pipeline of the proposed framework is shown in Fig 4. It is worth mentioning that although Capsule Network is the building block of models proposed in this study and our previous study [14], these two frameworks target different challenges, and different experiments are performed in the associated studies.

In what follows, different components of the proposed framework are described:

• Preprocessing: Raw CT scans, typically, contain uninformative components and unwanted artifacts (e.g., metallic artifacts), which can negatively affect the performance of the DL model. In addition, image sizes may vary and pixel intensities may be in different ranges when the images are acquired by different scanners. As such, we first extracted the lung areas from the CT images to remove the insignificant and distracting components. In this regard, we used a well-trained U-net-based segmentation model [31], which is fine-tuned on COVID-19 cases to specify lung areas in the first step. We then down-sampled all images into the (256 × 256) size to reduce the memory allocation and complexity without significant loss of information. Furthermore, we normalized each 2D image into the [0, 1] interval.
• Stage 1: The first stage performs the infection identification task, which aims to find slices with evidence of infection (caused by CAP or COVID-19) for each patient. The identified slices will then be classified into one of the three target classes in the second stage. The input of Stage 1 is the normalized lung area as a 2D image and the output is the label indicating whether the input image demonstrates infection or not. The classification model used in this stage is based on the Capsule Networks (CapsNets) [32], which have shown a superior discriminative capability compared to their CNN-based counterparts, especially when they are trained over small datasets [33–36]. Each capsule layer consists of multiple capsules, which are groups of neurons represented by a vector. Capsule Network benefits from an iterative process, known as the “Routing by Agreement”, that aims to evaluate the agreement between the capsules in a lower layer on the existence of an object in the higher layer. Using the Routing by Agreement process, the model can recognize the relation between multiple instances in an image. Furthermore, CapsNets have lower time and space complexity compared to conventional CNNs [14]. Such advantages make CapsNet-based models ideal in the case of COVID-19 where small annotated datasets are available and disease manifestations show specific spatial distributions in the lung. The detailed structure of the classification model in the first stage is shown in Fig 3(a).
  For the first stage, we adopted the same architecture as the model proposed in [14]. More specifically, the model in this stage uses a stack of four convolution layers, one batch normalization layer, and one max pooling layer to generate initial feature maps. Next, the output of the last convolution layer is reshaped to form the first capsule layer, followed by three consecutive capsule layers, as shown in Fig 3(a). The last layer contains two capsules representing the two target classes (i.e., slices with and without the evidence of infection). The length of each capsule represents the probability of the corresponding class being present. Different from COVID-FACT, residual connections are added between the convolution layers to transfer low-level features to the deeper layers. This modification further assists the model in identifying informative features. Additionally, we have added a dropout layer before the capsule layers to overcome the overfitting problems during the training. The labeled subset of the training dataset has been used to train this stage over 100 epochs using the Adam optimizer with the learning rate of 1e − 4. To account for the imbalanced number of slices in each class, we have used a weighted loss function to increase the contribution of the minority group (i.e., slices demonstrating infection) to the final loss value and balance the influence of each class. The balanced loss function used to train Stage 1 is given by (1) where w₁ and w₂ represent the weights corresponding to the loss value calculated for negative and positive samples, respectively. Term loss₁ denotes the loss associated with negative samples, while loss₂ is the loss associated with positive samples. Term N₁ represents the number of negative samples, and N₂ is the number of positive samples.
• Stage 2: The second stage takes the candidate slices from the previous stage and classifies them into one of the COVID-19, CAP, or normal cases. More specifically, we have used the slices demonstrating infection recognized by the first stage for all of the cases in the train set (with or without slice-level labels) to train a three-way classification model. Stage 2 utilizes a CapsNet architecture similar to the one used in the first stage but with smaller dimensions and three capsules in the last layer to represent three target classes. The architecture of stage two is shown in Fig 3(b). Similar to the first stage, we used a weighted loss function to cope with the imbalanced number of samples in some categories. At this stage, the loss weights associated with normal and CAP classes are set to 5 and the weight for the COVID-19 class is set to 1. Note that as the normal cases are extremely rare at this stage, the weights are set differently compared to those calculated by Eq 1, to maintain the stability of the training process, while enforcing the model to pay more attention to the minority classes. We also used the binary cross-entropy loss function, which translates the three-way classification problem at hand into three binary classification tasks. In fact, the loss value is calculated separately for each binary label associated with a target class (i.e., COVID-19, CAP, normal). Finally, a majority voting mechanism is adopted to transfer slice-level predictions into patient-level ones and determine the final label. It is worth noting that an accurate model in the first stage detects only a few candidate slices from normal cases. We can then apply a thresholding mechanism on the output of the first stage to identify those cases with only a few identified infectious slices in the first stage and label them as normal. We have used a threshold of 3% to specify normal cases immediately after the first stage. More specifically, if less than 3% of the slices in a volumetric CT scan are classified as infectious, the corresponding CT scan is classified as a normal case. Based on [37], the minimum lung lesion involvement in patients with COVID-19-related CT findings is 4%. In addition, the minimum percentage of slices demonstrating infection in our training dataset is 7%. In the case that the model in stage 1, misclassifies more than 3% of slices for a normal case, there is still a chance to classify the slices as normal in the second stage.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Detailed architecture of the classification model.

a) The structure of the CapsNet binary classifier in stage 1. b) The structure of the three-way classifier in stage 2. + sign denotes the residual addition.

https://doi.org/10.1371/journal.pone.0282121.g003

Unsupervised enhancement.

Unseen CT scans acquired by different scanners and scanning protocols contain heterogeneous characteristics leading to lower performance of a pre-trained model. To increase the robustness, we take advantage of the extra unlabeled samples that are available via the various test cases and utilize this extra set of CT scans in an unsupervised fashion. In other words, inspired by the ideas from “Active Learning [38–40]”, where different data samples are extracted to train the model in different stages, and “Semi-Supervised Learning [41, 42]”, where a label is assigned to unlabelled cases based on a pre-defined metric, we developed an autonomous mechanism to extract and label a part of data in the test sets using a probabilistic selection criteria with reduced complexity. The selected sample and the assigned labels are then used to re-train and boost the initially trained model. More specifically, we selected those test cases for which the model generated the most confident results (i.e., high probability). Similarly, among the selected cases, those with high confidence in slice-level predictions are used. To define the confident results, the probability of a volumetric CT scan belonging to a specific target class is considered to be equal to the ratio of the slices belonging to that class over the total number of slices (all slices containing the lung lesion), which can be written as follows (2) where represents the input volumetric CT scan, C represents the number of target classes, and denotes the number of slices belonging to the target class . Then, we introduced a confidence threshold value and considered a prediction confident if the probability of the input CT scan belonging to any of the target classes is more than the pre-set threshold. In this study, we have used 80% as the confidence threshold. A similar approach is used to extract confident slices and their corresponding labels. In this case, the probability of a slice belonging to a target class is determined by the output of the CapsNet classifier in Stage 2, which is the length (L₂Norm) of capsules in the last layer. It is worth mentioning that for those normal cases, which are identified in the first stage using the described thresholding mechanism, we only select the slices which are misclassified as infectious with a high probability (e.g., more than the confidence threshold). Such slices will be labeled as normal in the enhancement phase. Following the aforementioned steps, we can obtain a set of slices and their corresponding labels to augment the training dataset aiming to make the model more aware of the new features available in the unseen datasets and achieving more robust feature maps. Therefore, for each test set, we obtained a set of confident slices and their associated labels which have been added to the train set to re-train the model of the second stage. It is worth noting that the first stage has been kept unchanged in this approach. Finally, after re-training the benchmark model based on the confident slices acquired from each test set, we have obtained several enhanced models (each related to one test set) and averaged the associated patient-level probability scores to achieve the final prediction. This aggregation mechanism depends on the target test set. More specifically, to apply the model on each test set, we take the average of the predictions obtained by the models enhanced over the other test sets. For instance, the model developed for the diagnosis of cases in Test set 1 takes the average of probability scores provided by the models enhanced on Test sets 2 and 3. The main reason for using such an aggregation mechanism is that the enhancement based on a specific test set will further boost the probability scores of confidently predicted slices while having limited influence on other cases in the same set. As such, incorporating the model enhanced on a test set will not bring any further improvement to the evaluation process of the same set. The results presented in Table 4 further support this discussion. It is worth noting that we used the first three test sets to enhance the benchmark model and kept the fourth test set aside for only evaluation purposes. As such, upon receiving new test datasets, we can aggregate the results of the enhanced models on the individual test sets (each representing a specific center or scanning protocol) to provide the classification results for the new cases. The unsupervised model enhancement described above along with the subsequent ensemble averaging make the entire framework a robust automated framework that can be easily improved and updated upon receiving new datasets from different imaging centers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. The ratio of correctly classified cases over total cases in the associated class obtained for the proposed model, the benchmark model, and the partially enhanced models.

https://doi.org/10.1371/journal.pone.0282121.t004

Comparison

We have compared our proposed framework with the top six models [7–12] developed in the SPGC-COVID challenge. The Methods section of this paper provides a detailed description of each model, along with an overview of the development and evaluation steps of the challenge. In addition to the models proposed in the challenge, we have further compared our proposed framework with another model, which utilizes the same train and test sets (excluding the 4th test set) to target the same classification task [20]. A brief description of this model is also provided in the Methods section.

Experimental results demonstrate that our proposed framework outperforms its counterparts proposed in the SPGC-COVID challenge. Furthermore, it benefits from a scalable enhancement approach that can be integrated into most of the state-of-the-art models to improve their performance when testing on a heterogeneous dataset.

With regard to the development and evaluation process of the SPGC-COVID challenge, it is worth mentioning that the proposed framework and those models whose results are used as a comparison in this study have been developed in an entirely similar fashion. More specifically, the developers had access to the same datasets and labels, even the initial train/validation split of the data was the same. Moreover, no specific restrictions were applied to the development process to prevent privileging a specific type of model. In other words, the proposed framework was not designed to provide a baseline or reference standard for comparison in the challenge. In addition, we would like to highlight that even the proposed benchmark model without the incorporation of the enhancement approach achieved a higher performance than other models, as shown in Table 6. Therefore, given that the same scenario has been in place for all the developers (including us), we believe that the results present a fair and reliable comparison. The performance of the investigated models is presented in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Results obtained by the models that utilized the same train and test sets.

https://doi.org/10.1371/journal.pone.0282121.t006

Table 6 illustrates the performance of seven automated models developed to tackle the same task as that of this study using the same train and test datasets. We have also compared the overall performance of our proposed framework with the aforementioned models using the statistical McNemar’s test [44] with the significance level of 0.05. We tested the hypothesis that the models have the same proportion of errors on the entire test sets. The corresponding p-values are reported in Table 6 and indicate that the hypothesis is rejected for almost all the models except the first one as the corresponding p-value is slightly more than 0.05. In other words, there is a significant difference in the proportion of errors between our proposed framework and six of the aforementioned models while such difference is not significant in the case of the model proposed in Ref. [7].

Based on the key components of the models provided in Table 3 and the results reported in Tables 6 and 7, we can conclude that using an advanced lung region extraction model such as the U-Net R231 COVIDweb can improve the performance. Moreover, pre-training and data augmentation are used in most of the models, although such techniques were not utilized in our proposed CapsNets-based framework demonstrating the capability of Capsules to be trained using small datasets with limited data augmentation or pre-training. Furthermore, most models have not taken any specific measures to tackle the heterogeneity in the test cases. It is worth mentioning that although the model proposed in Ref. [20] used an online unsupervised learning approach to target this issue, it was not adequately trained and designed, in our opinion. This could possibly have led to its low performance. In addition to the components outlined in Table 3, we would like to note that the best-performing model in the challenge (i.e., Ref. [7]) used a customized mechanism to focus on middle slices showing large visible lung areas that could possibly improve its capability to perform well on various cases. However, their approach is static and needs to be adopted depending on slice thicknesses to provide a dynamic slice selection.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. The ratio of correctly classified cases over total cases in each class obtained for the top six models that were developed based on the SPGC-COVID dataset following the Signal Processing Grand Challenge (SPGC) on COVID-19 diagnosis.

https://doi.org/10.1371/journal.pone.0282121.t007

Clinical significance of chest imaging and CT in screening of COVID-19

First of all, we know that the role of chest imaging, especially CT, has evolved during the pandemic following the accumulation of experience and scientific data. The performance of chest imaging has been debated since the early period of the pandemic, where early studies from China showed the superiority of CT over the RT-PCR test [48, 49]. This might have been attributed to variabilities in viral load depending on the disease stage and sampling error, as well as the low availability and high demand of the test in the early days of the pandemic [50]. In this regard, some studies have recommended parallel testing using CT and RT-PCT, especially when consecutive tests are required to confirm the infection for treatment planning [51–53]. Currently, the RT-PCR test is the most commonly used diagnostic tool for COVID-19 detection, and some scientific societies do not recommend the use of chest CT for COVID-19 screening. On the other hand, chest imaging, specifically CT scanning, has a crucial role in different healthcare environments and clinical scenarios [54]. Although it is not recommended for asymptomatic or mildly symptomatic patients in the absence of accompanying risk factors in an environment, which is abundant in resources, unless they are at risk for disease progression, chest CT is recommended for medical triage of patients with suspected COVID-19 who present with moderate to severe symptoms and a high pretest probability of the disease regardless of the RT-PCR test, or in resource-constrained environments where RT-PCR tests may not be readily available or test results might be delayed. Chest CT imaging is also recommended for patients with worsening respiratory status. As such, it is clear that despite the widespread use of the RT-PCR test for screening of COVID-19 infection, the role of chest CT is irreplaceable in providing a baseline for future comparison, revealing an alternative diagnosis, establishing manifestations of important comorbidities in patients with risk factors for disease progression, and influencing treatment strategy and the intensity of monitoring for clinical deterioration [48].

In addition, it should be noted that the RT-PCR test is accompanied by a high false negative rate. Based on the latest guidance provided by the World Health Organization (WHO) for critical preparedness, readiness, and response actions for COVID-19 [55], such errors most likely occur due to technical reasons such as new virus mutations, the inhibition of the PCR reaction, sampling or storage errors, and timing of the test (i.e., too early or too late examinations). It is also worth noting that although a consecutive RT-PCR test could reduce the error probability, several studies [56, 57] have shown that such false negative results are not limited to the first test, and multiple negative PCR results have been reported for many cases while the significant progression of the disease has been confirmed through their CT scans.

In summary, the rt-PCR test is currently considered as the most reliable tool for screening for COVID-19. As stated above, there are certain clinical scenarios, which can include a significant number of cases, where the role of CT in diagnosing COVID-19 infection is irreplaceable. These scenarios include: (i) Suspected false negative PCR test, i.e., moderately or severely symptomatic patients with a high pretest probability of the disease regardless of the rt-PCR test; (ii) Unavailability of PCR test, i.e., resource-constrained environments, where the rt-PCR test may not be available, and; (iii) Latency, i.e., the rapidity with which the information is provided with chest CT compared to an rt-PCR test makes it the preferred method of diagnosis in certain environments. In conclusion, although we believe that there is a critical role for chest CT in correctly identifying COVID-19 infection in addition to being a critical approach for the detection of complications and prognosis, there are concerns about the limited clinical significance of chest CT as a tool for diagnosis of COVID pneumonia.

Radiation dose reduction in CT imaging

As the next part of our discussion, we would like to address the concerns raised around the potential risks of radiation exposure caused by CT imaging and its side-effects on the patients’ bodies. In particular, we would like to highlight the recent recommendations in the utilization of low-dose and ultra-low-dose CT imaging protocols, which ensure that minimized radiation is imposed on the body while the images are still of high quality and could reveal specific radiologic findings. Several studies have introduced specific technical settings that result in a significant reduction in the associated CT dose index (CTDIvol). As an example of such studies, Reference [58] reported adequate assessment of pulmonary opacities related to COVID-19 pneumonia at 100 kV with tin filter and iterative reconstruction technique with a CTDIvol of 0.4 mGy versus the standard-dose protocol with the CTDIvol of 3.4 mGy. Another study [58] applied 100 kV with tin filter and 0.6 second exposure time using a high pitch and fast gantry rotation time to acquire chest CT images at 0.6 mGy CTDIvol, which were comparable to standard-dose chest CT at 6.4 mGy. Furthermore, we would like to highlight that the low-dose scanning protocol used to acquire the CT scans in test sets 1 and 4 of this study has resulted in a significant reduction in the average radiation dose by reducing the reference mAs value from 50 to 15 − 20. More specifically, the radiation dose was reduced from the estimated value of 7mSv in standard-dose scans to 1 − 1.5mSv in LDCT scans, and 0.3mSv in the ULDCT ones, which is as low as that of a single chest radiograph. The results obtained for the aforementioned test sets demonstrate the effectiveness of such scans in providing specific CT findings that are efficiently captured by the proposed framework in an automated fashion. In conclusion, the concerns around cumulative radiation exposure on patients’ bodies could be mitigated to some extent, especially during a pandemic when a larger population is in need of being scanned.