https://orcid.org/0009-0000-9847-2695
Figures
Abstract
Accurate classification of pulmonary nodules is essential for early lung cancer diagnosis, yet the limited spatial information provided by a single computed tomography slice may hinder the characterization of complex nodule morphology. Although multi-view strategies can capture complementary anatomical information, conventional fusion methods often fail to account for lesion-specific features and differences in the contribution of individual views. To address these limitations, we propose a tri-view pulmonary nodule classification framework that combines Lesion-Guided Attention (LGA) with View-Weighted Fusion (VWF). The framework extracts complementary representations from axial, coronal, and sagittal slices. LGA introduces lesion masks as weak spatial priors to refine lesion-related feature responses while preserving contextual information, whereas VWF learns sample-dependent weights to adaptively aggregate the three view representations. The framework was evaluated on 869 nodules (448 benign and 421 malignant) derived from the public Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) dataset. It achieved an accuracy of 93.08%, an AUC of 97.06%, and an F1-score of 92.68%, representing the highest values for these three metrics among the models included in the baseline comparison. Ablation analysis further showed that the combined use of LGA and VWF provided the best overall balance among the evaluated configurations. In the displayed test cases, Gradient-weighted Class Activation Mapping visualizations showed that the learned responses were concentrated mainly within or adjacent to lesion-relevant regions. These findings support the potential value of combining lesion-aware feature refinement with adaptive multi-view fusion for pulmonary nodule classification within the evaluated LIDC-IDRI cohort.
Citation: Zhang L, Zhuo D, Wu X, He Y, Kang G (2026) Multi-view attention-based deep learning for benign–malignant classification of pulmonary nodules. PLoS One 21(9): e0357963. https://doi.org/10.1371/journal.pone.0357963
Editor: Yongjie Li, University of Electronic Science and Technology of China, CHINA
Received: March 14, 2026; Accepted: August 25, 2026; Published: September 15, 2026
Copyright: © 2026 Zhang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: This work was supported by the National Natural Science Foundation of China (Grant Nos. 62565010, 11464015, and 12064012 to GK; Grant No. 52268049 to DZ; https://www.nsfc.gov.cn/), the Natural Science Foundation of Hunan Province (Grant No. 2020JJ4496 to GK; https://kjt.hunan.gov.cn/), and the Science Research Foundation of the Education Department of Hunan Province (Grant No. 20A0585 to GK; https://jyt.hunan.gov.cn/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Background and motivation
Lung cancer remains one of the leading causes of cancer-related mortality worldwide. According to the 2020 global cancer estimates, approximately 2.2 million new cases and 1.8 million deaths occur annually [1]. Prognosis is strongly associated with the stage at diagnosis, and early detection can substantially improve patient survival [2]. Pulmonary nodules are common early radiological manifestations of lung cancer; therefore, accurate benign–malignant classification is important for screening, treatment planning, and follow-up management.
Computed tomography (CT), particularly low-dose CT (LDCT), has been widely used for lung cancer screening and has demonstrated benefits in early-stage detection and mortality reduction [3,4]. However, the interpretation of pulmonary nodules still depends heavily on radiologists’ experience and is affected by subjectivity, limited reproducibility, and increasing workload. Computer-aided diagnosis methods may assist radiologists by providing consistent assessments and visual evidence for pulmonary nodule evaluation [5].
Deep learning has achieved substantial progress in pulmonary nodule analysis. Convolutional neural networks (CNNs) can automatically learn texture, shape, and boundary features from CT images [5,6], while Transformer-based and CNN–Transformer hybrid architectures provide broader contextual modeling through self-attention [7,8]. Nevertheless, some existing methods rely primarily on two-dimensional slices, which may not adequately represent morphological variations across different spatial directions [6].
Multi-view strategies provide a practical means of incorporating complementary spatial information without the computational burden of full three-dimensional (3D) modeling [9,10]. However, simple concatenation or fixed-weight fusion cannot adapt to differences in the diagnostic information provided by individual views. In addition, attention mechanisms are often learned implicitly and may respond to irrelevant background regions rather than lesion-related features [11]. Although three-dimensional models can capture volumetric information, they generally require greater computational resources and larger training datasets [5,12]. Therefore, effectively combining lesion-oriented feature refinement with adaptive multi-view fusion remains an important problem in pulmonary nodule classification.
To address these limitations, we propose a tri-view pulmonary nodule classification framework based on axial, coronal, and sagittal CT slices. The use of tri-planar input alone is not considered the methodological novelty of this study because similar representations have been investigated previously [13]. Instead, the principal contribution lies in the coordinated integration of Lesion-Guided Attention (LGA) and View-Weighted Fusion (VWF). LGA uses lesion masks as weak spatial priors to refine lesion-related features while preserving contextual information, whereas VWF learns sample-dependent, Softmax-normalized weights to adaptively aggregate the three view representations. The framework is evaluated through comparisons with commonly used single-view backbones, module ablation experiments, and qualitative Grad-CAM visualization.
Related work and challenges
Convolutional neural network-based pulmonary nodule benign–malignant classification.
CNNs have been widely applied to pulmonary nodule classification because of their ability to learn local texture, shape, and edge features directly from CT images [5,6]. Representative studies, including the NoduleX framework, demonstrated that deep CNN features can provide effective malignancy prediction on the LIDC-IDRI dataset [14].
To incorporate spatial information beyond a single slice, several studies have introduced three-dimensional CNNs. Dou et al. developed multi-level contextual 3D CNNs for pulmonary nodule analysis [15], while Hussein et al. employed a 3D CNN-based multi-task framework for nodule risk stratification [16]. Multi-scale three-dimensional networks have also been investigated to improve nodule representation [17]. Although these methods capture volumetric characteristics, their computational and data requirements may limit their use in relatively small medical imaging datasets.
Multi-view and multi-slice modeling approaches.
Multi-view and multi-slice methods have been developed to represent the three-dimensional characteristics of pulmonary nodules using complementary two-dimensional images [9,10]. These methods extract information from multiple slices or anatomical directions and generally provide richer spatial representations than single-view approaches.
Chang et al. proposed a multiview residual selective kernel network using axial, coronal, and sagittal CT views together with handcrafted texture features [13]. This work represents a closely related tri-planar classification strategy. Unlike that approach, the present framework uses lesion masks as weak spatial priors for soft residual feature refinement and learns sample-dependent view contributions through Softmax-normalized weighting. Other collaborative or multi-task approaches have also demonstrated the value of combining complementary nodule representations [16,18]. However, adaptive integration of lesion-refined features across views remains insufficiently investigated.
Transformer-based approaches for medical image analysis.
Vision Transformers have been introduced into medical image analysis because self-attention can model long-range dependencies and global semantic relationships [19]. Vision Transformer (ViT) and Swin Transformer provide representative self-attention-based architectures for image recognition [20,21], and Transformer-based variants have increasingly been investigated in medical imaging applications [8,19]. However, pure Transformer models generally require substantial training data, and their performance may be less stable when labeled medical datasets are limited [19].
CNN–transformer hybrid architectures and attention mechanisms.
CNN–Transformer hybrid architectures combine the local feature extraction capability of CNNs with the contextual modeling capability of Transformers. Medical Vision Transformer (MedViT) integrates efficient convolutional modules and Transformer blocks and has demonstrated strong performance across several medical image classification tasks [22], providing the backbone architecture used in this study.
Attention mechanisms can further enhance responses to informative regions, while Gradient-weighted Class Activation Mapping (Grad-CAM) provides qualitative visualization of class-related activations [11,23]. Nevertheless, implicitly learned attention does not necessarily correspond to lesion regions [24]. This limitation motivates the introduction of explicit lesion guidance together with adaptive multi-view fusion in the proposed framework.
Materials and methods
Methodology
Overall model architecture.
This study develops a tri-view model for pulmonary nodule benign–malignant classification using MedViT as the encoder backbone, as illustrated in Fig 1. Representative axial, coronal, and sagittal slices are processed by independent encoding branches, refined using LGA, and adaptively aggregated using VWF before classification.
Representative axial, coronal, and sagittal slices are processed by independent MedViT branches. High-level features are refined by the Lesion-Guided Attention (LGA) module using lesion masks as weak spatial priors and then aggregated by the View-Weighted Fusion (VWF) module using sample-dependent weights. The fused representation is passed to the classifier for benign–malignant prediction.
Tri-view input encoding module.
To construct the tri-view input, a consensus mask is first generated from the LIDC-IDRI radiologist annotations using a 50% agreement criterion. A 50 × 50 × 50-voxel cube centered at the centroid of this consensus mask is then cropped. This size includes the nodule and limited perinodular context while restricting irrelevant background and computational cost. The slice with the largest lesion-mask area is selected from each axial, coronal, and sagittal plane and resized to 224 × 224; HU-based intensity normalization is described in the Dataset and preprocessing section. Let v∈{x,y,z} denote the axial, coronal, and sagittal views, respectively.
Each slice is processed by a MedViT branch with an identical architecture and output dimension but non-shared trainable parameters. In MedViT, Efficient Convolution Blocks (ECBs) primarily model local features, Local Transformer Blocks (LTBs) provide contextual feature modeling, and the Patch Momentum Changer (PMC) serves as a feature-regularization module [22]. Specifically, the Patch Embedding parameters, the convolutional and normalization parameters in the ECBs, and the attention, projection, and normalization parameters in the LTBs are maintained separately for the three branches. Let θx, θy, and θz denote the corresponding parameter sets. These branches are jointly optimized under the same classification objective, whereas the Patch Momentum Changer (PMC) is shared across views. At the l-th stage, the feature map of view v is expressed as:
where Cl denotes the channel dimension, whereas Hl and Wl denote the spatial dimensions of the feature map at stage l, respectively.
In the multi-view parallel encoding setting, feature distributions may vary across different views, which can negatively affect subsequent cross-view fusion. To mitigate inter-view distribution discrepancies, the Patch Momentum Changer (PMC) module from the original MedViT architecture is retained and applied after the first encoding stage [22]. The PMC module performs feature regularization by perturbing and reorganizing the statistical properties of mini-batch features, which was intended to smooth feature distributions before subsequent cross-view fusion [22]. In the implementation, PMC was applied only during training, with an activation probability of 0.5 and a mixing strength of 0.5. The same batch permutation and mixing coefficient were applied to the three view branches, whereas PMC was disabled during validation and testing. The transformation can be formulated as:
where denotes the feature representation after PMC processing.
Table 1 summarizes the input size, encoder configuration, parameter-sharing scheme, and output dimension of each view branch.
The tri-view encoding process is illustrated in Fig 2. The resulting view-specific features are subsequently refined by LGA before adaptive fusion.
Representative axial, coronal, and sagittal slices are processed by three encoder branches with identical architectures and non-shared trainable parameters. Each branch adopts MedViT as the encoder, whereas the Patch Momentum Changer (PMC) is shared across the branches after the first encoding stage. The resulting view-specific features are subsequently passed to the LGA module.
Lesion-Guided Attention module.
In pulmonary nodule benign–malignant classification, the nodule region typically occupies only a small portion of the CT image and is characterized by complex density variations and blurred boundaries. As a result, conventional convolutional or Transformer-based feature extraction processes may allocate excessive attention to irrelevant background regions, which can adversely affect classification performance and model interpretability [5]. To address this issue, a Lesion-Guided Attention (LGA) module is introduced to explicitly guide the spatial attention distribution by integrating lesion location priors with feature responses [24].
The LGA module takes as input the feature maps from the tri-view encoding module, where
denotes the axial, coronal, and sagittal views, respectively, along with the corresponding binary lesion masks Mv. The lesion masks used in all experiments were generated during preprocessing from the LIDC-IDRI radiologist annotations using a 50% consensus criterion and served as weak spatial priors for attention modulation [24,25]. No additional segmentation network was trained as part of the present classification pipeline.
The lesion masks are not used for hard cropping or hard masking. Each branch receives the same nodule-centered patch, and the aligned mask is introduced only after high-level feature extraction to generate attention weights. The residual pathway preserves the original representation while selectively enhancing lesion-related responses, thereby retaining limited perinodular context that may also be informative for classification [26]. This design is intended to reduce dependence on a single exact lesion boundary when radiologist delineations differ or nodule margins are indistinct [25].
Specifically, each binary lesion mask is replicated along the channel dimension and spatially aligned to the spatial resolution of the corresponding feature map before feature modulation. This operation yields an aligned guidance mask that is compatible with the feature map for element-wise multiplication. For notational simplicity, the mask symbol used in the following equations refers to this aligned guidance mask rather than to the original binary mask. The lesion-guided attention weight map is then computed as:
where and
denote learnable 1 × 1 convolution kernels, * represents convolution,
denotes element-wise multiplication,
is the Sigmoid function, and ReLU denotes the rectified linear unit activation. This operation generates a soft lesion-guided attention map from the masked feature responses. In the implementation, W1 projected the 256-channel input feature map to 64 hidden channels, whereas W2 projected the resulting representation to a single-channel attention map.
The guided feature enhancement is then performed as
where λ is a fixed scaling factor that controls the strength of lesion-guided spatial modulation and was set to 0.5 in all experiments. The identity term prevents complete suppression of the baseline feature response.
To control the influence of strong prior guidance and support stable feature integration, a residual enhancement pathway is incorporated into the LGA module before output fusion:
where α is a learnable residual scaling parameter initialized to 0.1 and jointly optimized with the remaining network parameters. Thus, λ controls the strength of spatial modulation, whereas α controls the contribution of the enhanced residual branch and is intended to limit excessive reliance on the lesion prior.
LGA adds mask resizing and spatial alignment, element-wise operations, and two 1 × 1 convolutions after backbone feature extraction. A dedicated runtime or floating-point-operation comparison with region-of-interest cropping was not performed; therefore, no computational superiority is claimed. Application to data without pre-existing lesion masks would require an upstream segmentation source.
The complete LGA processing flow is illustrated in Fig 3. Its output features are subsequently supplied to VWF for cross-view aggregation.
The lesion mask is expanded and spatially aligned with the corresponding feature map. Mask-guided responses are processed by two 1 × 1 convolutions with rectified linear unit activation and a Sigmoid function to generate the attention map. The enhanced features are combined with the original representation through a learnable residual scaling coefficient and subsequently passed to VWF.
Learnable View-Weighted Fusion module.
After LGA, the axial, coronal, and sagittal features contain complementary information but may contribute unequally to individual nodules. The VWF module therefore generates one score from each view descriptor and jointly normalizes the three scores using Softmax. The resulting weights are sample-dependent, non-negative, and sum to one, enabling end-to-end weighted aggregation without explicit cross-view attention interactions. The VWF processing flow is illustrated in Fig 4.
The LGA-enhanced feature maps from the axial, coronal, and sagittal views are globally average-pooled to obtain view-specific descriptors. A learnable view-scoring head produces three unnormalized logits, which are jointly normalized using Softmax to generate sample-dependent weights ,
, and
. The weights are non-negative and satisfy
. The fused representation is obtained through weighted aggregation of the three view features and is subsequently forwarded to the projection head and classifier for benign–malignant prediction.
Let v∈{x,y,z} denote the view index corresponding to the axial, coronal, and sagittal views. For notational simplicity, in this section denotes the LGA output feature map
defined above. VWF first applies global average pooling (GAP) to obtain a compact view descriptor:
A lightweight learnable view-scoring head is then used to produce an unnormalized view logit for each view:
where φ denotes a two-layer multilayer perceptron with dimensions 256 → 64 → 1 and a ReLU activation between the two linear layers. Separate scoring heads with the same architecture were used for the axial, coronal, and sagittal views. The three logits are jointly normalized by Softmax for each nodule:
The three coefficients are recalculated for each nodule and are jointly optimized with the encoders, LGA, and classifier. They represent model-internal relative view contributions rather than manually assigned clinical importance scores, and the mechanism is conceptually related to adaptive expert gating [27].
Finally, the fused feature representation is computed by weighted aggregation:
The resulting fused representation was subsequently passed through a 256-dimensional projection head, followed by a ReLU activation and a dropout layer with a rate of 0.3. A single-unit linear classifier then produced the malignancy logit.
Backbone building blocks (ECB and LTB).
To balance representational capacity and computational efficiency in medical imaging settings, we adopt the Efficient Convolution Block (ECB) and the Local Transformer Block (LTB) from the original MedViT framework as the fundamental feature extraction units [22]. It should be noted that we do not modify the internal architectures or computational procedures of ECB and LTB; detailed designs and implementation specifics can be found in the original MedViT paper [22]. This section provides further details on these building blocks to facilitate understanding of the overall model architecture.
The ECB combines depthwise separable convolutions with Multi-Head Convolutional Attention, enabling efficient modeling of local spatial patterns with a relatively low computational budget [22]. By leveraging multi-scale convolutional operations, ECB captures local contextual cues in nodule regions, while this attention mechanism models responses across channels and spatial locations, further enhancing local feature representations in an efficient manner [22].
The LTB introduces an Efficient Self-Attention mechanism to model long-range dependencies with controlled complexity [22]. Specifically, this mechanism projects the input feature map into Query, Key, and Value embeddings and applies multi-head self-attention to capture global contextual information. The resulting features are then refined through pointwise convolution and linear mappings to produce the output representation. In MedViT, the LTB complements the ECB by providing global context support for higher-level semantic representations [22].
In the proposed framework, ECB and LTB form the four-stage MedViT backbone, while LGA and VWF operate on the resulting high-level view features.
Loss function.
To optimize the proposed model for pulmonary nodule benign–malignant classification, we used the standard binary cross-entropy (BCE) objective. Let zi ∈ R denote the output logit for the i-th sample, pi = σ(zi) the corresponding predicted probability of malignancy, and yi∈{0,1} the ground-truth label, where 1 denotes malignant and 0 denotes benign. The BCE loss is expressed as:
where N denotes the number of samples in a mini-batch. In implementation, the raw logits were directly passed to PyTorch’s BCEWithLogitsLoss, which combines the Sigmoid operation and BCE calculation in a numerically stable form. No class weights were applied. Regularization was introduced through the decoupled weight decay of the AdamW optimizer rather than through an explicit penalty term added to the loss [28]. The weight decay coefficient was set to 0.05.
Experimental setup
Dataset and preprocessing.
Experiments were conducted on the publicly available Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) dataset, which contains 1,018 thoracic CT scans [25]. The dataset was obtained from The Cancer Imaging Archive (TCIA) (https://doi.org/10.7937/K9/TCIA.2015.LO9QL9SX).
Nodules identified by at least three radiologists were retained. The malignancy-labeling rule described by Chang et al. [13] was adopted. Each available radiologist assigned a malignancy score ranging from 1 to 5, and the median of the available ratings was used as the final score for each nodule. Nodules with a median score below 3 were labeled as benign, those with a median score above 3 were labeled as malignant, and those with a median score equal to 3 were regarded as indeterminate and excluded from the binary classification task. After excluding nodules for which valid consensus masks or complete tri-view inputs could not be generated, the final cohort comprised 869 nodules, including 448 benign and 421 malignant nodules.
These labels were derived from radiologist-assigned malignancy ratings rather than pathological confirmation and were therefore treated as operational labels for the present study. Excluding nodules with a median score of 3 reduced label ambiguity but restricted the analysis to a relatively less ambiguous subset of LIDC-IDRI.
Fig 5 provides a representative visualization of the individual radiologist annotations and the corresponding consensus mask.
Colored solid contours indicate the nodule boundaries independently delineated by four radiologists (red: Annotation 1; green: Annotation 2; blue: Annotation 3; yellow: Annotation 4). The black dashed contour represents the boundary of the 50%-agreement consensus mask. The consensus region captures the overlapping portion of the radiologist annotations and was used to construct the lesion mask for subsequent processing.
The 869 nodules were divided at the nodule level into training, validation, and test sets using stratified random sampling at an approximate ratio of 70:15:15, yielding 609 training nodules, 130 validation nodules, and 130 test nodules. The corresponding benign/malignant distributions were 314/295, 67/63, and 67/63, respectively. All axial, coronal, and sagittal images and lesion masks derived from the same nodule were assigned to the same subset. Because partitioning was performed at the nodule level rather than the patient level, distinct nodules from the same patient could be assigned to different subsets. Therefore, the protocol prevented nodule-level overlap but did not enforce complete patient-level independence. Unless otherwise specified, all performance metrics were calculated and reported at the nodule level.
The CT volumes were processed on their native voxel grids without isotropic spatial resampling. For each eligible nodule, a fixed-size 50 × 50 × 50-voxel patch was cropped around the centroid of the consensus mask. From each patch, the slice with the largest lesion-mask area was selected in each of the axial, coronal, and sagittal directions rather than the geometrical middle slice. The CT slices were resized to 224 × 224 using bilinear interpolation, whereas the corresponding binary masks were resized using nearest-neighbor interpolation. Fig 6 provides representative examples of tri-view nodule images, highlighting the complementary anatomical information captured by different planes.
To reduce intensity variability caused by different scanners and reconstruction settings, all images are normalized based on Hounsfield Units (HU). Specifically, intensity values are clipped to the range [−1000, 400] HU and then linearly rescaled to the range [0, 1] to provide a stable input distribution for feature learning. Each normalized grayscale slice was then replicated across three channels to provide an RGB-compatible input for the encoder networks. To enhance generalization, we applied on-the-fly (online) data augmentation during training. Random transformations were generated when each training sample was loaded rather than being pre-generated and stored as additional image files. Therefore, the same original nodule could undergo different transformations across training epochs. For each nodule, synchronized two-dimensional geometric transformations were applied to the axial, coronal, and sagittal images and their corresponding masks within each training iteration. These transformations included small rotations (±7°), translations (≤5%), scaling (0.95–1.05×), random horizontal flipping (probability 0.5), and random cropping retaining 90–100% of the image area. Mild photometric perturbations, including gamma correction (0.9–1.1) and contrast adjustment (0.9–1.1), were applied only to the CT images. During validation and testing, no random augmentation was applied; only deterministic preprocessing was performed.
Training samples were randomly shuffled at each epoch using the standard data-loading procedure. No class-balanced sampler, class reweighting, or class-weighted loss was applied because the training subset was approximately balanced.
Experimental settings.
Data preprocessing was performed on a local workstation running Windows 10 using Python version 3.9.8. The LIDC-IDRI data were accessed and organized using pylidc version 0.2.3 and pydicom version 2.4.4. Core numerical and image-processing operations were implemented using NumPy version 2.0.2, pandas version 2.3.3, OpenCV (opencv-python) version 5.0.0.93, and Pillow version 11.3.0. Model development, training, and inference were conducted on an AutoDL workstation equipped with a single NVIDIA graphics processing unit with 32 GB of memory. The model environment used Python version 3.10.8, PyTorch version 2.1.2, torchvision version 0.16.2, and einops version 0.8.2, with CUDA 12.1 support. To support reproducibility, the random seed for model training and data loading was fixed to 42. Data loading used eight worker processes with pinned memory enabled.
During inference, the three preprocessed views of each nodule were jointly processed as a single sample to generate one benign–malignant probability for nodule-level evaluation.
All models were trained from scratch without ImageNet or other pretrained weights. Model parameters were initialized using the default initialization routines of the corresponding PyTorch modules, and no additional global initialization procedure was applied. AdamW was used with an initial learning rate of 5 × 10−5 and a weight decay of 0.05. The learning rate was linearly increased from 1 × 10 − 6–5 × 10 − 5 during the first five epochs and was subsequently reduced to a minimum of 1 × 10 − 6 using cosine decay. Training was performed for up to 30 epochs with a batch size of 16, and the gradient norm was clipped at 1.0. Early stopping was triggered if the validation AUC did not improve for 10 consecutive epochs, and the checkpoint with the highest validation AUC was retained for test-set evaluation.
For baseline comparison, ResNet50 [29], DenseNet121 [30], EfficientNet-B3 [31], ViT[20], Swin Transformer [21], and MedViT[22] were trained and evaluated using the same nodule-level data partition, preprocessing pipeline, augmentation settings, loss function, model-selection criterion, and test-set evaluation procedure as the proposed model. Each baseline model used the representative axial slice as the single-view input and was trained without pretrained weights.
After the checkpoint with the highest validation AUC was selected, the classification threshold was determined on the validation set by maximizing the Youden index and was then applied unchanged to the test set. AUC was calculated from the continuous prediction probabilities.
Ablation study design
To assess the contribution of each component, five configurations were evaluated: (1) Baseline, a single-view MedViT model; (2) MV, a tri-view model without LGA or VWF; (3) MV + VWF; (4) MV + LGA; and (5) Full, which incorporated MV, LGA, and VWF. In the MV and MV + LGA configurations, the three view features were combined by direct concatenation, whereas VWF was used only in configurations explicitly marked with VWF. All configurations used the same nodule-level data partition, preprocessing pipeline, loss function, model-selection criterion, and evaluation protocol. Accuracy (ACC), AUC, and F1-score were used as the primary metrics for the ablation analysis.
Visualization and interpretability analysis
To qualitatively examine the spatial evidence associated with model predictions, Grad-CAM was applied to the single-view MedViT baseline and the full model [23]. Heatmaps were generated for the predicted class using a predefined high-level feature layer and resized to the corresponding input-image resolution. Selected nodules from the held-out test set were used to compare the activation patterns of the two models. The same preprocessing and inference procedures used for test-set evaluation were applied. The heatmaps were used only for qualitative inspection of lesion-relevant activation patterns and were not interpreted as quantitative localization metrics or evidence of clinical validity.
Evaluation metrics
Classification performance was evaluated using accuracy (ACC), area under the receiver operating characteristic curve (AUC), sensitivity (recall), specificity, and F1-score.
Accuracy (ACC) measures the proportion of correctly classified samples among all samples and is defined as:
where TP, TN, FP, and FN denote true positives, true negatives, false positives, and false negatives, respectively, with malignant nodules treated as the positive class. Although the class proportions were closely matched across the training, validation, and test sets through stratified sampling, ACC and F1-score were interpreted together with sensitivity, specificity, and AUC because threshold-dependent metrics can be influenced by class prevalence.
Sensitivity (also referred to as Recall) evaluates the detection capability for malignant nodules, i.e., the proportion of true malignant cases correctly identified as malignant:
Specificity measures the ability to correctly reject benign nodules, i.e., the proportion of true benign cases correctly classified as benign:
Sensitivity and specificity were interpreted jointly because they reflect complementary errors involving false negatives and false positives.
To balance Precision and Recall, we further report the F1-score, defined as the harmonic mean of Precision and Recall:
where Precision is the proportion of predicted malignant nodules that are truly malignant:
Finally, we use AUC to quantify the overall discriminative ability of the model across different decision thresholds. Unlike threshold-dependent metrics, AUC provides a threshold-free assessment and is widely adopted for binary classification in medical imaging studies [5].
Because malignant nodules were treated as the positive class, sensitivity was interpreted together with ACC, specificity, F1-score, and AUC.
Statistical analysis
Statistical analyses were performed in Python version 3.10.8 using scikit-learn version 1.7.2 [32]. AUC was calculated from the continuous malignancy probabilities. For threshold-dependent metrics, the decision threshold was selected on the validation set by maximizing the Youden index, defined as sensitivity plus specificity minus one, and was subsequently applied unchanged to the held-out test set. ACC, sensitivity, specificity, and F1-score were calculated using this fixed threshold. To quantify uncertainty in the test-set performance estimates, 95% confidence intervals (CIs) were calculated using 5,000 stratified bootstrap resamples while preserving the benign and malignant class counts. The validation-derived decision threshold was kept fixed during bootstrap resampling, and the 2.5th and 97.5th percentiles were used as the CI bounds. No hypothesis testing or multiple-comparison procedures were performed.
Ethics statement
This study performed a secondary analysis of the publicly available, de-identified LIDC-IDRI dataset obtained from The Cancer Imaging Archive (TCIA). No human participants were recruited, and no identifiable private information was accessed; therefore, institutional review board approval and informed consent were not required.
Results
Comparison with baseline models
The test-set performance of the proposed model and baseline models is summarized in Table 2.
The corresponding receiver operating characteristic (ROC) curves are shown in Fig 7.
(A) ResNet50; (B) DenseNet121; (C) EfficientNet-B3; (D) Vision Transformer (ViT); (E) Swin Transformer; (F) MedViT; and (G) the proposed tri-view model. The area under the ROC curve (AUC) for each model is reported in the corresponding subplot.
The proposed tri-view model achieved an ACC of 93.08% (95% CI: 88.46–96.92%), an AUC of 97.06% (95% CI: 93.89–99.46%), and an F1-score of 92.68% (95% CI: 87.39–96.83%), with a sensitivity of 90.48% and a specificity of 95.52%. It obtained the highest ACC, AUC, and F1-score among all evaluated models. Among the single-view baselines, MedViT achieved the highest ACC (92.31%), F1-score (91.80%), and specificity (95.52%), DenseNet121 achieved the highest AUC (96.52%), and EfficientNet-B3 achieved the highest sensitivity (96.83%). Relative to the strongest single-view baseline for each metric, the proposed model improved ACC, AUC, and F1-score by 0.77, 0.54, and 0.88 percentage points, respectively, while matching the highest specificity. Bootstrap-derived 95% CIs for all evaluated models are provided in S2 Dataset.
Ablation study
The ablation results are summarized in Table 3.
The single-view Baseline achieved an ACC of 92.31%, an AUC of 94.65%, and an F1-score of 91.80%. Introducing tri-view input with direct concatenation (MV) increased the AUC to 95.78%, while the ACC and F1-score decreased to 90.77% and 90.32%, respectively.
Compared with direct concatenation in the MV configuration, replacing it with VWF increased the AUC from 95.78% to 96.14%, while the ACC and F1-score remained unchanged at 90.77% and 90.32%, respectively. The MV + LGA configuration improved the ACC, AUC, and F1-score to 91.54%, 97.28%, and 91.34%, respectively. The Full configuration achieved the highest ACC (93.08%) and F1-score (92.68%), whereas its AUC (97.06%) was slightly lower than that of MV + LGA (97.28%). Compared with the single-view Baseline, the Full configuration improved ACC, AUC, and F1-score by 0.77, 2.41, and 0.88 percentage points, respectively.
Grad-CAM visualization results
Selected Grad-CAM visualizations for test-set nodules are presented in Fig 8. The first column shows the original nodule images, while the second and third columns show the heatmaps generated by the MedViT baseline and the proposed multi-view model, respectively.
In the displayed cases, the MedViT baseline showed more diffuse activation patterns, with some responses extending into the surrounding lung tissue. The proposed model showed activations that were more concentrated within or adjacent to the nodule region. In several displayed cases, prominent responses were observed around portions of the nodule margins and internal regions.
Discussion
This study evaluated a tri-view pulmonary nodule classification framework combining lesion-prior-guided soft residual feature modulation with sample-adaptive view-weighted fusion. On the held-out test set, the proposed multi-view model achieved an ACC of 93.08%, an AUC of 97.06%, and an F1-score of 92.68%, obtaining the highest values for these three metrics among the models included in the baseline comparison. In the ablation study, the Full configuration achieved the highest ACC and F1-score, whereas MV + LGA achieved a slightly higher AUC. Its specificity of 95.52% also matched the highest single-view result. These findings support the effectiveness of combining lesion-aware feature refinement with adaptive multi-view fusion for pulmonary nodule classification.
Tri-planar representation provides complementary spatial information but is not itself considered the methodological novelty of this study. Compared with the closely related framework of Chang et al. [13], the present model uses lesion masks as weak spatial priors for high-level soft residual feature refinement and learns sample-specific, Softmax-normalized weights for adaptive view fusion. Table 4 summarizes these conceptual differences.
Because case selection, label definitions, exclusion criteria, data partitioning, preprocessing pipelines, input representations, and evaluation protocols differ across studies, cross-study performance values were not interpreted as direct evidence of superiority. The quantitative conclusions of the present study therefore rely primarily on the baseline comparison and ablation experiments conducted using the same nodule-level data partition and evaluation protocol.
The ablation results showed that tri-view input with direct concatenation increased AUC but reduced ACC and F1-score relative to the single-view Baseline. Replacing direct concatenation with VWF further increased AUC, while ACC and F1-score remained unchanged. By contrast, adding LGA to MV improved all three reported metrics and produced the highest AUC among the evaluated configurations. The Full configuration achieved the highest ACC and F1-score, although its AUC was slightly lower than that of MV + LGA. These results indicate that the modules affected AUC and threshold-dependent metrics differently. AUC evaluates the model’s ability to rank malignant nodules above benign nodules across different decision thresholds, whereas ACC and F1-score depend on the single decision threshold selected from the validation set. Direct concatenation and VWF improved AUC without corresponding improvements in ACC and F1-score, while LGA improved all three metrics. Accordingly, the Full model achieved the highest ACC and F1-score at the selected threshold, whereas MV + LGA achieved the highest AUC.
Selected Grad-CAM visualizations provided qualitative information regarding the spatial patterns associated with model predictions. In the displayed cases, the proposed model showed activation patterns that were less diffuse in the surrounding lung tissue and more concentrated within or adjacent to the nodule region than those of the MedViT baseline. These observations are useful for qualitative inspection of whether model predictions are associated with lesion-relevant regions. However, Grad-CAM heatmaps should not be interpreted as quantitative localization measurements, clinical evidence, or causal explanations of model decisions.
From a potential clinical-use perspective, the proposed framework could serve as a computer-aided second-reader tool after a pulmonary nodule has been identified and a lesion mask is available. The consensus masks used in this study were derived from existing LIDC-IDRI annotations and do not imply that four radiologists would be required to delineate each lesion in future clinical use. In practice, the required lesion mask could potentially be obtained from an existing clinical contour or an automated segmentation method, although these alternatives were not evaluated in the present study. The model could then provide a malignancy probability based on complementary tri-view information to support radiologist assessment rather than replace the radiologist’s diagnostic judgment. However, prospective clinical evaluation, including its effect on reader performance and workflow efficiency, is required before such use can be established.
Several limitations should be acknowledged. First, all experiments were conducted on a single LIDC-IDRI cohort, and no independent external validation was performed. The LGA module also relies on annotation-derived lesion masks as weak spatial priors. Application to external data would require manual or automatically generated masks, and robustness to imperfect masks remains to be assessed. In addition, data partitioning was performed at the nodule level rather than the patient level. Consequently, distinct nodules from the same patient could occur in different subsets, introducing residual patient-level correlation and potentially yielding less conservative performance estimates than a fully patient-independent evaluation. Although no individual nodule or its derived tri-view images and masks crossed subsets, future studies should evaluate the framework using patient-level partitions and independent external cohorts.
Second, the benign–malignant labels were defined from the median radiologist-assigned malignancy scores rather than pathological confirmation. Excluding nodules with a median score of 3 reduced label ambiguity but restricted the evaluation to relatively less ambiguous nodules. Third, selecting only one two-dimensional slice with the largest lesion-mask area in each view does not fully capture continuous volumetric context. Future work should assess the framework on independent multi-center cohorts, evaluate its robustness under heterogeneous acquisition conditions and imperfect segmentation masks, incorporate relevant clinical variables, and investigate efficient volumetric modeling strategies.
Conclusion
We developed a tri-view pulmonary nodule classification framework that integrates lesion-prior-guided soft residual feature modulation with sample-adaptive view-weighted fusion to exploit complementary information from axial, coronal, and sagittal views. On the held-out LIDC-IDRI test set, the proposed framework achieved an ACC of 93.08%, an AUC of 97.06%, and an F1-score of 92.68%. The ablation results showed that combining LGA and VWF provided the best overall performance balance among the evaluated configurations. In the displayed test cases, Grad-CAM visualizations showed that model activations were concentrated mainly within or adjacent to lesion-relevant regions.
Because the study was conducted using a single public dataset and annotation-derived lesion masks, independent multi-center validation and further evaluation with imperfect or automatically generated masks are needed before broader application.
Supporting information
S1 Dataset. Nodule-level data-partition manifest. This file contains the nodule identifiers, ground-truth labels, and training, validation, or test-set assignment for the 869 eligible nodules.
https://doi.org/10.1371/journal.pone.0357963.s001
(ZIP)
S2 Dataset. Per-nodule prediction results, validation-derived classification thresholds, and bootstrap confidence intervals.
This file contains the ground-truth labels, per-nodule prediction probabilities, validation-derived decision thresholds, and bootstrap-derived 95% confidence intervals for the evaluated models.
https://doi.org/10.1371/journal.pone.0357963.s002
(ZIP)
S1 Raw Images. Source images underlying the image-based figures reported in the manuscript.
This study did not involve blot or gel experiments. The file includes computer-generated methodological schematics, CT images, radiologist-provided annotations, consensus masks, tri-view model-input images, ROC curves, and Grad-CAM visualizations.
https://doi.org/10.1371/journal.pone.0357963.s003
(PDF)
Acknowledgments
The authors thank The Cancer Imaging Archive (TCIA) for hosting the publicly available LIDC-IDRI dataset used in this study.
References
- 1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71(3):209–49.
- 2. Blandin Knight S, Crosbie PA, Balata H, Chudziak J, Hussell T, Dive C. Progress and prospects of early detection in lung cancer. Open Biol. 2017;7(9):170070. pmid:28878044
- 3. National Lung Screening Trial Research Team, Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395–409. pmid:21714641
- 4. Henschke CI, Yip R, Shaham D, Markowitz S, Cervera Deval J, Zulueta JJ, et al. A 20-year follow-up of the International Early Lung Cancer Action Program (I-ELCAP). Radiology. 2023;309(2):e231988. pmid:37934099
- 5. Gu Y, Chi J, Liu J, Yang L, Zhang B, Yu D, et al. A survey of computer-aided diagnosis of lung nodules from CT scans using deep learning. Comput Biol Med. 2021;137:104806. pmid:34461501
- 6. Shen W, Zhou M, Yang F, Yu D, Dong D, Yang C, et al. Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognit. 2017;61:663–73.
- 7.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. In: Guyon I, von Luxburg U, Bengio S, Wallach H, Fergus R, Vishwanathan S, et al. editors. Advances in Neural Information Processing Systems 30. Proceedings of the 31st Conference on Neural Information Processing Systems; 2017 Dec 4–9; Long Beach, CA, USA. Red Hook (NY): Curran Associates, Inc.; 2017. p. 5998–6008. Available from: https://proceedings.neurips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
- 8. Jin H, Yu C, Zhang J, Zheng R, Fu Y, Zhao Y. Multitask Swin Transformer for classification and characterization of pulmonary nodules in CT images. Quant Imaging Med Surg. 2025;15(3):1845–61. pmid:40160630
- 9. Liu K, Kang G. Multiview convolutional neural networks for lung nodule classification. Int J Imaging Syst Tech. 2017;27(1):12–22.
- 10. Kang G, Liu K, Hou B, Zhang N. 3D multi-view convolutional neural networks for lung nodule classification. PLoS One. 2017;12(11):e0188290. pmid:29145492
- 11. Schlemper J, Oktay O, Schaap M, Heinrich M, Kainz B, Glocker B, et al. Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal. 2019;53:197–207. pmid:30802813
- 12.
Zhu W, Liu C, Fan W, Xie X. DeepLung: deep 3D dual path nets for automated pulmonary nodule detection and classification. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV); 2018 Mar 12–15; Lake Tahoe, NV, USA. Piscataway (NJ): IEEE; 2018. p. 673–81. https://doi.org/10.1109/WACV.2018.00079
- 13. Chang H-H, Wu C-Z, Gallogly AH. Pulmonary nodule classification using a multiview residual selective kernel network. J Imaging Inform Med. 2024;37(1):347–62. pmid:38343233
- 14. Causey JL, Zhang J, Ma S, Jiang B, Qualls JA, Politte DG, et al. Highly accurate model for prediction of lung nodule malignancy with CT scans. Sci Rep. 2018;8(1):9286. pmid:29915334
- 15. Dou Q, Chen H, Yu L, Qin J, Heng P-A. Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng. 2017;64(7):1558–67. pmid:28113302
- 16.
Hussein S, Cao K, Song Q, Bagci U. Risk stratification of lung nodules using 3D CNN-based multi-task learning. Information Processing in Medical Imaging: 25th International Conference, IPMI 2017; 2017 Jun 25–30; Boone, NC, USA. Cham: Springer; 2017. p. 249–60. https://doi.org/10.1007/978-3-319-59050-9_20
- 17. Onishi Y, Teramoto A, Tsujimoto M, Tsukamoto T, Saito K, Toyama H, et al. Investigation of pulmonary nodule classification using multi-scale residual network enhanced with 3DGAN-synthesized volumes. Radiol Phys Technol. 2020;13(2):160–9. pmid:32358643
- 18. Xie Y, Xia Y, Zhang J, Song Y, Feng D, Fulham M, et al. Knowledge-based collaborative deep learning for benign-malignant lung nodule classification on chest CT. IEEE Trans Med Imaging. 2019;38(4):991–1004. pmid:30334786
- 19. Shamshad F, Khan S, Zamir SW, Khan MH, Hayat M, Khan FS, et al. Transformers in medical imaging: a survey. Med Image Anal. 2023;88:102802. pmid:37315483
- 20.
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16x16 words: transformers for image recognition at scale. International Conference on Learning Representations; 2021 May 3–7; Virtual Event; 2021. Available from: https://openreview.net/forum?id=YicbFdNTTy
- 21.
Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, et al. Swin Transformer: hierarchical vision transformer using shifted windows. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision; 2021 Oct 10–17; Montreal, QC, Canada. Piscataway (NJ): IEEE; 2021. p. 10012–22. https://doi.org/10.1109/ICCV48922.2021.00986
- 22. Manzari ON, Ahmadabadi H, Kashiani H, Shokouhi SB, Ayatollahi A. MedViT: a robust vision transformer for generalized medical image classification. Comput Biol Med. 2023;157:106791. pmid:36958234
- 23.
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. Proceedings of the 2017 IEEE International Conference on Computer Vision; 2017 Oct 22–29; Venice, Italy. Piscataway (NJ): IEEE; 2017. p. 618–26. https://doi.org/10.1109/ICCV.2017.74
- 24.
Rizhko M, Erdman L, Rickard M, Sheth K, Alvarez D, Velaer KN, et al. Improving identically distributed and out-of-distribution medical image classification with segmentation-guided attention in small dataset scenarios. Proceedings of the 7th International Conference on Medical Imaging with Deep Learning, vol. 250; 2024 Jul 3–5; Paris, France. Proceedings of Machine Learning Research. 2024. p. 1282–96. Available from: https://proceedings.mlr.press/v250/rizhko24a.html
- 25. Armato SG 3rd, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, et al. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys. 2011;38(2):915–31. pmid:21452728
- 26. Beig N, Khorrami M, Alilou M, Prasanna P, Braman N, Orooji M, et al. Perinodular and intranodular radiomic features on lung CT images distinguish adenocarcinomas from granulomas. Radiology. 2019;290(3):783–92. pmid:30561278
- 27. Jacobs RA, Jordan MI, Nowlan SJ, Hinton GE. Adaptive mixtures of local experts. Neural Comput. 1991;3(1):79–87. pmid:31141872
- 28.
Loshchilov I, Hutter F. Decoupled weight decay regularization. International Conference on Learning Representations; 2019 May 6–9; New Orleans, LA, USA; 2019. Available from: https://openreview.net/forum?id=Bkg6RiCqY7
- 29.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition; 2016 Jun 27–30; Las Vegas, NV, USA. Piscataway (NJ): IEEE; 2016. p. 770–8. https://doi.org/10.1109/CVPR.2016.90
- 30.
Huang G, Liu Z, van der Maaten L, Weinberger KQ. Densely connected convolutional networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition; 2017 Jul 21–26; Honolulu, HI, USA. Piscataway (NJ): IEEE; 2017. p. 4700–8. https://doi.org/10.1109/CVPR.2017.243
- 31.
Tan M, Le QV. EfficientNet: rethinking model scaling for convolutional neural networks. In: Chaudhuri K, Salakhutdinov R, editors. Proceedings of the 36th International Conference on Machine Learning, vol. 97; 2019 Jun 9–15; Long Beach, CA, USA. Proceedings of Machine Learning Research; 2019. p. 6105–14. Available from: https://proceedings.mlr.press/v97/tan19a.html
- 32. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825–30.