2.1 Healthcare AI

Healthcare AI represents a transformative approach in modern medicine that leverages artificial intelligence to analyze complex medical data and improve healthcare delivery [27]. This interdisciplinary field combines machine learning, natural language processing, and computer vision to extract meaningful insights from diverse healthcare data sources including electronic health records, medical images, and genomic sequences. The fundamental principle involves training algorithms on large medical datasets to recognize patterns, predict outcomes, and assist clinical decision-making, while continuously improving through iterative learning processes. Various architectures such as deep neural networks [28], random forests [29], and support vector machines [30] are adapted to handle the unique challenges of healthcare data including its heterogeneity, temporal nature, and privacy requirements.

The technology can be broadly categorized into diagnostic AI [31], predictive analytics [32], and operational optimization systems [33]. Diagnostic tools [34] primarily focus on image recognition and clinical text interpretation, while predictive models forecast disease progression and treatment responses [35,36]. Operational applications streamline hospital workflows and resource allocation. Recent advances incorporate explainable AI techniques [37] to enhance model transparency and hybrid approaches that combine multiple data modalities. Emerging subfields include federated learning for privacy-preserving multi-institutional collaborations and reinforcement learning for personalized treatment optimization.

Key advantages of healthcare AI include its ability to process vast amounts of data beyond human capacity, identify subtle correlations invisible to clinicians, and provide real-time decision support. These systems demonstrate particular strength in repetitive pattern recognition tasks, reducing diagnostic errors and variability in clinical interpretations. AI applications can operate continuously without fatigue, enabling round-the-clock patient monitoring and early warning systems. The technology also facilitates personalized medicine by analyzing individual patient characteristics against population-level data, potentially improving treatment efficacy while reducing adverse effects.

Application scenarios span the entire healthcare continuum from prevention to palliative care. In clinical settings, AI assists with medical imaging interpretation, clinical documentation, and risk stratification. Pharmaceutical companies employ AI for drug discovery and clinical trial optimization. Public health organizations utilize predictive models for disease surveillance and outbreak forecasting. Patient-facing applications include virtual health assistants and remote monitoring tools. Administrative functions benefit from AI-powered billing automation, fraud detection, and resource scheduling systems.

2.2 EHR data analysis

EHR data analysis [10] has emerged as a critical discipline in modern healthcare informatics, focusing on extracting meaningful insights from the vast amounts of structured and unstructured data contained in electronic health records. The field employs various computational techniques ranging from traditional statistical methods [38] to advanced machine learning algorithms [39], all designed to process heterogeneous medical data including clinical notes, laboratory results, medication records, and demographic information. At its core, EHR analysis involves data preprocessing [40] to handle missing values and inconsistencies, feature engineering [41] to identify clinically relevant variables, and modeling approaches tailored to healthcare’s unique characteristics such as temporal dependencies and irregular sampling frequencies [42]. The technology stack typically incorporates natural language processing for clinical text mining [43], temporal modeling for longitudinal patient data analysis, and network analysis [44] for understanding disease comorbidities. These methods enable the transformation of raw EHR data into actionable knowledge while addressing challenges specific to medical data such as privacy concerns, documentation variability across institutions, and the need for interpretable results in clinical settings.

The applications of EHR data analysis span multiple categories including descriptive analytics for population health management [45], predictive modeling for clinical risk stratification [46], and prescriptive analytics for decision support. Major advantages include the ability to uncover hidden patterns in large-scale patient populations, facilitate evidence-based medicine through real-world data analysis, and enable personalized care by identifying patient-specific risk factors. Temporal analysis techniques allow tracking of disease progression and treatment effectiveness over time, while federated learning approaches enable multi-institutional studies without compromising data privacy. Despite these benefits, significant challenges remain in ensuring data quality, handling the high dimensionality of medical features, and maintaining model generalizability across diverse patient populations. The field continues to evolve with advancements in deep learning architectures capable of processing multimodal EHR data, as well as the development of standardized frameworks for evaluating analytical models in healthcare contexts.

2.3 Clinical text mining and clinical decision systems

Clinical text mining [47] represents a crucial component of modern healthcare informatics, focusing on extracting structured medical knowledge from unstructured clinical narratives such as physician notes, discharge summaries, and radiology reports. This field primarily utilizes natural language processing (NLP) techniques [48] ranging from rule-based systems to advanced deep learning models, with transformer-based architectures [49] recently demonstrating superior performance in understanding clinical context and medical terminology. The technology enables automated identification of key clinical entities including diagnoses [50], symptoms [51], medications [52], and procedures, while addressing challenges specific to medical documentation such as abbreviations, negations, and temporal references. These extracted structured data elements serve as fundamental inputs for various downstream clinical applications and decision support systems.

Clinical decision systems leverage processed clinical data to provide actionable insights and evidence-based recommendations at the point of care. These systems can be broadly categorized into knowledge-based systems [53] that rely on predefined medical rules and machine learning-based systems [54] that learn patterns from historical patient data. Modern implementations increasingly combine both approaches to balance clinical validity with adaptability to local practice patterns. The technology demonstrates particular strength in areas such as differential diagnosis generation, medication error prevention, and personalized treatment suggestions, while maintaining capabilities for continuous learning and improvement as new clinical evidence emerges.

The integration of clinical text mining with decision support systems offers significant advantages including improved diagnostic accuracy, enhanced patient safety, and increased healthcare efficiency. Automated processing of clinical narratives reduces documentation burden while ensuring critical information is captured in computable formats. Decision systems powered by comprehensive patient data can identify subtle clinical patterns that might be overlooked in manual review, potentially leading to earlier interventions and better outcomes. However, challenges remain regarding system interpretability, seamless workflow integration, and maintaining performance across diverse healthcare settings with varying documentation practices and patient populations.

3.1 Overview

This study proposes an integrated deep learning framework for automated clinical diagnosis classification from electronic health records, combining three synergistic technical components in a carefully designed processing pipeline. The system first employs a Transformer-based architecture to process unstructured clinical narratives, generating dense semantic embeddings that capture critical medical concepts and their contextual relationships through self-attention mechanisms. These comprehensive text representations, enriched with structured patient metadata, subsequently feed into a multi-task learning module that simultaneously optimizes multiple clinically relevant prediction tasks through shared parameter learning. The entire framework benefits from transfer learning initialization, where the Transformer component is pretrained on external medical corpora before domain-specific fine-tuning, effectively addressing data scarcity challenges while preserving generalized medical knowledge.

The proposed architecture addresses several fundamental challenges in clinical NLP through its modular design. The Transformer module overcomes information extraction difficulties from heterogeneous clinical documentation by automatically identifying and encoding key diagnostic indicators, symptoms, and treatments. The MTL component enhances diagnostic accuracy by leveraging inter-task relationships while reducing computational overhead compared to separate single-task models. The transfer learning approach not only accelerates convergence but also improves cross-institutional generalizability by bridging the gap between different EHR systems. This integrated solution provides an end-to-end framework that transforms raw clinical text into actionable diagnostic predictions while maintaining adaptability to various healthcare settings and evolving medical terminologies through its modular and extensible architecture.The structure of the model proposed in this study is shown in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The structure of the model proposed in this study.

It is an end-to-end pipeline integrating Transformer-based text embedding, multi-task learning, and transfer learning modules.

https://doi.org/10.1371/journal.pone.0329963.g001

3.2 Data preprocessing

The data preprocessing pipeline is specifically designed to handle the unique characteristics of clinical narratives while maintaining compatibility with the subsequent Transformer-based feature extraction module. Raw EHR data undergoes comprehensive cleaning to remove noise and standardize medical terminologies, employing domain-specific rules and medical ontologies to normalize abbreviations and synonyms. This step ensures consistent input representation for the downstream neural network while preserving critical clinical semantics that are essential for accurate diagnosis classification.

A specialized tokenization approach is implemented to address the complex linguistic patterns in medical documentation, combining subword tokenization with clinical concept recognition. The preprocessing retains temporal markers and negation cues that are particularly important for clinical decision-making, while handling the irregular structure of physician notes through intelligent sentence boundary detection. This processing maintains the contextual relationships between medical entities that the Transformer architecture subsequently leverages through its self-attention mechanisms.

For structured metadata integration, the pipeline implements automated feature engineering that aligns temporal clinical measurements with corresponding text entries. This synchronization enables the multi-task learning module to effectively combine textual and numerical patient characteristics. The preprocessing specifically preserves temporal sequences of clinical events that prove crucial for modeling disease progression patterns in the subsequent analysis stages.

The innovation lies in the context-aware preprocessing that goes beyond conventional NLP pipelines by incorporating medical knowledge graphs to resolve clinical ambiguities during the cleaning phase. This approach maintains the integrity of medically significant relationships while reducing noise, directly supporting the model’s ability to capture subtle diagnostic indicators. The processed output provides optimized input for the Transformer module while ensuring all clinically relevant information is preserved in a machine-interpretable format.The data preprocessing process in this study is shown in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The data preprocessing process in this study (This figure details the specialized clinical text normalization workflow, including medical concept recognition, temporal marker preservation, and ontology-based noise reduction to prepare EHR data for Transformer processing).

https://doi.org/10.1371/journal.pone.0329963.g002

3.3 Transformer-based text embedding

The Transformer-based text embedding module serves as the foundation of our clinical feature extraction pipeline, processing preprocessed EHR narratives into dense semantic representations. Building upon the cleaned and normalized clinical text from the preprocessing stage, the model employs multi-head self-attention mechanisms to capture long-range dependencies between medical concepts. The embedding process begins with token-level representations combining word embeddings and positional encodings :

(1)

where x_i denotes the i-th token in the clinical text, and d_model represents the embedding dimension. This initial representation preserves both semantic meaning and temporal ordering of medical events, crucial for accurate diagnosis interpretation.

The core innovation lies in our domain-adapted attention mechanism that weights clinical concepts differently based on their diagnostic significance. For each attention head k, the scaled dot-product attention computes:

(2)

where , , and are learned linear transformations of the input embeddings, and d_k is the dimension of key vectors. Our medical-specific modification introduces a diagnostic relevance bias term B_med learned from clinical annotations:

(3)

This enhancement allows the model to prioritize clinically important relationships while maintaining the flexibility to discover novel diagnostic patterns.

The module stacks L identical layers, each applying multi-head attention followed by position-wise feed-forward networks (FFN) with layer normalization (LayerNorm) and residual connections:

(4)

(5)

where W₁, W₂ are learned weights and b₁, b₂ are bias terms. The hierarchical processing enables the model to build increasingly sophisticated representations of clinical narratives.

The final embedding combines the [CLS] token representation with aggregated concept-specific features through medical concept pooling:

(6)

where C denotes the set of clinically significant concept positions identified during preprocessing. This dual-representation approach provides both global context and focused medical knowledge for downstream tasks.

The output embeddings z capture rich clinical semantics while maintaining computational efficiency through parallel processing of entire patient records. This design specifically addresses the challenge of modeling lengthy, irregular clinical narratives while preserving critical diagnostic information for the subsequent multi-task learning module.The principle of Transformer-based text embedding process is shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The principle of Transformer-based text embedding process in this study.

https://doi.org/10.1371/journal.pone.0329963.g003

3.4 Multi-task learning framework

The multi-task learning (MTL) framework is designed to leverage the rich clinical representations () generated by the Transformer module, combining them with structured patient metadata () for comprehensive diagnostic analysis. The input fusion is achieved through a gated concatenation mechanism that dynamically weights different data modalities:

(7)

where W_z, W_m are learnable projection matrices, b_z, b_m are bias terms, and σ denotes the sigmoid activation function. This adaptive fusion preserves the most informative aspects of both unstructured clinical narratives and structured patient characteristics.

The shared encoder architecture employs stacked dense layers with residual connections to extract task-agnostic clinical features:

(8)

for shared layers. The residual connections ensure stable gradient flow while maintaining the integrity of the original clinical embeddings throughout the network depth.

Our innovation lies in the clinical task relationship matrix , where T is the number of tasks, which learns cross-task dependencies during training:

(9)

where U_ij are learnable similarity metrics. This matrix automatically discovers and exploits clinically meaningful relationships between different diagnostic categories and prediction tasks.

Task-specific decoders branch from the shared representation h_shared, each employing a tailored architecture:

(10)

for classification tasks, where W_t and b_t are task-specific parameters. For continuous predictions, we use linear output layers with appropriate activation functions.

The joint optimization combines task losses through our novel clinical importance-weighted scheme:

(11)

where represents the clinical significance score for task t, derived from medical knowledge bases. This ensures that more critical diagnostic predictions receive appropriate emphasis during training.

The framework incorporates gradient conflict resolution through our modified GradNorm algorithm:

(12)

which dynamically balances task-specific gradients based on their learned relationships, preventing negative transfer while promoting beneficial knowledge sharing.The multi-task learning (MTL) framework in this study is shown in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. The multi-task learning framework in this study.

https://doi.org/10.1371/journal.pone.0329963.g004

3.5 Transfer learning optimization

The transfer learning optimization module addresses the critical challenge of limited annotated EHR data by leveraging knowledge from source domain medical corpora to enhance performance on target domain EHR data . The process begins with a pre-trained clinical Transformer model initialized on large-scale medical text, which provides robust semantic representations of general medical concepts. Our domain adaptation employs a two-phase optimization strategy:

(13)

where preserves general medical knowledge, adapts to institution-specific patterns, and λ controls adaptation strength. This balanced approach prevents catastrophic forgetting while enabling effective specialization.

The innovation lies in our clinical domain adapter layers that bridge general medical knowledge and local EHR characteristics. For each Transformer layer l, we insert lightweight adapter modules A_l with bottleneck architecture:

(14)

where , () project activations to reduced dimension and back. These adapters enable efficient fine-tuning while freezing 90% of pre-trained parameters, making the approach practical for resource-constrained healthcare settings.

We introduce a novel clinical domain confusion loss to improve cross-institution generalization:

(15)

where D is a domain classifier trained to distinguish source and target domain samples, encouraging the model to learn domain-invariant clinical representations. This is combined with the primary diagnostic loss through adaptive weighting:

(16)

where γ adjusts the trade-off and D_acc is the domain classifier accuracy, automatically reducing confusion as domains align.

The optimization employs a curriculum learning schedule that progressively focuses from general medical concepts to specific diagnostic patterns:

(17)

where t is training step and τ controls the transition pace. This mimics clinical learning trajectories, first building broad medical knowledge before specializing.

For final model deployment, we apply knowledge distillation to compress the adapted model into a more efficient architecture suitable for clinical environments:

(18)

balancing performance preservation with computational efficiency. The complete transfer learning pipeline yields models that maintain 98% of the accuracy of full fine-tuning while requiring only 30% of the training data. The transfer learning framework in this study is shown in Fig 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. The transfer learning framework in this study (This figure explains the two-phase optimization with clinical domain adapters and confusion loss, showing how pretrained ClinicalBERT weights are adapted to target institutions while mitigating catastrophic forgetting through curriculum learning).

https://doi.org/10.1371/journal.pone.0329963.g005

4.1 Experimental setup

The experimental environment was configured to ensure reproducible and efficient model training. All experiments were conducted on a high-performance computing cluster equipped with 8 NVIDIA A100 GPUs (40GB memory each) and AMD EPYC 7763 processors. The software stack included Python 3.9, PyTorch 1.12 with CUDA 11.6 acceleration, and the HuggingFace Transformers library for the pretrained clinical language model components. Memory allocation was optimized through automatic mixed precision training and gradient checkpointing to handle the large-scale EHR datasets. The entire framework was containerized using Docker for consistent deployment across different clinical research environments.

Model parameters were carefully selected based on extensive preliminary experiments. The Transformer backbone used 12 layers with 768-dimensional hidden states and 12 attention heads, totaling 110M trainable parameters. The multi-task learning head comprised 3 shared dense layers (512, 256, 128 units) with ReLU activation and dropout (p = 0.3). Batch sizes were dynamically adjusted between 16-64 depending on task complexity, with gradient accumulation steps employed for stability. The clinical concept embedding dimension was fixed at 256 across all experiments to maintain compatibility with existing medical knowledge graphs. All trainable parameters were initialized using Xavier uniform initialization except for pretrained components.

Training procedures incorporated several optimization strategies. We employed the AdamW optimizer with weight decay (0.01) and linear warmup over the first 10% of training steps. The learning rate was set to 3e-5 for pretrained layers and 1e-4 for task-specific components. Early stopping monitored validation loss with patience of 10 epochs. For the transfer learning phase, we used a triangular cyclical learning rate schedule between 1e-5 and 5e-5 to escape local optima. Each training run was repeated 5 times with different random seeds to assess stability, with final metrics reported as mean±standard deviation across runs.

4.2 Experimental setup

To ensure rigorous assessment of model performance, we selected a comprehensive set of evaluation metrics that capture different aspects of clinical utility. These metrics were chosen based on their relevance to medical decision-making and their ability to provide complementary perspectives on model effectiveness.The primary evaluation metrics included:

Accuracy measures overall classification performance by comparing correct predictions against all cases:

where TP, TN, FP, and FN represent true positives, true negatives, false positives, and false negatives respectively. While useful for balanced datasets, accuracy can be misleading in cases of class imbalance.

F1-score provides a balanced measure between precision and recall, particularly important for clinical applications where both false positives and false negatives carry consequences:

We employed class-weighted averaging to account for any dataset imbalances.

Precision and Recall metrics were also tracked individually, as they offer distinct clinical insights:

For binary classification tasks, we computed the Area Under the Receiver Operating Characteristic curve (AUC-ROC):

where TPR (true positive rate) and FPR (false positive rate) are evaluated across all possible classification thresholds. The AUC provides a robust measure of model discrimination ability, with values ranging from 0.5 (random guessing) to 1.0 (perfect discrimination).

4.3 Dataset and benchmarks

The datasets used in this study include MIMIC-III,eICU Collaborative Research Database, PhysioNet 2020 Challenge Dataset, and UK Biobank.

The MIMIC-III database is a comprehensive, freely accessible critical care dataset containing de-identified health data from over 40,000 patients admitted to intensive care units at a tertiary care hospital between 2001 and 2012. It includes detailed clinical information such as vital signs, laboratory measurements, medications, caregiver notes, imaging reports, and mortality outcomes. The database consists of multiple modules covering hospital admissions (MIMIC-Core), ICU stays (ICU module), emergency department visits (ED module), and chest x-rays (MIMIC-cxr). For this study, MIMIC-III provides valuable longitudinal ICU patient data with rich clinical annotations, enabling the investigation of temporal patterns in critical care conditions and their outcomes. The dataset’s granular physiological measurements and clinical notes are particularly useful for developing predictive models in intensive care settings.

The eICU Collaborative Research Database contains de-identified health records from more than 200,000 ICU admissions across multiple hospitals in the United States between 2014 and 2015. This multicenter database includes high-frequency vital signs, laboratory results, medication orders, diagnosis codes, treatment information, and clinician notes collected from ICU information systems. Unlike single-center datasets, eICU offers geographic and institutional diversity, capturing practice variations across different healthcare settings. For this research, eICU complements MIMIC-III by providing broader population representation and more recent data, allowing for validation of findings across different healthcare systems. The dataset’s scale and multicenter nature make it particularly valuable for developing generalizable models in critical care analytics.

The PhysioNet 2020 Challenge Dataset comprises 42,801 12-lead ECG recordings from multiple sources including the CPSC, INCART, PTB, PTB-XL, and Georgia databases, with sampling frequencies ranging from 257Hz to 1000Hz. Each recording includes waveform data and SNOMED-CT coded diagnoses for 27 cardiac abnormalities. The dataset features diverse demographics (male: 22,368; female: 20,433) and contains recordings of varying lengths (6-60 seconds). For this study, this comprehensive ECG dataset enables the development and evaluation of machine learning models for automated cardiac abnormality detection. The inclusion of multiple data sources with different recording characteristics provides opportunities to test model robustness across acquisition protocols and patient populations.

UK Biobank is a large-scale biomedical database containing genetic, lifestyle, and health information from 500,000 UK participants aged 40-69 years at recruitment. The resource includes extensive phenotypic data from questionnaires, physical measurements, biological samples, imaging studies, and electronic health records. For cardiovascular research, it provides ECG data, cardiac MRI images, and associated clinical outcomes. In this study, UK Biobank offers population-level data with long-term follow-up, enabling the investigation of risk factors and disease progression patterns. The dataset’s combination of genomic, phenotypic, and health outcome data allows for comprehensive analyses of biological mechanisms underlying critical care conditions in a general population context.

The benchmark models used in this study include logistic regression, random forest, and CNN model.

The logistic regression model serves as a simple yet interpretable baseline, utilizing statistical features extracted from ECG signals such as RR intervals and demographic variables. It applies a sigmoid function to predict the probability of cardiac abnormalities, providing a linear decision boundary that establishes fundamental performance benchmarks for more complex models.

The random forest classifier operates by constructing multiple decision trees during training and aggregating their predictions. Each tree is built using a random subset of features from the ECG data, including morphological and temporal characteristics, enabling robust performance through ensemble learning while mitigating overfitting compared to single decision trees.

The CNN processes raw ECG waveforms through hierarchical feature extraction using convolutional layers, followed by fully connected layers for classification. This deep learning approach automatically learns discriminative patterns from the input signals, capturing both local and global characteristics essential for accurate cardiac abnormality detection.

4.4 Analysis of experimental results

4.4.1 Robustness.

To evaluate model robustness across datasets, we conduct three experiments comparing our proposed model against logistic regression, random forest, and CNN baselines on MIMIC-III, eICU, and PhysioNet data. Each baseline is trained and tested separately on these critical care datasets to assess performance variations in ECG classification under different clinical settings and data distributions.

The logistic regression baseline demonstrated varying performance across datasets, achieving 72.3% accuracy on MIMIC-III, 68.1% on eICU, and 75.6% on PhysioNet (Table 1). This linear model’s performance differences reflect its sensitivity to data distribution shifts - the higher PhysioNet accuracy suggests its features better align with the model’s linear decision boundary assumption. The 4.2% drop from MIMIC-III to eICU indicates challenges in generalizing across hospital systems with different recording protocols.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Logistic regression performance across datasets.

https://doi.org/10.1371/journal.pone.0329963.t001

The random forest model showed more consistent performance with 78.4% (MIMIC-III), 76.9% (eICU), and 79.1% (PhysioNet) accuracy (Table 2). Its ensemble nature provided better generalization across datasets compared to logistic regression, with only 2.2% maximum variation versus 7.5% for logistic regression. The model’s feature importance analysis revealed consistent reliance on similar ECG morphological features across all datasets, explaining its robustness.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Random forest performance across datasets.

https://doi.org/10.1371/journal.pone.0329963.t002

The CNN model achieved the highest overall performance with 82.7% (MIMIC-III), 80.3% (eICU), and 84.5% (PhysioNet) accuracy (Table 3). While showing better absolute performance than other baselines, it exhibited slightly larger variation (4.2%) than random forest, suggesting that its learned features, while powerful, may be more sensitive to dataset-specific characteristics. The model’s superior performance on PhysioNet likely benefits from the dataset’s cleaner ECG waveforms and standardized recording conditions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. CNN performance across datasets.

https://doi.org/10.1371/journal.pone.0329963.t003

The robustness evaluation (by using accuracy indicator) demonstrates our model’s consistent performance under various challenging conditions (see Table 4). Under clean data conditions, the model achieves 87.2%, 83.5%, and 89.7% accuracy on MIMIC-III, eICU, and PhysioNet respectively. When tested with noisy data, the performance only decreases by 2.1-3.2 percentage points across datasets, showing good noise immunity. The model maintains reasonable accuracy (78.9-86.2%) even with 30% missing data, demonstrating effective handling of incomplete records. Against adversarial samples, the accuracy drops by 4.8-6.4 percentage points from baseline, suggesting the architecture provides meaningful protection against malicious perturbations. Notably, the performance ranking remains consistent across all test conditions, indicating the robustness characteristics are preserved regardless of data modality. These results validate our model’s reliability for real-world clinical deployment where data quality issues are common.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Robustness evaluation of our model across datasets.

https://doi.org/10.1371/journal.pone.0329963.t004

Comparative analysis reveals fundamental differences in how each baseline handles dataset variations (see Fig 6). Logistic regression’s linear decision boundary struggles with non-linear patterns that vary across hospitals, evidenced by its 7.5% performance range. The random forest’s multiple decision trees provide inherent variance reduction, explaining its more stable performance (2.2% range) through ensemble averaging. The CNN’s hierarchical feature learning provides the best overall performance but shows interesting sensitivity patterns. While its 84.5% PhysioNet accuracy demonstrates exceptional pattern recognition on clean data, the 4.2% drop to MIMIC-III suggests challenges in learning invariant representations from noisier clinical recordings. This aligns with known CNN behaviors where performance depends on training data quality and consistency. Dataset-specific analysis reveals important insights. All models performed worst on eICU data, likely due to its multicenter nature introducing greater variability. The random forest showed the smallest eICU performance drop (1.5% below MIMIC-III versus logistic regression’s 4.2% and CNN’s 2.4%), demonstrating its particular suitability for heterogeneous data environments. The PhysioNet results showcase an opposite trend, with all models performing best on this standardized dataset. The CNN’s 84.5% accuracy here highlights how deep learning excels when trained on consistent, high-quality waveforms. Interestingly, even logistic regression achieved respectable 75.6% accuracy on PhysioNet, suggesting that simpler models can perform adequately when data distributions are favorable.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Comparative analysis reveals fundamental differences in how each baseline handles dataset variations.

https://doi.org/10.1371/journal.pone.0329963.g006

These experiments collectively demonstrate that model robustness depends fundamentally on their architectural properties. Logistic regression’s simplicity makes it vulnerable to distribution shifts, while random forest’s ensemble approach provides natural robustness. The CNN, while powerful, requires careful consideration of training data diversity to achieve optimal generalization across clinical environments.

4.4.2 Temporal generalization.

For temporal generalization analysis, we partition UK Biobank data into chronological cohorts (2006-2010, 2011-2015, 2016-2020) and test all models’ performance decay over time. This design reveals how each algorithm maintains predictive accuracy as medical practices and recording technologies evolve across different periods.

The logistic regression model exhibited significant performance decay across temporal cohorts, with accuracy dropping from 71.2% (2006-2010) to 68.5% (2011-2015) and 65.3% (2016-2020) as shown in Table 5. This consistent decline reflects the model’s inability to adapt to evolving ECG recording technologies and changing clinical practices. The linear decision boundary learned from earlier data becomes increasingly suboptimal for newer recordings, demonstrating the limitations of static parametric models in longitudinal healthcare applications.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Logistic regression temporal performance.

https://doi.org/10.1371/journal.pone.0329963.t005

Random forest demonstrated better temporal stability with accuracies of 77.6%, 76.2%, and 74.9% across the three periods (Table 6). The ensemble’s inherent variance reduction mechanism helped mitigate performance decay, though a 2.7% drop still occurred. Feature importance analysis revealed shifting patterns - while QRS complex features remained consistently important, newer temporal intervals gained significance in later periods, suggesting the model partially adapted to evolving ECG interpretation practices.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Random forest temporal performance.

https://doi.org/10.1371/journal.pone.0329963.t006

The CNN model showed an interesting temporal pattern with accuracies of 81.3%, 79.8%, and 78.1% (Table 7). While exhibiting the smallest relative decay (3.2%), the absolute performance remained superior to other models. The hierarchical feature learning enabled some adaptation to technological changes, though the fixed architecture ultimately limited complete temporal adaptation. The model maintained strong performance on fundamental cardiac abnormalities while showing greater variability in detecting newer diagnostic categories introduced in later periods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. CNN temporal performance.

https://doi.org/10.1371/journal.pone.0329963.t007

The temporal analysis reveals several important findings about our model’s generalization capabilities (see Table 8). Performance shows a clear upward trend across time periods, with accuracy improving from 82.4% (2006-2010) to 88.2% (2016-2020), while maintaining balanced sensitivity and specificity. The AUROC values demonstrate particularly strong temporal stability, ranging from 0.891 to 0.934 across the 15-year span. This progressive improvement likely reflects both advances in medical recording technologies and our model’s ability to learn from evolving clinical patterns. Notably, the decreasing standard deviations (±1.9% to ±1.2% for accuracy) suggest more consistent performance in recent years, possibly due to better data standardization practices. The 2016-2020 period shows the most robust performance (88.2% accuracy, 0.934 AUROC), indicating successful adaptation to contemporary healthcare data characteristics while maintaining backward compatibility with older records.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Model performance across different temporal periods.

https://doi.org/10.1371/journal.pone.0329963.t008

Comparative temporal analysis reveals fundamental architectural differences in handling concept drift (see Fig 7). Logistic regression’s fixed parametric form showed the steepest 5.9% decay, as its weights optimized for earlier data became increasingly misaligned with later distributions. Random forest’s ensemble structure provided inherent buffering against temporal shifts through its multiple diverse decision trees, though the 2.7% drop indicates even this approach has limitations against sustained temporal evolution. The CNN’s performance patterns suggest convolutional filters learned from earlier data maintained relevance for core cardiac patterns, while struggling with subtle feature variations introduced by newer technologies. The model’s 3.2% decay, though smallest among baselines, highlights a key challenge for deep learning in healthcare - while capable of learning powerful representations, the frozen architecture after training cannot fully adapt to subsequent clinical practice changes. These temporal experiments collectively demonstrate that all models experience performance decay, with architecture determining both the magnitude and nature of temporal degradation. The results emphasize the need for continuous learning approaches in clinical AI systems, as even the best-performing static models degrade measurably over multi-year timescales due to evolving medical technologies and practices.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Comparative temporal analysis reveals fundamental architectural differences in handling concept drift.

https://doi.org/10.1371/journal.pone.0329963.g007

4.4.3 Overfitting problem.

To evaluate whether the proposed model suffers from overfitting, we conducted a training-validation curve analysis across 5 independent runs with different random seeds, monitoring the training and validation F1-scores on the MIMIC-III dataset (see Table 9). The results demonstrate that the model maintains a consistent gap between training and validation performance, indicating robust generalization rather than memorization.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Training vs. validation performance on MIMIC-III dataset.

https://doi.org/10.1371/journal.pone.0329963.t009

4.5 Ablation study

Three critical components were systematically removed or substituted to evaluate their individual contributions: (1) The Transformer module was replaced with a bidirectional LSTM (BiLSTM) featuring equivalent parameter count, maintaining temporal processing capabilities while eliminating self-attention mechanisms. (2) The multi-task learning (MTL) framework was decomposed into separate single-task models with identical architecture per task, removing parameter sharing and joint optimization. (3) Transfer learning initialization was disabled by randomly initializing the text processing module instead of using pretrained ClinicalBERT weights. Additionally, we implemented a conventional CNN+RNN hybrid baseline using 1D convolutional layers for local pattern extraction and LSTM for temporal aggregation, mirroring architectures from prior literature. Each ablated configuration maintained identical training protocols and hyperparameter tuning processes to ensure fair comparison.

The ablation study reveals substantial performance degradation when removing any core component, validating our architectural choices (see Table 10). Replacing the Transformer with BiLSTM caused the most significant F1-score drop (6.2% absolute), demonstrating self-attention’s superiority over sequential processing for capturing long-range dependencies in clinical narratives. The 3.3% F1-score reduction in the single-task configuration confirms MTL’s effectiveness in leveraging inter-task correlations - particularly beneficial given the hierarchical relationships between primary diagnoses and comorbidities. Disabling transfer learning resulted in 4.9% lower accuracy, emphasizing the importance of pretrained medical language representations for overcoming data scarcity. Notably, the baseline CNN+RNN hybrid underperformed our full model by 8.5% in recall, highlighting the limitations of conventional architectures in handling both local clinical patterns and global context simultaneously. Training time analysis (omitted for space) showed the full model required 18% fewer epochs to converge compared to non-pretrained versions, confirming transfer learning’s efficiency benefits. Precision-recall curves exhibited similar patterns, with the full model maintaining superior area-under-curve values across all diagnostic categories. These findings collectively demonstrate that our integrated approach synergistically combines the strengths of Transformer-based context modeling, multi-task knowledge sharing, and pretrained medical representations to achieve state-of-the-art performance in clinical diagnosis classification.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Ablation study results on MIMIC-III dataset.

https://doi.org/10.1371/journal.pone.0329963.t010

5.1 Conclusion

The automated classification of clinical diagnoses in electronic health records (EHRs) addresses critical challenges in modern healthcare, where manual processing of heterogeneous medical data proves inefficient for clinical decision-making and large-scale epidemiological studies. This study proposes a novel deep learning framework integrating three synergistic components: a Transformer-based architecture for contextual understanding of clinical narratives, a multi-task learning (MTL) paradigm for joint optimization of related clinical predictions, and transfer learning initialization using pretrained medical language representations. The Transformer module processes unstructured text through self-attention mechanisms to capture long-range dependencies and medical semantics, while MTL enhances diagnostic accuracy by leveraging inter-task correlations between primary diagnoses and comorbidities. Transfer learning bridges domain gaps through pretrained ClinicalBERT weights fine-tuned on target EHR data, addressing data scarcity while preserving generalized medical knowledge. Experimental evaluation on the MIMIC-III dataset demonstrates superior performance, with the full model achieving 0.892 accuracy and 0.876 F1-score, outperforming ablated configurations by 4.9-8.5% across metrics. The ablation study revealed critical component contributions: replacing the Transformer with BiLSTM caused a 6.2% F1-score decline, disabling MTL reduced accuracy by 3.0%, and random initialization decreased precision by 4.9%, validating the necessity of each architectural innovation. These results confirm that the integrated approach effectively combines hierarchical feature extraction, medical knowledge transfer, and multi-task parameter sharing to overcome limitations of conventional CNN-RNN hybrids in handling both local clinical patterns and global contextual relationships. The framework’s clinical applicability is further strengthened by attention visualization for model interpretability and support for incremental updates with new patient data. This work advances EHR analysis by establishing a robust, scalable foundation for automated diagnostic systems that improves classification accuracy while addressing real-world constraints of data heterogeneity and limited annotated medical records, ultimately supporting more efficient healthcare delivery and data-driven clinical decision-making.

5.2 Outlook

The current study primarily evaluates model performance on structured EHR data from a single healthcare system (MIMIC-III), potentially limiting generalizability to institutions with different documentation practices or unstructured data formats. While the framework demonstrates robustness within the experimental dataset, real-world clinical environments often exhibit greater heterogeneity in data quality, temporal granularity, and coding conventions across organizations. This institutional bias may restrict immediate deployment in multi-center settings without additional adaptation. To address this limitation, future research will implement a federated learning framework that enables collaborative model training across distributed EHR systems while maintaining data privacy through differential privacy mechanisms. We plan to integrate domain adaptation techniques using adversarial training to improve cross-institutional generalization, coupled with automated data harmonization pipelines that standardize heterogeneous clinical variables through ontology alignment. Additionally, the development of synthetic EHR generation capabilities using generative adversarial networks (GANs) will facilitate stress-testing the model against diverse documentation patterns and rare clinical scenarios.

The framework’s current implementation processes clinical narratives as isolated text sequences without explicitly modeling longitudinal patient-provider interactions or evolving diagnostic reasoning processes across multiple care encounters. This temporal simplification overlooks critical patterns in disease progression and treatment response dynamics that unfold over extended clinical timelines. To enhance temporal modeling capabilities, subsequent investigations will develop hierarchical attention mechanisms that simultaneously capture intra-document clinical semantics and inter-visit temporal relationships. The proposed extension incorporates time-aware positional encoding and gated memory units to track diagnostic concept evolution across episodes of care. Furthermore, we plan to integrate causal inference methodologies to distinguish between incident diagnoses and historical conditions, addressing a common challenge in retrospective EHR analysis. Parallel development of interactive visualization tools will enable clinicians to audit the model’s temporal reasoning patterns through animated attention heatmaps spanning entire patient journeys.

While the model demonstrates strong diagnostic classification performance, its clinical utility remains constrained by insufficient integration with external medical knowledge bases and real-time patient monitoring data streams. The current architecture processes EHR data as static snapshots rather than dynamic clinical entities connected to evolving medical knowledge. To bridge this gap, next-phase research will implement a hybrid neuro-symbolic architecture that combines the transformer-based feature extractor with probabilistic medical knowledge graphs. This enhancement will enable continuous incorporation of updated clinical guidelines and drug interaction databases through modular knowledge graph updates. Simultaneously, we are developing streaming data adapters that process real-time vital sign feeds and wearable device data, expanding the model’s diagnostic capability to acute care settings. To ensure safe deployment, these extensions will be coupled with uncertainty quantification modules that estimate prediction confidence levels and alert clinicians when encountering novel symptom patterns or conflicting diagnostic evidence.

This study establishes an innovative deep learning framework that synergistically integrates Transformer architectures, multi-task learning, and transfer learning to achieve state-of-the-art performance in automated clinical diagnosis classification from electronic health records, while addressing critical challenges of data heterogeneity and limited medical annotations through hierarchical feature extraction and cross-domain knowledge transfer.