Fig 1.
Pipeline for dataset preparation for chronic kidney disease forecasting utilizing MIMIC-IV electronic health records.
The workflow delineates: (1) patient identification via ICD-9/10 diagnosis codes, (2) extraction of time-stamped laboratory measurements (creatinine, eGFR, potassium, BUN), (3) daily aggregation of all measurements per patient, (4) three-tiered missing value management (forward-filling, linear interpolation, and selective deletion), and (5) formulation of longitudinal sequences with continuous regression targets for prospective biomarker prediction.
Fig 2.
Architecture of the Eigen-Guided Transformer showing the complete data-driven design pipeline.
Fig 3.
CKD dataset feature correlation matrix eigenvalue analysis and cumulative explained variance ratio.
The cumulative explained variance ratio is shown by the red curve, while the blue bars show eigenvalues in descending order. The variance threshold is the horizontal dashed line at 90%. 11 major components (corresponding to 11 attention heads) must capture at least 90% of the dataset’s variance, giving a data-driven criterion for attention head selection rather than random setup.
Table 1.
Baseline Hyperparameter Configurations.
Fig 4.
Validated architecture depth search scores.
(a) Validation MSE vs. transformer block count: 2 blocks yield minimum error (MSE = 0.078), while more blocks lead to overfitting. At block 4, marginal improvement (ΔLoss) falls below the threshold (τ = 0.0001), resulting in early stopping. When attention heads are selected by eigenvalue analysis, 2 transformer blocks are the ideal architecture depth for CKD forecasting.
Fig 5.
Training and validation loss curves for the Eigen-Guided Transformer over 90 epochs.
Table 2.
Ablation Study – Component-wise Contribution Analysis based on MIMIC-IV dataset.
Fig 6.
Performance comparison on the test set Original and Eigen-Guided Transformer.
The results, categorized by dataset and metric, indicate that Eigen-guided optimization markedly decreases both MSE and MAE. In the MIMIC-IV cohort, MSE decreased from 0.21 to 0.089, representing a 57.6% reduction, but in eICU, MAE improved from 0.13 to 0.0254. This demonstrates that data-driven structural initialization offers a superior inductive bias for multi-institutional clinical forecasting.
Fig 7.
Performance comparison across MIMIC-IV and eICU datasets.
The Eigen-Guided Transformer demonstrates superior predictive accuracy in both MIMIC-IV (MSE: 0.089, MAE: 0.132) and eICU (MSE: 0.0117, MAE: 0.0254) relative to state-of-the-art models like TFT (MIMIC MSE: 0.10). Lower error values indicate enhanced precision and successful cross-institutional validation.
Table 3.
Uncertainty Quantification – Calibration Analysis based on MIMIC-IV dataset.
Fig 8.
(Left) Calibration curve showing expected vs. observed coverage.
(Right) 90% prediction intervals for representative test samples based on MIMIC-IV dataset.
Table 4.
Subgroup Performance Analysis based on MIMIC-IV dataset.
Fig 9.
Subgroup performance analysis by age, sex, and ethnicity based on MIMIC-IV dataset.
Table 5.
Sensitivity Analysis over Variance Thresholds based on MIMIC-IV dataset.
Fig 10.
Attention-Based Interpretability Analysis.
(a) Temporal attention patterns and per-head attention weights at the last query step, (b) Feature loadings on top principal components used for attention head initialization.