Dataset and patients’ enrollment

The complete procedure followed in this study is illustrated in Fig 1. This study used two datasets that comprised clinical data from patient groups of Greeks and Americans. Patients with HF, specifically CAD, and ages ranging from 33 to 88 years old (n = 303) were included in both datasets. According to the ASE/EACVI recommendations, these patients were split into 129 HFpEF, 92 HFmEF, and 82 HFrEF groups.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. An illustration of the overall procedure followed in this study.

https://doi.org/10.1371/journal.pone.0295653.g001

Patients from seven cardiology departments in Greece who were engaged in the PRESERVE EF trial contributed to the Greek patient cohort [28]. The enrolment protocol of patients for the PRESERVE EF study(clinicaltrials.govidentifier NCT02124018) was approved by the ethics committee at each of the seven selected cardiology departments at Greece and was endorsed by the Hellenic Society of Cardiology. The Hellenic Society of Cardiology established and maintained a database [29]. Each patient signed a consent form before enrolling in the trial at each cardiology department. Patients needed to meet the following requirements to be eligible for enrolment: (1) having a post-angiographically proven MI at least 40 days after the event or 90 days after any CABG surgeries, if applicable; (2) being revascularized; (3) not being revascularized but lacking evidence of any active ischemia in the previous six months; and (4) completing an effective course of medical treatment.

Additionally, any patient who had a secondary prevention indication for the placement of an implantable cardioverter defibrillator (ICD), a permanent pacemaker, persistent, long-standing persistent, or permanent atrial fibrillation, any neurological symptoms of syncope or pre-syncope within the previous six months, or the presence of any systemic illnesses like liver failure, renal diseases, rheumatic diseases, thyroid dysfunction, or cancer was excluded from the study. For all patients, the American Society of Echocardiography’s recommendations are followed when doing echocardiographic tests, and a GE Healthcare GETEMED CardioDay Holter system (recorder CardioMem CM4000 and software CardioDay v. 2.4, GE Healthcare, Fairfield, CT, USA) is used.

The Intercity Digital Electrocardiography (ECG) Alliance (IDEAL) study archives at the University of Rochester Medical Centre Telemetric and Holter ECG Warehouse (THEW) were used to identify the American patient cohort [30]. The database enrolment protocol complied with the Declaration of Helsinki and Title 45, U.S. Code of Federal Regulations, Part 46, Protection of Human Subjects (revised: November 13, 2001-effective: December 13, 2001). Additionally, the IDEAL protocol was authorised by the University of Rochester’s research subject review board [31]. Before taking part in the trial, each patient gave their written consent.

The following requirements had to be met to qualify for enrolment in the IDEAL study: (1) having evidence of either a prior MI or exercise-induced ischemia; (2) being in the stable phase of ischemic heart disease at least two months after the last event; (3) not having been given a congenital heart failure diagnosis; and (4) having sinus rhythm. Additionally, any patients with dilated cardiomyopathy—defined as having a left ventricular diameter (LVD) > 60 mm and an ejection fraction (EF) 40%—congenital heart failure (CHF), coronary artery bypass grafting (CABG) surgery, non-sinus rhythm, and any cerebral, severely hepatic, or malignant diseases—were disqualified from the trial. All patients underwent an echocardiography examination to determine their LVEF levels. In addition, a 24-hour ECG test was performed on each patient utilising the pseudo-orthogonal lead configurations (X, Y, and Z).

From eligible individuals, a 24-hour ECG Holter recording was obtained and sampled with 200 Hz. In this study, patients with missing recordings of an hour or more were excluded. Thus, the dataset included only 229 patients; split into 105 HFpEF, 60 HFmEF, and 64 HFrEF according to the ASE/EACVI guidelines. More details on the selected dataset are provided in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Clinical characteristics of the heart failure patients based on their LVEF categories.

https://doi.org/10.1371/journal.pone.0295653.t001

Pre-processing and feature extraction

The 24-hour circadian ECG was fixed to start at 12:00 am for all the patients using a cosinor ftting analysis [32]. The SDROM-ADF filter [33] was used to filter each hour of the Holter ECG data for noise. Time- and frequency-domain features were extracted from the denoised ECGs, as explained next. The P-QRS-T components of the ECG wave have been accurately analysed using a variety of intricate approaches. A typical ECG is composed of the P wave, QRS complex, and T wave. Electric currents created by the ventricles’ depolarization before to contraction, through the ventricular myocardium’s depolarization extension, result in the QRS complex. The P wave, on the other hand, is created by electrical currents that the atria depolarize before contracting. The related definition and description of the ECG features are provided in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Definitions of ECG features.

https://doi.org/10.1371/journal.pone.0295653.t002

The QT interval varies significantly according to gender, age, heart rate, and drug use. As a result, the QT interval can be calculated using Bazett’s method while taking variations into account [34]. RR refers for the R-R interval, while QTc (QTc = QT/RR) stands for the QT interval corrected. The Slope Intersect (SI) approach was used in this study’s automatic QT interval detection [35]. The Pan Tompkins ECG QRS detector was applied in Matlab to find the QRS complex [36]. Furthermore, TP, PR, and ST-T were extracted using the ECG Matlab Toolbox [37]. The power spectrogram’s initial moment was used to estimate the instantaneous frequency of the signal using 255-time frames. In addition, the spectral entropy of the power spectrogram was calculated to gauge how flat and spiky a signal’s spectrum is.

Machine learning and training settings

To provide an effective regression process, it is crucial to have the ability to train and estimate LVEF levels in HF patients using machine learning algorithms. In this study, four models were used, including Support Vector Machine (SVM), Extreme Gradient Boosting (XGBOOST), Gaussian Process Regression (GPR), and Decision Tree Regression (TREE), to assess the effectiveness of various machine learning methods.

1) SVM

A standard and adaptable machine learning method is often used for classification problems. However, Support Vector Regression (SVR), a variation of SVM, can be utilised for regression tasks. It manages interactions between the input characteristics and the target variable that are both linear and nonlinear. It captures complicated patterns and nonlinearity by mapping the data into a higher-dimensional space using a kernel function [38]. SVR is less sensitive to outliers compared to some other regression techniques.

Finding a "tube" or margin around the best-fitting function is the fundamental tenet of SVR; data points inside the tube are regarded as well-fitted, while those outside the tube are penalised. Using a margin around the regression line allows for some error tolerance leading to robustness to outliers.

SVM regression has only a few tuning parameters, such as the choice of kernel and the regularization parameter, making it easy to use and implement [39]. The two main hyperparameters to tune are the kernel and the regularization parameter;

1. Kernel Selection: SVRs can capture nonlinear relationships since it uses a kernel function to translate the input into a higher-dimensional space. The SVR model’s performance can be considerably impacted by kernel selection. Typical kernels include linear kernel, polynomial kernel, and radial basis function (RBF).

2. Regularization Parameter (C): The regularization parameter (C) determines the trade-off between maximizing the margin and minimizing the training error. A smaller C allows a larger margin but may lead to more training errors, while a larger C leads to fewer errors but may result in a smaller margin.

In this work, the kernel types are selected to be Linear, Polynomial, and RBF, and we defined the C Values in the range of: [0.1, 1, 10,14].

2) XGBOOST

It is an ensemble learning technique that builds a powerful predictive model by combining several weak predictive models, often decision trees, commonly used to predict heart diseases [40, 41]. The XGBoost approach is a development of gradient boosting, adding decision trees to the model iteratively while each tree tries to fix the mistakes caused by the preceding one. However, XGBoost incorporates several enhancements to improve performance, speed, and generalization; to reduce overfitting and increase model generalisation, this model incorporates L1 (Lasso) and L2 (Ridge) regularisation terms. In addition, it employs a novel method to accelerate convergence while improving model accuracy by optimising the loss function during tree construction.

XGBoost provides a measure of feature importance, which helps find the most noteworthy features for making predictions. It can automatically handle missing data in the input features, making it more robust in real-world datasets. It also supports k-fold cross-validation, aiding in assessing model performance and hyperparameter tuning. Although many hyper-parameters are included in XGBoost, in this work, the hyper-parameters adopted for tuning include:

1) Learning Rate: The step size shrinkage used in the update to prevent overfitting. Lower values make the boosting process more conservative.

2) Maximum Depth: The maximum depth of a tree. Deeper trees can capture more complex patterns but may lead to overfitting.

2) Gamma: The minimum loss reduction required to partition a leaf node further. It acts as a regularization term, controlling the complexity of the trees and preventing overfitting.

The learning rate (eta) is set to be in the range of: [0.1, 0.2], the maximum depth values are: [3, 5], and the selected gamma values are: [0, 0.01]. We use the default settings in Matlab for the rest of the other hyperparameters.

3) GPR

A potent non-parametric probabilistic regression method for modelling and forecasting continuous data is called GPR. It is based on Gaussian processes [42], which are groups of randomly distributed variables with a common distribution. In GPR, the objective is to develop a distribution over potential functions that could explain the data to represent the link between input data and output values. In contrast to parametric regression models, which require learning specific model parameters, GPR predicts a distribution across functions consistent with the training data. A mean function and a covariance function, referred to as a kernel function, represent this distribution. The fundamental assumption in GPR is that nearby data points in the input space should have similar output values. The covariance function captures the similarity between data points and plays a crucial role in shaping the posterior distribution over functions. The choice of covariance function determines the smoothness and complexity of the resulting regression model.

GP has also been used in cardiovascular modelling [43–45] as a machine learning tool for various regression tasks. It is particularly useful when dealing with small to moderate-sized datasets, where it can capture complex patterns and provide uncertainty estimates in predictions. The hyperparameters in GPR can be broadly categorized into two types: kernel hyperparameters and regularization hyperparameters.

1) Kernel Function: The choice of the kernel function, also known as the covariance function or radial basis function, determines how the covariance between data points is computed. Common kernels include the radial basis function (RBF or squared exponential), Matérn, polynomial, and more.

2) Regularization Hyperparameters (Alpha): In GPR implementations, regularization hyperparameters may be available to prevent overfitting and improve model generalization. These hyperparameters control the trade-off between fitting the training data and simplifying the model.

The kernel functions in this work are set to Rational Quadratic and Matern52. In addition, the alpha range is selected to be: [0.01, 0.1, 1].

4) TREE

Decision Tree Regression is a supervised machine-learning technique used for regression tasks. Decision tree regression predicts continuous numerical values unlike decision trees for classification, which output discrete class labels. The core idea behind decision tree regression is to partition the feature space into regions and assign a constant value to each region, which predicts any input falling into that region.

The algorithm begins by partitioning the input data into subsets based on the values of the input features. It selects the feature and the corresponding threshold that best splits the data, aiming to minimize the variance (or other suitable loss function) of the target values within each partition. The selection is based on the chosen feature and threshold at each internal node of the tree (the decision node) [46]. The algorithm determines if the feature value of the input data point exceeds or falls short of the threshold. Depending on the result, the algorithm moves to the left or right child node.

Each subset is subjected to data partitioning and decision-making, resulting in a hierarchical structure of decision and leaf nodes. The target variable is given a prediction value when the algorithm reaches a leaf node (terminal node). The mean or median of the goal values for that location serves as this prediction value. Based on the feature values of the input, the algorithm moves through the tree from the root node to a particular leaf node to forecast a new input sample. The model’s output for that input is the predicted value at the leaf node.

The interpretability of Decision Tree Regression is its main benefit because the output tree can be inspected and understood immediately. It is robust against outliers and can handle numerical and categorical data, making it widely used in heart disease modelling and prediction [47]. However, obtaining the best performance and reliable generalisation requires rigorous hyperparameter altering and countermeasures against overfitting. In the context of tuning decision tree hyperparameters, several key parameters are considered, focusing on controlling the tree’s complexity and potential for overfitting. These hyperparameters include but are not limited to maximum depth, minimum samples split, minimum samples leaf, and maximum leaf nodes. During the hyperparameter tuning process, the Min Leaf Sizes are set to: [2, 4, 6, 8, 10], and the Surrogate Options are chosen from the set: [off, on]

The best SVR hyperparameter combination was C = 14 and a linear kernel type. A high C value indicates that the SVR model has a low tolerance for errors in the training data. In other words, the model aims to fit the training data as closely as possible, even if it means allowing more margin violations. Using a linear kernel, the SVR model attempts to draw a hyperplane in the feature space that best fits the data points while maximizing the margin between the positive and negative support vectors. This simplicity and interpretability of the linear model make it advantageous in scenarios where the relationship between the input features and the target variable can be adequately approximated by a linear function [48]. For XGBoost, the best performance was achieved with a Learning Rate of 0.1, a Maximum Depth of 3, and a Gamma value of 0.01. A moderate Learning Rate, a shallow Maximum Depth, and a conservative Gamma value indicate a well-balanced XGBoost model. It leverages a cautious approach in building trees to avoid overfitting while still allowing the model to capture relevant data patterns in the data effectively [49]. This hyperparameter combination will likely produce a robust and accurate XGBoost model, providing reliable estimates of LVEF levels in heart failure patients across distinct categories (HFpEF, HFmEF, and HFrEF).

In the case of GPR, the best performance was observed using the kernel function of rational quadratic and an alpha of 0.1. The rational quadratic kernel combines the characteristics of both the squared exponential kernel and the Matérn kernel. Matérn kernel is a versatile covariance function that can adapt to various levels of smoothness in data and provides a balance between the squared exponential kernel and the Gaussian kernel. The rational quadratic kernel is more flexible than the squared exponential kernel, allowing it to capture various patterns in the data, including short-range and long-range correlations. An alpha value of 0.1 indicates that the model is giving more importance to the squared exponential term, making it more sensitive to large-scale variations [42]. This choice could be appropriate if the underlying data has smooth patterns and long-range dependencies. Finally, for the Decision Tree Regression model, the best hyperparameter combination comprised a Min Leaf Size of 4 and ’off’ for the Surrogate Option. Choosing a minimum leaf size of 4 strikes a balance between capturing some of the complexity in the data while preventing excessive overfitting [50]. This choice suggests that the data might have sufficient complexity to warrant more splits but still benefits from regularization to control overfitting. Our results indicate that each regression model exhibited exceptional performance when tuned with its best hyperparameter combination.

Training and testing configuration

This study introduces a nested loops framework (Fig 1), which synergistically combines leave-one-out cross-validation (LOOCV) as the outer loop and 5-fold cross-validation as the inner loop. This carefully designed approach is advocated for its ability to offer a robust and comprehensive methodology for model evaluation and hyperparameter tuning. In the outer loop, each data point is iteratively withheld as a validation set while the model is trained on the remaining data instances. This process is repeated for all data points, allowing comprehensive evaluation across various training and validation scenarios to obtain reliable performance estimates. Within the outer loop, the inner loop performs 5-fold cross-validation, partitioning the data into five subsets. The model is trained on four subsets while the remaining one is validated. This cycle is repeated five times, robustly assessing the model’s performance. The average performance metrics from these iterations are then utilized to evaluate the model’s effectiveness more comprehensively.

To optimize the model’s hyperparameters efficiently, the nested loop approach incorporates grid search technology. Grid search systematically explores a predefined hyperparameter grid, covering various combinations of hyperparameter values. At each iteration of the nested loops, the model’s performance is evaluated based on different hyperparameter configurations. This allows for an exhaustive search for the optimal set of hyperparameters that maximizes the model’s accuracy and generalization capabilities.

Using nested loops with LOOCV, 5-fold cross-validation, and grid search technique provides a rigorous and reliable framework for regression tasks in machine learning. It enables meticulous hyperparameter tuning, ensures robust model evaluation, and ultimately leads to the selection of the best-performing regression model with enhanced predictive capabilities of the LVEF estimation and increased applicability to heart failure diagnosis.

The evaluation of the regression model’s performance using key metrics, including the average Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and correlation analysis, were computed. Furthermore, the estimation process was visualized through Bland-Altman plots and correlation plots [48, 51], comprehensively analysing the model’s predictive capabilities.

Furthermore, the feature importance metric quantifies the impact of each feature on reducing the impurity (e.g., RMSE) in the model. The more a feature contributes to reducing the impurity during tree construction, the more important it is considered. This importance is typically computed as the total reduction in impurity achieved by each feature over all nodes in the tree. In nested cross-validation (LOOCV and 5-fold cross-validation), the feature importance is calculated iteratively over multiple folds or model training and evaluation iterations. This ensures robustness in estimating feature importance and helps mitigate overfitting issues.

Clinical relevance and ECG features importance

ECG features obtained through ambulatory Holter monitoring have been established as valuable predictors of total mortality and the progression of heart failure. This continuous monitoring technique offers unique insights into a patient’s cardiac health, allowing for a comprehensive evaluation of heart rhythm and electrical activity over an extended period. Even when accounting for other known risk factors, such as age, gender, and comorbidities, the information derived from Holter monitoring provides additional and valuable prognostic information. However, estimating LVEF in heart failure patients hourly using ECG features can offer valuable insights into the temporal patterns and fluctuations of cardiac function. This hourly analysis can enhance our understanding of how LVEF levels vary throughout the day, potentially identifying when LVEF estimation is more accurate and best correlated with clinical variations or when heart function is at its optimal state. This understanding of temporal variation may be crucial for adjusting treatment strategies or interventions based on the patient’s circadian rhythm and physiological fluctuations. This work demonstrates superior performance during the evening (22–23) and early morning hours (12 am-1 and 8–9) for estimating LVEF levels. Additionally, these hourly intervals align with previously reported high-risk periods for increased mortality and myocardial infarcts observed in the early morning and evening hours [55–57].

Furthermore, using the most effective regression model, we evaluated feature importance during this study. Our findings revealed that the most crucial features contributing to LVEF prediction are QTc, QRS, ST, and TP in descending order of importance. These specific ECG parameters significantly influenced the target variable, supplying valuable insights in characterizing cardiac function and highlighting their potential as important clinical indicators in assessing heart failure patients.

The prominence of QTc as the most important feature can be attributed to several key factors [58–61], QTc interval stands for the duration of ventricular depolarization and repolarization, encompassing the electrical conduction time within the heart. In HF patients, electrical conduction abnormalities are common due to ventricular remodelling and changes in ion channel kinetics. Prolonged QTc intervals may show delayed repolarization, which can be associated with impaired ventricle (which involves alterations in ventricular size, shape, and function) and reduced LVEF. HF patients are at increased risk of arrhythmias, and QTc prolongation is a known risk factor for life-threatening ventricular arrhythmias, abnormal QTc intervals are indicative of arrhythmogenic substrate and can influence LVEF by potentially triggering life-threatening arrhythmias that may affect cardiac function. In addition, QTc interval is influenced by autonomic nervous system activity, specifically parasympathetic and sympathetic inputs to the heart. HF patients often have altered autonomic regulation, and QTc prolongation may be linked to autonomic dysfunction, which can affect cardiac function and LVEF. Some medications commonly used in HF management can affect the QTc interval. The QTc-prolonging effects of certain drugs may contribute to the importance of this feature in the model, as medication use is an essential consideration in LVEF estimation.

Several studies have shown that QRS duration is a prognostic marker in HF patients [62–64]. The QRS complex is the ventricles’ electrical activation and later mechanical contraction. In HF patients, mechanical desynchrony is a common phenomenon, where different regions of the ventricles contract uncoordinated. QRS prolongation can indicate bundle branch blocks (BBBs). These conduction abnormalities can affect the ventricular function and contribute to reduced LVEF [65]. In addition, for patients undergoing cardiac resynchronization therapy (CRT), QRS duration is a critical parameter for determining eligibility for CRT implantation. CRT effectively improves mechanical synchrony in HF patients with electrical desynchrony and prolonged QRS duration. By considering QRS duration in LVEF prediction, clinicians can gain insights into potential CRT candidacy and tailor treatment options for better patient outcomes.

Moreover, the STT interval is closely related to T-wave morphology. T-wave abnormalities, such as inversion or flattening, are commonly seen in heart failure patients [66–68]. These T-wave changes can indicate valuable information about myocardial ischemia or injury and can contribute to LVEF prediction accuracy. Variations in the TP interval may show abnormalities in repolarization, which can be linked to impaired cardiac function in HF patients. Those alterations have been associated with an increased risk of arrhythmias, such as ventricular tachycardia and ventricular fibrillation. Several studies have reported a correlation between abnormal TP intervals and reduced LVEF in HF patients. Prolonged TP intervals have been associated with impaired ventricular function, while shortened TP intervals may show myocardial remodelling and potential deterioration of cardiac performance.

The prioritization of these specific features based on their impact on the LVEF estimation reinforces the model’s ability to discern critical patterns and relationships among various ECG parameters. This knowledge is instrumental in enhancing our understanding of the underlying physiological mechanisms that govern cardiac health and dysfunction.

The proposed technology, which leverages regression models based on circadian ECG features, has the potential to serve as a valuable tool for early HF diagnosis and risk assessment. A more detailed explanation of its practical applications and requirements for implementation:

1. Data collection and Integration:

• Data sources: Access to ECG data is a fundamental requirement. This data can be collected from various sources, including hospitals, clinics, and ambulatory ECG monitors.

• EHR Integration: Integration with electronic health records (EHR) systems is essential for streamlined data access and patient history analysis. This enables healthcare providers to have a comprehensive view of the patient’s health status.

2. Machine learning infrastructure:

• Training and validation: Rigorous model training and validation, using large and diverse datasets, are critical to ensure the reliability and accuracy of the technology.

3. Clinical validation:

• Collaboration with healthcare professionals: Close collaboration with cardiologists and other healthcare professionals is crucial for the clinical validation of the technology. Their expertise ensures that the technology aligns with established medical practices and standards.

• Clinical trials: Conducting clinical trials to assess the technology’s performance in real-world healthcare settings is a key component. This helps establish its clinical efficacy and safety.

4. Regulatory compliance:

• Quality assurance: Implementing quality assurance protocols and adhering to Good Clinical Practice (GCP) and Good Manufacturing Practice (GMP) standards are important for maintaining high-quality healthcare technology.

5. Privacy and security:

• Data protection: Safeguarding patient data is paramount. Adherence to data protection regulations, such as Health Insurance Portability and Accountability Act (HIPAA) in the United States or General Data Protection Regulation (GDPR) in the European Union, is mandatory.

• Security measures: Implementing robust security measures to protect patient data from breaches and unauthorized access is essential.

6. Clinical workflow:

• Integration into ECG testing: Our technology can be seamlessly integrated into the clinical workflow as a supplementary tool during routine ECG testing.

• Automated LVEF estimation: When an ECG is performed, the technology can automatically generate an LVEF estimation, which is then made available to the attending physician.

• Physician assessment: Physicians can use this information to identify patients at risk for HF and initiate further diagnostic tests or interventions as necessary.

In conclusion, implementation in medical practice involves adherence to specific protocols related to data collection, machine learning infrastructure, clinical validation, regulatory compliance, data privacy, and a secure clinical workflow. These protocols are vital to ensure the accurate and ethical application of this technology for the benefit of patients and healthcare providers.