Predicting blood glucose levels.

In recent years, some studies had tried to estimate upcoming values of BG levels, using ML techniques. Accurate estimators for prediction, even of short horizons, may allow reducing the risks of serious life-threatening complications, including stroke and heart disease [21]. Pappada et al. [22] utilized a Neural Network-based model to predict BG levels. The model integrated manually recorded data of food intake, insulin and physical activity, in order to increase the precision of the BG level estimation. While this model accurately predicted hyperglycemic episodes, it failed to predict hypoglycemic episodes, a possible reason for that is the lesser occurrence of hypoglycemic events within the training data. Another model for BG levels prediction, as described by Plis et al. [23], applied a Kalman filter to CGM data to generate features, which were then utilized by a Support Vector Regression (SVR) algorithm. The algorithm depended on past and current values of insulin, carbohydrates intake and BG levels to estimate future values. The root mean square error (RMSE) values obtained were 22.6 mg/dL for a prediction horizon (PH) of 30 minutes and 35.8 mg/dL for a prediction horizon of 60 minutes. In a similar study [24], a SVR method was also used, but unlike that of Plis, it was only based on CGM data of twelve Type 1 Diabetes (T1D) patients and the results were improved by adding Differential Evolution (DE) algorithms. The results were better than Plis, with RMSE values of 10.78 and 12.95 mg/dL for PH of 30 and 60 minutes, respectively. With the development of Artificial Neural Networks (ANNs), researchers started to use them in the BG prediction domain. Pérez-Gandía et al. [25] relied on CGM data of thirteen T1D patients and proposed a method based on ANN. Their method did not improve the results of those which used classic ML models, and achieved RMSE values of 17.45 and 25.08 mg/dL for PH of 30 and 45 minutes, respectively. In other studies, [26, 27] the approach for predicting BG levels included deep learning methods using Recurrent Neural Network (RNN) based on Long Short-Term Memory (LSTM). The authors shows that the using of LSTM models, which suitable for sequential data such as time series, achieved better results than previous models. While many prior studies have focused on crafting personalized models, a general model would be more flexible and helpful in real clinical situations, especially when patient data availability is limited. In the study by Cobelli et al. [4], patients’ recordings of insulin dosages and meals were utilized by a multi-layer convolutional Neural Network (NN) to learn a generalized model. In [28], a LSTM model is trained using multi-patient BG profiles. Bosoni et al. [29] presented a multi-patient and multivariate deep learning approach, based on LSTM that used flash glucose monitoring (FGM), heart rate (HR), sleep, and physical activity. This approach exhibited superior performance compared to models relying solely on blood glucose (BG) levels.

Time-series forecasting.

Forecasting stands out as one of the most popular and useful methods in time series analysis. This method aims to predict future values of a variable by leveraging its past values, sometimes supplemented with additional variables. The two primary basic techniques of forecasting are Auto Regression and Moving Average. Auto-regression is based on regression models, where the x variable, or explaining variable is the time.

Moving Average is a forecasting technique that is usually used to identify the direction of a trend. It’s based on the average of a certain number of preceding values. Additionally, there exists a more advanced variant known as the weighted moving average (WMA), which assigns specific weights to each of the preceding time point values. However, determination of the weights can be challenging. This task is often accomplished through optimization techniques such as minimizing the average error of the model. Over the past decade, there has been a growing interest in time series forecasting (TSF) [30, 31], Which has led to the development of new methods that incorporate the use of models from the field of deep learning. The most popular DL models are neural networks based on RNN-LSTM [32] and convolutional neural networks (CNNs). Tan et al. [33] studied different methods for time series forecasting tasks and compared existing solutions and adaptions of Time Series Classification (TSC) algorithms. They compared the different algorithms on a novel archive of 19 datasets. The algorithms they compared included classic ML and ensemble models and DL models such as LSTM and CNN (Resnet [34], Inception [35], FCN [36]) and a novel fast and accurate model named Rocket [37]. They showed that the state-of-the-art TSC algorithm Rocket achieves the highest overall accuracy when employed for regression, outperforming other models. Another notable TSF model is Prophet [38], an open-source tool released by Facebook’s Core Data Science team. Prophet utilizes an additive model that decomposes the time series trends into several components such as seasonality (yearly, weekly, and daily), and also holiday effects. The method’s main advantages include its robustness to shifts in the trend, its ability to handle missing data effectively, and its good handling of outliers. Prophet is particularly well-suited for time series data with strong seasonal effects and multiple seasons of historical data.

Smartwatch’s sensors’ measurements.

To collect physiological measurements of a user, such as hand movements, heart rate, etc., a smartwatch sensors are used. In our study we used the “Mobvoi TicWatch Pro 2020” smartwatch, for which a dedicated Wear OS-based application called “TicWatchSensorReader” was developed, which enables us to extract the relevant sensors. Wear OS, previously known as Android Wear, is a Google’s Android operating system specifically tailored for smartwatches and various other wearable devices. The Wear OS applications are run directly on a smartwatch, given access to the smartwatch’s hardware sensors. For our study, an Android Foreground Service which runs in the background, as long as the smartwatch is kept on hand, was developed. When new data is observed in any of the sensors, the service stores it in the smartwatch’s internal storage, so the data can be accessed directly in the smartwatch or be exported to a smartphone or a computer. Since the service simply documents a measurement when there is a change, the measurements are measured irregularly—in the next section we will describe how we resolved this issue. The measured data is stored on the watch’s memory and can be uploaded via Bluetooth to a smartphone or via Wi-Fi to the cloud, sent by an e-mail, or to a dedicated server. The following four categories of sensors were sampled: 1. Body Sensor: Heart Rate (HR) in units of beats per minute. 2. Activity Sensor: Step Detector which triggers an event each time a step is taken by the user. 3. Motion Sensors: a. Accelerometer: measures the acceleration applied to the device’s x, y, and z axes, including the force of gravity. b. Gyroscope: measures the rate of rotation in rad/s around a device’s x, y, and z axes of a device. c. Gravity: The gravity sensor provides a three-dimensional vector, which indicates both the direction and magnitude of the phone’s orientation relative to gravity. This sensor is commonly utilized to determine the device’s relative orientation in space. 4. Position Sensor: A geomagnetic field sensor that enables the monitoring of changes in the Earth’s magnetic field.

CGM based BG measurements.

In order to measure the BG levels, we used a “Dexcom G6” CGM device that measures glucose levels continuously and at a constant rate of one measurement every 5 minutes (which is the highest frequency in the market) for about 10 days. The measurements history is stored on the device memory, containing the last three hours. In order to save the entire measurements history, the user is required to download the “Dexcom” application to their smartphone and to pair the smartphone with the CGM device via Bluetooth. The measurements are transmitted automatically to the smartphone using Bluetooth and is also uploaded to DEXCOM’s dedicated server from the smartphone. The measurements history can be viewed on their app or server. The information on the server can be accessed and exported through the DEXCOM’s web interface, which allows it to be further downloaded.

Smartphone user activities.

To collect the activities performed by the user (eating, drinking, etc.), we developed an Android-based smartphone application, called “GlucoDataSaver”, that allows the user to manually record his activities. The application includes screens that allow the user to choose activity by type and to save it as a record which includes the start time and end time. For example, when the user wants to document eating activity, he opens the application and presses the “Eating” button. At that moment, the start time and the activity type are saved, and the user is shown a “Eating” screen mode which contains the duration time and “Finish” button. Once the user has finished to eat, he presses the “Finish” button the application return to the main screen and the activity is saved. In addition, the application also allows the user to document activity after the fact, meaning the user can manually document the activity in case he forgot to document it. Fig 2 presents the two main screens of the application. The data is stored on the smartphone memory and in addition, the data can be sent via Wi-Fi to a cloud, e-mail, or dedicated server.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. GlucoDataServer app.

Screenshots of the android application “GlucoDataSaver”. The right screenshot represents the main screen, and the left screenshot represents the config and “documenting after the fact” screen.

https://doi.org/10.1371/journal.pone.0307136.g002

Predicting blood glucose values a prediction time ahead.

Using the data from the previous step, a sliding observation time window was used, based on which data a value of the blood glucose was predicted a prediction time ahead, as illustrated in Fig 4. This mechanism eliminates time dependence and creates independent continuous windows. It is very important since the smartwatch’s and phone’s sensors were monitored only during the day.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Sliding window structure.

Illustration of the multiple temporal variables along time, in which each horizontal sequence of dots represents a temporal variable, and their tones illustrates different values. Based on all the temporal variables in the observation time period, a value in a prediction time period ahead is predicted.

https://doi.org/10.1371/journal.pone.0307136.g004

Prediction modelling.

We describe here briefly the prediction models that were used as part of the method for the blood glucose forecasting.

Rocket.

Rocket [37] is a time series classification and regression method. Rocket’s primary advantage is achievement of high accuracy with low runtime. It transforms the data using random convolutional kernels, such as those used in a CNN, by projecting it into a higher dimensional space. These kernels were shown to capture relevant features such as shape, frequency or variance without requiring handcrafted features. The transformed features are subsequently inputted into a linear classifier, such as Ridge Regression. This method operates at high speed since it bypasses the need to learn kernels, unlike deep learning methods.

Prophet.

Additional model we used as a baseline in our research is the Prophet [38]. Prophet is an open-source software released by Facebook’s Core Data Science team. This forecasting approach uses an additive model to analyze time series data, breaking down trends into several components, including seasonality (yearly, weekly, and daily) and holiday effects. Prophet demonstrates robustness in handling missing data and trend shifts, while also exhibiting a capability to effectively manage outliers. The parameters of the Prophet model we used are changepoint_prior_scale, seasonality_prior_scale, holidays_prior_scale and seasonality_mode.

LSTM.

Another model we tried is a LSTM-based model. LSTM is a type of artificial recurrent neural network (RNN) architecture commonly utilized in the field of deep learning [32]. Unlike standard feedforward neural networks, LSTM has feedback connections. It can process not only individual data points but also entire sequences of data. A LSTM unit comprises a cell, an input gate, an output gate, and a forget gate. LSTM networks are highly suitable for classifying, processing, and predicting based on time series data. This is because they can effectively handle situations where there are lags of unknown duration between significant events in a time series.

ResNet.

Additional model is Residual Networks (ResNets) [34], a deep convolutional network. Its architecture is designed to support hundreds or thousands of convolutional layers. The fundamental concept involves bypassing blocks of convolutional layers through shortcut connections. These connections allow the network to learn residual mappings instead of attempting to learn direct mappings, which enables the successful training of much deeper networks.

Inception.

A more recent model is the Inception [35], which represents a substantial advancement over existing deep learning models. It has demonstrated competitive performance compared to other state-of-the-art models. The key innovation of the Inception model is the use of inception modules in Convolutional Neural Networks, which consist of multiple parallel convolutional layers of different kernel sizes. The inception modules were devised to address both the issues of computational expense and overfitting.

Models parameters and baselines.

To run the prediction models that were described before, it is required to tune the model’s parameters. Each model type has different tuning parameters that were used: for the DL models—LSTMs, Resnet, Inception—we used the parameters: number of epochs, batch size and learning rate. For the Prophet model we used the parameters: changepoint_prior_scale, seasonality_prior_scale, holidays_prior_scale and seasonality_mode, and for the Rocket model we used the parameter: number of kernels.

Each model was trained and tested on each participant separately. Each participant had a large dataset of sliding windows. A nested cross validation was used to get appropriate results and for hyper-parameter tuning.

Experiment 1—Observation time duration.

The goal of this experiment was to evaluate the best observation time windows’ duration (research question 1). For that 30, 60 or 90 minutes durations of observation time windows were used, including: glucose measurements, heart rate, step detector, activities, and the motion sensors. The observation time durations were experimented with of 5, 10, 15, 30, 45 or 60 minutes prediction time periods. We applied the models: Resnet, Inception, RNN-LSTMs, Prophet, and Rocket, with all the temporal variables combinations, and separately, and different model’s parameters, such as batch size, epochs, learning rate for the ANN models and number of kernels for the Rocket model.

Experiment 2—Best prediction model.

To answer the rest research questions, we would like to find which model achieves the best results. For that, we performed the second experiment which focused on evaluating the performance of the prediction models—Resnet, Inception, RNN-LSTMs, Prophet, and Rocket—to find out the best model based on a specific observation time window. For that the best observation time window, which performed best in experiment 1 was intended to be used. For each model we evaluated the best model’s parameters by applying each model with different model’s parameters, and temporal variables combinations as described in experiment 1.

Experiment 3—Best temporal variables combinations.

This experiment focused on evaluating the best prediction results and which temporal variables combinations achieve those results (research questions 4). Since we tested all combinations of the temporal variables, we can also answer the research questions 2, 3. For that the best observation time window, which evaluated in experiment 1, and the best model which evaluated in experiment 2 were used.

Experiment 4—Comparing our method to previous methods.

This experiment focused on evaluating our method against previous methods in the literature. Because our dataset is unique, it is not true to compare our results (RMSE) to previous articles results, so we applied previous methods on our dataset and compared the results. The article we found as the best for comparing is the article of Bosoni et al. [29], whose method achieved better results compared to other methods on the same datasets. In addition, their method and experiments are clearly described so that we could reproduce the method accurately. The temporal variables we used for their method are glucose and HR, as they did in the article. The results of the Bosoni et al. [29] we compared to our best results, as we found in experiment 3.

Evaluation metrics.

The metrics we used to evaluate the results are Root Mean Square Error(RMSE) and Clarke Error Grid Analysis (EGA).

RMSE effectively highlights the magnitude of prediction errors, including the impact of larger deviations, while Clarke EGA provides a critical assessment of the clinical implications of those errors visually. Their combination ensures that the model’s predictions are both numerically accurate and safe for patient care.

Root Mean Square Error (RMSE).

RMSE is a commonly used metric for evaluating the performance of a regression model. It is the standard deviation of the residuals (prediction errors). Residuals are the differences between the observed and predicted values. They measure how far from the regression line data points are. The RMSE formula contains the predicted value (), the observed value (y) and the number of observations (n).

Clarke Error Grid Analysis (EGA).

The EGA was introduced by Clarke et al. in 1987 [40] and is widely used in the evaluation of glucose monitoring systems. The metric evaluates the differences between estimated and actual BG levels over a specified time horizon. These differences are visually represented on a chart. EGA uses a grid with five zones, each represents a different level of clinical significance in terms of the impact on patient management. Zone A is defined as clinically accurate. It includes predicted values that have a deviation of no more than 20% from the real values, or values falling within the hypoglycemic range (<70 mg/dL). In Zone B, the differences between the predicted and real measurements exceed 20%, but it causes little or no harm to the patient, so it’s still acceptable. Zone C represents significant errors that could lead to unnecessary treatment changes. In Zone D, instances signify a failure to detect and address deviations in blood glucose levels. This occurs when the actual blood glucose levels fall outside the acceptable range, despite the predictions being within acceptable limits. Zone E identifies instances where predicted values contradict real or measured blood glucose levels, potentially leading to treatments contrary to recommended actions.