Table 1.
17 Types of features in the chart events.
Table 2.
Demographic features.
Fig 1.
Statistical feature computation.
For numerical chart events, we conduct linear regression on the 48-hour data points and record the rate and bias value as the feature. For categorical events, we simply compute the average occurrence of the categories.
Fig 2.
The 1D multi-filter convolutional neural network.
We conduct the convolution on the time axis with 48-hour time window and D dimension using filter size 2, 3 or 4 accordingly. The computed feature maps are finally concatenated and fully connected to a dense decision layer with one output neuron.
Fig 3.
The data structure of input data used with CNN and LSTM models.
D: dimension, h: hour.
Fig 4.
A bidirectional LSTM combined with an additional LSTM layer, followed by a dense decision layer with one output neuron activated by a sigmoid function. Overall, we have 16 hidden units in our LSTM layer.
Table 3.
Performance comparison of various machine learning models on different sets of features.
Fig 5.
Combination of LSTM and CNN models.
(a) CNN+LSTM model, the CNN follows a multi-filter convolution computation with zero padding to maintain the timestamp consistency for different groups of feature maps. The following LSTM only outputs the hidden units of the last time stamp. (b) LSTM+CNN model, CNN computes the feature maps without zero padding after receiving the output hidden unit sequence from LSTM.
Fig 6.
ROC curve of selected high performing machine learning models.
The color bar is the error bar of the ROC curve with five-fold cross-validation. LSTM-CNN model performs relatively better than other ones. CE: chart events. D: Demographic features.
Table 4.
Performance comparison of machine learning models at high-sensitivity and high-specificity operating points.
Fig 7.
The results of feature ablation test.
The importance of chart events for predicting the ICU readmission. The y-axis shows the changing ratio of the prediction results after we replace the original feature with its normal value.
Fig 8.
Cumulative density function curve of LSTM-LR-C (red line) and LSTM-C (blue line). Figure shows that there are more patient records in the LSTM-C which have at least one chart event with high oscillation sequence. Essentially, compared to Logistic Regression, our LSTM+CNN model is capable of capturing high volatile time series behavior, a common pattern in high-risk ICU patients.
Fig 9.
(a) A selected ICU stay with the highest heart rate event oscillation, and (b) another case with the highest oscillation of respiration rate. These two patients are predicted correctly by the LSTM-CNN model, but wrongly by the traditional models. In both cases, the abnormal sequence has oscillated around the normal value of the chart event, which in return a linear model would regress it to a normal value with a negligible slope. Effectively, our LSTM-CNN is capable of capturing such high volatile behavior, a common pattern among high-risk ICU patients with unstable status.
Table 5.
Kolmogorov–Smirnov (K-S) test for the distribution of fluctuation between LSTM-C and LR-C for each chart event.
Fig 10.
Cumulative density function (CDF) plots and probability density function (PDF) plots.
Table 6.
Summary of the number of cases correctly predicted by the corresponding baseline model as well as the LSTM+CNN.
Fig 11.
CDF plots of wn for sets in Table 6.