Fig 1.
Outline of the main steps in this study.
(1) Patient surveys and data acquisition from our institute’s medical records, (2) brain tumor segmentation using pre- and postoperative MRI, (3) imaging feature extraction using a pretrained variation autoencoder combined with a convolutional neural network, and (4) prediction model development using a neural network and training. The performance metrics in the training set were evaluated using 10 repetition of fivefold cross-validation. GBM = glioblastoma; HGG = high-grade glioma; VAE = variational autoencoder.
Fig 2.
Tumor segmentation using preoperative and postoperative MRI.
Using a pretrained U-net-based segmentation module, pre- and postoperative segmented images were generated from 24 slices of FLAIR, T1W, T1Gd, ADC, and DWI images. The brain parenchyma is displayed in gray, enhanced tumor lesions and necrosis in red, and peritumoral edema and non-enhancing lesions in green. T1Gd = contrast-enhanced T1W.
Fig 3.
Extraction of deep imaging features from the latent space of a variational autoencoder.
Twenty-four slices of the segmented images were separated into tumor lesions (red or green) and brain mask images (gray). The tumor lesion images were processed by VAE 1, where internal 3D convolutional neural networks extracted features and reduced dimensionality using encoder 1. As a result, a 48-dimensional latent space was formed, and the generated deep imaging features were incorporated into the KPS prediction model. Similarly, the brain mask images were processed by being input into VAE 2. Both VAEs 1 and 2 were pretrained using the BraTS 2020 dataset. VAE = variational autoencoder; KPS = Karnofsky performance status; BraTS = brain tumor segmentation challenge.
Table 1.
Baseline clinical characteristics.
Fig 4.
Development of models to predict 6-month postoperative KPS score of <70 using the training set.
During model development, performance was evaluated using the training set with 10 repetitions of fivefold cross-validation. The ROC curves for each repeat of the fivefold cross-validation are represented in gray, whereas the mean ROC curve for the 10 repeats is shown in blue. The area under the curve of the model was 0.715 when using clinical parameters (A), 0.651 when using deep imaging features from pre- and postoperative MRI (B), and 0.785 when combining clinical parameters with deep imaging features (C). (D) The top five feature contributions in the multimodal model are evaluated by grouped permutation importance. (E) The predicted performance of each model. Data are shown as the mean score ± standard deviation. KPS = Karnofsky performance status.
Fig 5.
Prediction performance in the test set.
The clinical-based, MRI-based, and multimodal models were pretrained using the training set, then their prediction performance was evaluated using the test set.