Table 1.
Overall description of the different RMK metrics.
Fig 1.
Dependence of the length of the ant’s path L (log scale) on the value of R2.
As can be seen from this plot, L increases 10-fold as R2 decreases by 0.2 units up to a R2 value of ~0.2, and at a much greater rate for R2 values lower than 0.2.
Fig 2.
Schematic illustration of the ensemble neural network models used to predict the clinical scales from the RMK variables.
Each subset of features identified by the artificial ant algorithm was used to construct 10 independent neural network models using exactly the same network topology and training parameters but a different random seed number (and thus different initial synaptic parameters and presentation sequence of the training samples). The predictions of these 10 models were averaged to produce the aggregate prediction of the ensemble.
Table 2.
Descriptive statistics.
Fig 3.
SPE map of the correlation distances of the clinical and RMK parameters for the completers cohort.
The map was derived by computing the pairwise Pearson correlation coefficients (R) for all pairs of features, converting them to correlation distances (1-abs(R)), and embedding the resulting matrix into 2 dimensions in such a way that the distances of the points on the map approximate as closely as possible the correlation distances of the respective features. The clinical parameters are highlighted in red, the RMK parameters on the affected side in blue, and the RMK parameters on the unaffected side in green. The map also shows distinct clusters of correlated variables which are preserved on both the affected and unaffected sides (outlined by green and blue ellipses, respectively).
Fig 4.
Cross-validated R2 of the best models derived from the completers (solid lines) and validated with the non-completers (dashed lines) for each of the four clinical scales, using 2, 4, 6, 8, 10, 12 and 14 robot-derived RMK features.
The figure shows the ability of the robot-derived RMK models to predict the clinical scales with an increasing number of features. The model performance exhibits an asymptotic behavior with respect to the number of RMK features, reaching the point of diminishing returns at approximately 8 features for all four clinical scales. Note the small variance in the prediction of the trained data as shown by the small “whiskers,” which for the most part, are not visible in the figure.
Fig 5.
Cross-validated R2 of the best 8-feature models derived using three different approaches for feature selection and model building: 1) features selected by artificial ants with neural networks, model derived with neural networks (dark blue); 2) features selected by artificial ants with neural networks, model derived with multi-linear regression (light blue); and 3) features selected by PCA, model derived with multi-linear regression (red).
Every model was cross-validated using the same 10-fold cross-validation procedure described in the Methods session.
Fig 6.
Importance of individual RMK features in predicting the four clinical scales, obtained by systematically removing each feature from the training sample and repeating the feature selection, aggregation and cross-validation procedure for each derived data set.
Only models with 8 input and 2 hidden units are shown. The left-most data point and the horizontal solid line on the top part of the plot represent the cross-validated R2 of the best model with all features included, averaged over all 10 cross-validation runs (standard deviations shown as error bars). Each subsequent point shows the R2 of the corresponding apo model, i.e., the model derived by omitting the feature shown on the x axis (the model still includes 8 features, just not the one shown on the x axis). The individual markers at the bottom part of the plot indicate whether there is a statistically significant difference between the R2 distributions of the all-feature and the respective apo models (the presence of a marker indicates that the difference between the two distributions is statistically significant, and the absence that it is not).
Fig 7.
Optimization of effect size for robot-derived RMK metrics.
The horizontal lines show the day 7 to day 90 effect size for comparable patients of the historical VISTA data for the NIH, as well as the effect sizes for the NIH, FM and MP assessment scales for our completers cohort. The figure also shows the performance of the robot-derived RMK composites optimized for effect size for the trained (solid lines) and cross-validated sets (dashed lines). Note the increase of over 20% in cross-validated effect size for the RMK composites over the clinical scales with 4-features for this study (and over 70% over the historical data).
Fig 8.
Correlation (expressed as R2) of the FM, MP and NIH original scales with the corresponding optimized 8-feature composites obtained with the greedy algorithm.