Fig 1.
Framework for explainable artificial intelligence for medical simulation training.
The top section provides an overview of standard machine learning methodology to obtain a predictive model. The bottom section harvests the power of the machine learning model for education. The simulation task generates raw data, which can be manipulated to create metrics of performance. Using algorithms, statistical methods, or expert opinions, metrics are selected based on their ability to differentiate between two or more groups. The selected metrics are fed to a machine learning algorithm for training and testing. A final predictive model can be selected based on predictive accuracy to build the Virtual Operative Assistant. Upon recruitment of new participants, their metrics of performance can be calculated and normalized. These are then fed to the Virtual Operative Assistant to provide a group classification (e.g. skilled or novice) as well as an individual breakdown of metric performance. The feedback reinforces positive behaviour while providing detailed information on which behaviours to improve. The Virtual Operative Assistant is an iterative program optimized for user learning.
Fig 2.
Confusion matrix for the classification of skilled and novice participants by the linear support vector machine.
The algorithm correctly classified all skilled participants while correctly classifying 18 of the 22 novice participants (82%).
Table 1.
Demographics information for two groups of participants performing the virtual reality neurosurgical task.
Table 2.
Selected metrics of performance for simulated neurosurgical task.
Fig 3.
Educational paradigm of the Virtual Operative Assistant.
(A) The trainee performs a simulated subpial tumor resection scenario on the NeuroVR (CAE Healthcare, Montreal, Quebec, Canada) platform using a simulated ultrasonic aspirator in the trainee’s dominant hand and a simulated bipolar in the non-dominant hand. (B) The scenario involves removal of a cortical tumor (yellow) with minimal damage to healthy brain regions (white). (C) Upon completion of the simulated task, the data is automatically saved and uploaded to the Virtual Operative Assistant software to provide instant feedback on two monitors.
Fig 4.
Performance assessment with the Virtual Operative Assistant.
User received feedback on performance on two monitors. (A) The first screen informs the trainee of their classification along with a corresponding percentage for each class. (B) The second screen provides individualized feedback on 2 safety metrics. Metrics are accompanied with a positive statement (if competency achieved) or instructions to improve (if failed to achieve competency). Overall written and auditory feedback is provided on the second monitor. If the trainee has not achieved competency in all safety metrics (top row), the trainee cannot proceed further and must redo the scenario. If the trainee has achieved competency in all safety metrics (bottom row), the trainee can move on to step 2. (C) The third screen provides individualized feedback on 2 instrument motion metrics, accompanied by statements on their performance.