https://orcid.org/0000-0002-0289-0743
Figures
Abstract
In a changing learning environment where young neurointerventionalists spend less time in the operating room, computer simulators have been established as a new training model. Our aim was the comparison of silicone models and computer simulators, and the evaluation of their influence on subjective self-confidence of operators. Pre- and postquestionnaires of 27 participants and 9 tutors were evaluated after the participation in a three-days interventional stroke course using silicone models and computer simulators. Training on computer simulators was considered as more realistic and important before patient contact than training on silicone models. Participants rated their own abilities as significantly better after participation in the course and felt significantly better prepared for patient care. Training on computer simulators can increase the subjective self-confidence of trainees. We suggest a stepwise training program, comprising both ex-vivo and the porcine in-vivo model, finished by conventional operating room teaching, to prepare neuroradiologists for optimal patient care when performing interventions.
Citation: Mathern N, Sandmann J, Sichtermann T, Ridwan H, Riabikin A, Stockero A, et al. (2022) Can training on ex-vivo models increase neurointerventionalists’ subjective self-confidence in the operating room? PLoS ONE 17(2): e0264180. https://doi.org/10.1371/journal.pone.0264180
Editor: Juan Manuel Marquez-Romero, Instituto Mexicano del Seguro Social (IMSS) HGZ 2, MEXICO
Received: July 12, 2021; Accepted: February 4, 2022; Published: February 22, 2022
Copyright: © 2022 Mathern et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: • Martin Wiesmann has the following disclosures: Consultancy: Stryker. Payment for Lectures: Bracco, Medtronic, Siemens, Stryker. Payment for Developemtn of Educational Presentations: Bracco, Codman, Medtronix,Phenox, Siemens. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Ischaemic stroke is one of the most frequent causes of death worldwide [1]. Endovascular stroke therapy has been established as the standard treatment option for emergent large vessel occlusion stroke. As a result, treatment has become more effective, but also more complex, necessitating more efficient training.
In the past, practical training of young neurointerventionalists took place in the operating room. However, the aim of increasing patient safety and increased complexity of neurointerventional procedures have led to the development of new training methods [2–4]. A major advantage of ex-vivo models is patient safety, as mistakes remain without clinical consequence [5–7]. By now, the most common training programs for neurointerventionalists consist of training on silicone and animal models. Computer simulators fill the gap between the opportunities, offered by these models. They are increasingly used in neurointerventional training because of their high degree of realism and the possibility to train various scenarios [8–11].
A major aim of such training programs is not only to train the actual procedure and to reduce complication rates, but also to increase the self-confidence of the trainee. To date, there are no studies that address the impact of computer-based simulators on subjective self-assessment. In this study we assessed whether computer-based simulators or silicone models improve subjective self-confidence of neurointerventionalists.
For this purpose, we prospectively evaluated an interventional stroke course, which offered hands-on training on silicone models and computer simulators, organized by the German Society of Neuroradiology (“German Stroke School”) held in Frankfurt, Germany, in October 2018. Using pre- and postquestionnaires, we asked participants whether their skills improved subjectively and whether they felt better prepared for patient care after joining this course. In a second part of the study we compared the two ex-vivo models by asking tutors and participants for their subjective assessment, hypothesizing that participants would benefit rather from computer simulators than from silicone models.
Material and methods
Endovascular training program
The three-days course took place in Frankfurt/Main in October 2018 and was aimed at young neurointerventionalists with first experience in intervention and angiography. Thirty-five national and international participants with different levels of experience in neuroradiology joined this course and were supervised by sixteen advanced tutors. The supervising tutors had several years of experience in interventional neuroradiology as well as in the use of computer simulators and silicone models.
The course consisted of a comprehensive lecture program and an intensive hands-on training on computer simulators and silicone models. To reduce the number of participants per group, the attendees were split up in two groups prior to participation. On the first day, they received a theory lesson and a short introduction in the use of silicone models and computer simulators. On the second and third day, four sessions per day were held: two practical and two theoretical sessions on various aspects of neurointerventional stroke therapy. During the practical parts, two attendees each trained on the silicone models and computer simulators under the guidance of one tutor. The practical training involved lessons about the access to supraaortal vessels, stent-retriever extraction, and carotid-stenting. Each session lasted a maximum of two hours with breaks in between.
Questionnaires
Participants and tutors received two questionnaires to be filled out: one before and one after the course. Both questionnaires included questions in English language.
In the prequestionnaire, the respondents were asked how many years of experience they had, the number of thrombectomies they had assisted with, and the number of thrombectomies they had performed on their own. Furthermore, they were asked about their previous experience with silicone models and computer simulators and their opinion about the usage of these ex-vivo models in neurointerventional training. The respondents were also asked how realistic they expected the training on silicone models and computer simulators to be, how well they felt prepared for patient care before joining the course, and how they assessed their own abilities.
The postquestionnaire contained analogous questions to the prequestionnaire: The participants and tutors were asked about how realistic the training on silicone models and computer simulators was, and which model they would have liked to spend more time with. As in the prequestionnaire, the respondents were asked to assess their own abilities and whether they felt well prepared for patient care after participation in the course. They were also asked if they felt it to be important to train on ex-vivo models before performing thrombectomies in humans and which model they felt to be the most suitable one for such hands-on training. In addition, there was a question about whether their expectations about joining the course had been fulfilled.
Data and statistical analysis
We used Chi-square, Fishers exact, and Wilcoxon test for the statistical analysis. P-values of an α level ≤.05 were considered as statistically significant. The statistical analysis was calculated in SPSS Statistics 25 software (IBM, Armonk, New York, USA).
For the statistical evaluation of the questionnaires, we evaluated the answers of the total group, the participants, and the tutors.
In a first part of the evaluation, we assessed whether the subjective self-confidence of the participants improved after joining the course. The group of participants was divided into participants with previous experience with computer simulator training (Group A) and those without previous experience with this kind of training (Group B). We determined, if there was a significant difference between the groups, and within a group before and after participation in the course.
For a comparison of both models, we investigated if there was a significant difference between the group of all participants and the tutors, in the second part of the evaluation. For questions asked before and after participation in the course and the comparison between both ex-vivo models, we also evaluated the differences within a group.
Ex-vivo models
Silicone models.
For the practical training two silicone models were provided. The silicone models (Neuro Testing Model Plus, United Biologics, Santa Ana, California, USA) were filled with distilled water, with addition of shampoo to reduce friction of the devices on the silicone. A pulsatile pump was connected to imitate blood flow. The models were made of soft and transparent silicone, which enabled the participant to practice under direct visualization. A camera and a screen were used to enable indirect visualization. The silicone models include the aorta, the supraaortic branches, the circle of Willis, and the proximal segments of the anterior, middle and, posterior cerebral artery (A2, M2, P2). Training comprised catheterization of the aortic arch and the intracranial arteries as well as stent-retriever deployment.
Computer simulators.
Twelve computer simulators (VIST G5, Mentice, Gothenburg, Sweden) were used. The vascular intervention system trainer (VIST) is a state of the art computer-based system that uses a geometric vessel representation which allows a high degree of realism for the user. Actual endovascular devices used in clinical practice, such as standard microcatheters, guidewires, and stent-retrievers can be applied in these computer simulators. In this course the clinical software module “Acute Ischemic Stroke Intervention” was used (Mentice, Gothenburg, Sweden). Scenarios comprised thrombectomies in different branches using stent-retrievers and balloon-guide catheters.
This study was part of our continuous quality control procedures and as such approved by the Ethics Committee of the University Hospital RWTH Aachen. Need for consent of participants was waived since the questionnaires did not contain any personal information and were thus fully anonymous.
Results
We received a total of 39 questionnaires. Thirty-six were completely filled in, of which nine (25%) were completed by tutors and 27 (75%) by participants.
Evaluation of the course
For the first part of the evaluation, the group of participants was split into those with previous experience with computer simulator training (Group A: 13 participants, 48%) and those without previous experience with this kind of training (Group B: 14 participants, 52%). Tables 1 and 2 provide the answers of all participants and the comparison between and within group A and B.
We found a significant difference between the groups (p = 0.031) when the participants were asked about their previous experience with training on silicone models (Group A: 77%, Group B: 36%). Before joining the course, 31% of group A and 14% of group B indicated that they felt well prepared for patient care. Afterwards, 85% of group A, and 86% of group B agreed with this statement. There was a significant difference before and after participation in this course within both groups (Group A: p = 0.008, group B: p = 0.001; Fig 1).
Before joining the course, 31% of participants with previous experience with computer simulators (group A) and 14% of participants without previous experience with computer simulators (group B) indicated that they felt well prepared for patient care. Afterwards, 85% of group A, and 86% of group B agreed with this statement. There was a significant difference before and after participation in this course within both groups (Group A: p = 0.008, group B: p = 0.001).
Computer simulator versus silicone model
Tutors and participants were asked to evaluate computer simulators and silicone models (Tables 3 and 4): Silicone models were considered to be realistic by 91% and 86% of all respondents before and after joining the course, respectively.
Before attending the course, training on computer simulators was assessed to be realistic by 84% of the participant and 100% of the tutor group. Afterwards, all participants (100%) and almost all tutors (89%) agreed with this statement. Training on silicone models was considered to be important by 77% (71% of the participant group, 100% of the tutor group, p = 0.161) and training on computer simulators by 100% of both groups. We found a significant difference within the group of participants (p = 0.008), when we compared how important they assessed the training on silicone models or computer simulators before doing thrombectomies in humans.
After the course, 35% of all respondents indicated that they wanted to spend more time on the silicone model (43% of the participants, 13% of the tutors). By contrast, 80% of all respondents, 86% of the participant group, and 50% of the tutor group agreed to the statement that they would have liked to spend more time on the computer simulator. When comparing the two models, a significant difference was found within the participant group (p = 0.001).
All data that support the findings of this study are provided in full detail in the supporting information files.s
Discussion
Our results show that training on silicone models and computer simulators is considered to be useful before patient care.
Tutors and participants rated the training on computer simulators as more important and realistic than training on silicone models. Silicone models offer a simplified training setting compared to the computer simulator, because direct visualization is possible. This is in line with previous studies, which have shown that silicone models are particularly helpful for the training of basic skills [12].
Simulators provide a broad spectrum of scenarios and uniform standards, which makes them effective ex-vivo models in endovascular training for novice and advanced neurointerventionalists. It is a predominant advantage of simulators, that surgical skills can be acquired in a safe environment [13,14]. Techniques can be practiced and repeated before they are applied to the patient. Unlike the silicone model, simulation-based training allows to carry out the most common procedures, such as guidewire and catheter navigation, but also more complex scenarios like carotid artery stenting [10,15]. Tutors rated the computer simulator training as more important and realistic than the training on silicone models. For experienced neurointerventionalists who have already basic knowledge, computer simulation offers the advantage of training complication management, expanding own skills, and learning new procedures. In addition, the use of the latest generation of simulators enables the objective assessment of key competencies and the evaluation of one’s own improvement over time [16].
Our results show that training on silicone models and computer simulators improves self-confidence of aspiring neurointerventionalists. Previous training with ex-vivo and porcine in-vivo models can help the trainee to feel more comfortable in the operation room, which is often perceived as more stressful than the educational setting. Pre-acquired basics in all available models can prepare the trainee for optimal patient care. Stolarek et al. claimed that intensive simulation-based training can be as effective as clinical experiences [17]. Other studies state that ex-vivo models will not be able to replace the operating room as a training area for residents [4,17]. In our opinion, the ideal training consists of a combination of ex-vivo and in-vivo models and the conventional operating room training. A training for neurointerventional residents should include a sort of pretraining, ensuring that the trainee has a theoretical understanding of the skills, the devices, and the correct methods of the performance [2]. After the theoretical part, basic skills and new techniques should be trained on silicone models. Computer simulation training allows first processing complete procedures. Liebig et al. even demand, that trainees must reach a certain level of proficiency on the computer simulator, before treating a real patient [18]. Porcine in-vivo models could therefore be the last step, completed by the final training in the operating room on real patients. For ethical reasons, porcine in-vivo models should be used after solid basic skills have been acquired in both ex-vivo models. All these training models should not be seen as a replacement for traditional residents training, but as an essential addition [18].
Limitations
The ideal measure for the effectiveness of training is not self-confidence but improved clinical skills and ultimately improved clinical outcome. As participants of our course came from different clinics and countries with different practices and infrastructure, it was impossible to assess whether our training had an impact on clinical outcome of subsequent patients. An alternative to measuring the real-life performance would be testing the performance with varying simulation scenarios. However, this involves the risk of testing specifically trained simulation-skills instead of real-life skills. In summary, the transferability of simulation-learnt skills to clinical practice is a major limitation of simulation-based training, which has not been addressed sufficiently in the literature [19,20]. This is why we focused our analysis on self-assessment as a subjective, but reliable factor. However, even though an increase in self-confidence is generally perceived as an advantage in clinical practice, we see a risk in the discrepancy between the actual ability and subjective self-assessment. Recurring scenarios and repetitions of the same exercises may lead to increased self-confidence and a false sense of security.
Conclusion
We come to the conclusion that a variety of simulators should be used. Importantly, both tutors and participants rated the training on computer simulators as more important and realistic than training on silicone models. Therefore, a fixed part in the training curriculum of neurointerventionalists should be computer simulators as they provide many benefits. Residents today have to deal with increased patient safety standards and the effect of time constrains in their education. In this changing learning environment, simulation based learning could fix the gap between theory and practice [17].
Acknowledgments
The authors wish to express their sincere thanks to Johanna Breuning, Nina Cryns, Kevin Hädrich, and Inga Holter for their commitment to plan and implement the German Stroke School 2018.
Members of the German Stroke School Group: Ansgar Berlis, Diagnostic and Interventional Neuroradiology, University Hospital Augsburg, Augsburg, Germany
Marc Brockmann, Department of Neuroradiology, University Hospital Mainz, Mainz, Germany
René Chapot, Department of Radiology and Neuroradiology, Alfried Krupp Krankenhaus, Essen, Germany
Johannes Gerber, Department of Neuroradiology, University Hospital Dresden, Dresden, Germany
Hans Henkes, Neuroradiological Clinic, Klinikum Stuttgart, Stuttgart, Germany
Olav Jansen, Department of Radiology and Neuroradiology, University Hospital Schleswig-Holstein Campus Kiel, Kiel, Germany
Thomas Liebig, Department of Neuroradiology, Ludwig Maximilian University, Munich, Germany
Markus Möhlenbruch, Department of Neuroradiology, Heidelberg University Hospital, Heidelberg, Germany
Walter Möller-Hartmann, Department of Radiology, Neuroradiology and Nuclear Medicine, Hospital Ludmillenstift, Meppen, Germany
Anastasios Mpotsaris, University Clinic of Neuroradiology, Otto-von-Guericke-University Magdeburg, Germany; mpotsaris@unitybox.de
Hadi Nasri, Department of Diagnostic and Interventional Neuroradiology, University Hospital RWTH Aachen, Germany
Marios Psychogios, Clinic of Radiology and Nuclear Medicine, University Hospital Basel, Basel, Switzerland
Marc Ribo, Hospital Universitari Vall d’Hebron, Barcelona, Spain
Stefan Rohde, Department of Radiology and Neuroradiology, Klinikum Dortmund, Dortmund, Germany
Michael Schönfeld, Department of Radiology, University Hospital Cologne, Cologne, Germany
Peter Schramm, Department of Neuroradiology, University Hospial Schleswig-Holstein Campus Lübeck, Lübeck, Germany
Bernd Turowski, University Dusseldorf, Medical Faculty, Department of Diagnostic and Interventional Radiology, Dusseldorf, Germany
Horst Urbach, Department of Neuroradiology, University Hospital Freiburg, Freiburg, Germany
Isabel Wanke, Swiss Neuroradiology Institute, Zurich, Switzerland
References
- 1. Meyers PM, Schumacher HC, Connolly ES Jr., Heyer EJ, Gray WA, Higashida RT. Current status of endovascular stroke treatment. Circulation. 2011;123(22):2591–601. pmid:21646506
- 2. Samia H, Khan S, Lawrence J, Delaney CP. Simulation and its role in training. Clin Colon Rectal Surg. 2013;26(1):47–55. pmid:24436648
- 3. Paramasivam S, Baltsavias G, Psatha E, Matis G, Valavanis A. Silicone models as basic training and research aid in endovascular neurointervention—a single-center experience and review of the literature. Neurosurg Rev. 2014;37(2):331–7; discussion 7. pmid:24463914
- 4. Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, et al. Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg. 2005;241(2):364–72. pmid:15650649
- 5. Satava RM. Accomplishments and challenges of surgical simulation. Surg Endosc. 2001;15(3):232–41. pmid:11344421
- 6. Schimmel DR, Sweis R, Cohen ER, Davidson C, Wayne DB. Targeting clinical outcomes: Endovascular simulation improves diagnostic coronary angiography skills. Catheter Cardiovasc Interv. 2016;87(3):383–8. pmid:26198625
- 7. Namba K, Mashio K, Kawamura Y, Higaki A, Nemoto S. Swine hybrid aneurysm model for endovascular surgery training. Interv Neuroradiol. 2013;19(2):153–8. pmid:23693037
- 8. Hsu JH, Younan D, Pandalai S, Gillespie BT, Jain RA, Schippert DW, et al. Use of computer simulation for determining endovascular skill levels in a carotid stenting model. J Vasc Surg. 2004;40(6):1118–25. pmid:15622365
- 9. Dayal R, Faries PL, Lin SC, Bernheim J, Hollenbeck S, DeRubertis B, et al. Computer simulation as a component of catheter-based training. J Vasc Surg. 2004;40(6):1112–7. pmid:15622364
- 10. Miller ZA, Amin A, Tu J, Echenique A, Winokur RS. Simulation-based Training for Interventional Radiology and Opportunities for Improving the Educational Paradigm. Tech Vasc Interv Radiol. 2019;22(1):35–40. pmid:30765075
- 11. Koch A, Pfandler M, Stefan P, Wucherer P, Lazarovici M, Navab N, et al. Say, What Is on Your Mind? Surgeons’ Evaluations of Realism and Usability of a Virtual Reality Vertebroplasty Simulator. Surg Innov. 2019;26(2):234–43. pmid:30646810
- 12. Sandmann J, Muschenich FS, Riabikin A, Kramer M, Wiesmann M, Nikoubashman O. Can silicone models replace animal models in hands-on training for endovascular stroke therapy? Interv Neuroradiol. 2019;25(4):397–402. pmid:30895839
- 13. Patel R, Dennick R. Simulation based teaching in interventional radiology training: is it effective? Clin Radiol. 2017;72(3):266 e7–e14. pmid:27986263
- 14. Spiotta AM, Rasmussen PA, Masaryk TJ, Benzel EC, Schlenk R. Simulated diagnostic cerebral angiography in neurosurgical training: a pilot program. Journal of Neurointerventional Surgery. 2013;5(4):376–81. pmid:22576472
- 15. Dawson DL, Meyer J, Lee ES, Pevec WC. Training with simulation improves residents’ endovascular procedure skills. J Vasc Surg. 2007;45(1):149–54. pmid:17210400
- 16. Ernst M, Kriston L, Romero JM, Frolich AM, Jansen O, Fiehler J, et al. Quantitative Evaluation of Performance in Interventional Neuroradiology: An Integrated Curriculum Featuring Theoretical and Practical Challenges. PLoS One. 2016;11(2):e0148694. pmid:26848840
- 17. Procedural Stolarek I. and examination skills of first-year house surgeons: a comparison of a simulation workshop versus 6 months of clinical ward experience alone. N Z Med J. 2007;120(1253):U2516. pmid:17514217
- 18. Liebig T, Holtmannspotter M, Crossley R, Lindkvist J, Henn P, Lonn L, et al. Metric-Based Virtual Reality Simulation: A Paradigm Shift in Training for Mechanical Thrombectomy in Acute Stroke. Stroke. 2018;49(7):e239–e42. pmid:29866758
- 19. Fargen KM, Arthur AS, Bendok BR, Levy EI, Ringer A, Siddiqui AH, et al. Experience with a simulator-based angiography course for neurosurgical residents: beyond a pilot program. Neurosurgery. 2013;73 Suppl 1:46–50. pmid:24051882
- 20. Chaer RA, Derubertis BG, Lin SC, Bush HL, Karwowski JK, Birk D, et al. Simulation improves resident performance in catheter-based intervention: results of a randomized, controlled study. Ann Surg. 2006;244(3):343–52. pmid:16926560