http://orcid.org/0000-0002-6708-2499
Figures
Abstract
Purpose
Surgical education videos currently all use a single point of view (POV) with the trainee locked onto a fixed viewpoint, which may not deliver sufficient information for complex procedures. We developed a novel multiple POV video system and evaluated its training outcome compared with traditional single POV.
Methods
We filmed a hip resurfacing procedure performed by an expert attending using 8 cameras in theatre. 30 medical students were randomly and equally allocated to learn the procedure using the multiple POV (experiment group [EG]) versus single POV system (control group [CG]). Participants advanced a pin into the femoral head as demonstrated in the video. We measured the drilling trajectories and compared it with pre-operative plan to evaluate distance of the pin insertion and angular deviations. Two orthopedic attendings expertly evaluated the participants’ performance using a modified global rating scale (GRS). There was a pre-video knowledge test that was repeated post-simulation alongside a Likert-scale questionnaire.
Results
The angular deviation of the pin in EG was significantly less by 29% compared to CG (p = 0.037), with no significant difference in the entry point’s distance between groups (p = 0.204). The GRS scores for EG were 3.5% higher than CG (p = 0.046). There was a 32% higher overall knowledge test score (p<0.001) and 21% improved Likert-scale questionnaire score (p = 0.002) after video-learning in EG than CG, albeit no significant difference in the knowledge test score before video-learning (p = 0.721).
Citation: Wang H, Sugand K, Newman S, Jones G, Cobb J, Auvinet E (2019) Are multiple views superior to a single view when teaching hip surgery? A single-blinded randomized controlled trial of technical skill acquisition. PLoS ONE 14(1): e0209904. https://doi.org/10.1371/journal.pone.0209904
Editor: Lars-Peter Kamolz, Medical University Graz, AUSTRIA
Received: May 15, 2018; Accepted: December 13, 2018; Published: January 9, 2019
Copyright: © 2019 Wang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This project was funded by the Uren Foundation to JC. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Surgical education has traditionally been on an apprenticeship model in which residents are given increasing autonomy while performing live surgical cases at the discretion of a supervising surgeon [1, 2], but in the past decades there has been a dramatic reconfiguration of surgical training driven by (i) increased public awareness of iatrogenic errors and lawsuits [3], (ii) restriction of duty hours due to the European Working Time Directive (EWTD) and American Accreditation Council for Graduate Medical Education, with an estimated 80% decrease in dedicated operating time as a result of EWTD [4, 5], (iii) an emphasis on efficient use of theatre time due to cost pressures [6,7]. Additionally, the visible surgical field is practically often limited, and it is difficult to have an adequate view to facilitate training [8]. Consequently, developing surgical training methods away from the operating theatre is being considered.
Outside operating theatres, orthopedic trainees practice technical skills on cadavers and sawbones. However, these teaching techniques are limited by the expense and short supply of cadavers, and the lack of high-fidelity of sawbones. Recently, virtual reality (VR) simulators have provided trainees with the opportunity to practice and overcome the learning curve in a high-fidelity, safe and controlled environment without compromising patient safety [9–13]. But the expense associated with their purchase and maintenance and lack of realism has restricted their use. Also, in orthopedic specialty, VR simulators are mostly suitable for teaching basic psychomotor skills in arthroscopy and fracture fixations [14–18], whilst the majority of orthopedic interventions consist of open procedures (such as arthroplasty) with complex anatomy that are not easily amenable to simulation yet. Consequently, adjunctive methods of acquiring surgical skills away from the operating room are necessary without compromising patient safety.
One solution to supplement surgical training has been the use of videos which have become increasingly popular because of their low cost and easy access. Existing video systems all use a single point of view (POV) with 2D or 3D rendering [19–23] whereby the trainees are locked into a single viewpoint. However, some open procedures, such as arthroplasty, require visuospatial awareness in multiple dimensions and from multiple viewpoints. Single and fixed POV may therefore not be sufficient to provide the necessary information for training purposes.
The challenge in surgery teaching resides in teaching both knowledge and spatial perception and gesture transmission. There by, to assess a 3D information such the orientation of a tool in space versus anatomical references, single POV video might reduce the exposure of the student to essential information.
The objective was to develop a novel multiple POV video system and to evaluate the training outcome of multiple versus traditional single POV. The following three outcomes were observed:
- Knowledge acquisition of the operative steps with a questionnaire
- Real-time technical and objective performance metrics achieved after completing the learning session with a GRS score marked by two expert orthopaedic surgeons
- Assessing the precision of pin positioning
The null hypothesis was observing no difference in technical skill and knowledge acquisition between single (control group) and multiple POV (experimental group) videos.
Materials and methods
Video recordings and development of a multiple POV video system
The following two procedures were recorded:
- Recording of a procedure performed on a sawbone by an expert orthopedic surgeon with clear explanations of each step,
- Recording of an actual hip resurfacing procedure in theatre, performed by an expert orthopedic surgeon (>20 years of experience).
Both videos were recorded by 14 commercial cameras (12 JVC HD camera, Yokohama, Japan; 2 Go Pro Hero 4 Black, California, USA) arrayed around the operating table after getting informed consent from the patient for the second video. Of these, 12 were fixed to the plastic laminar flow hood which surrounds the operating table, with one fixed to the handle of the surgical lamp and the remaining one mounted to the surgeon’s head. After review, 8 cameras considered to have captured optimal viewpoints were retained, whilst the remaining 6 were excluded. Based on the videos recorded from 8 POVs, a novel multiple-view video system was developed, which allowed the subject to switch between viewpoints. For mono-video control group, the view point was the camera attached to the light pole. No identifiable information or image of the patient was retained in the video. The individuals in this manuscript have given written informed consent (as outlined in PLOS consent form) to publish these case details.
Participant recruitment and randomization
The recruitment followed the process described in Fig 1. 30 student doctors were voluntarily recruited from Imperial College London. The designated tasks included learning the procedure using either multiple versus single POV video before conducting the guide wire insertion part of the operation on a simulated sawbone model. All participants were blinded with respect to the nature of the two arms of the study. Participants were randomly allocated to two different groups: (i) experiment group (multiple POV) and (ii) control group (single POV). They were given a unique identification number and randomly allocated to each cohort using a random number generator (Excel, Microsoft, Washington state, USA) by the research coordinator.
Schema of the recruitment process.
Ethical approval and consent.
The study was approved by the Medical Education Ethics Committee at Imperial College and given the reference number MEEC1415-32. Written informed consent from every participant was obtained, in which they agreed on the recording of their procedures and the use of the footage for research purposes only. The individual in this manuscript has given written informed consent (as outlined in PLOS consent form) to publish these case details.
Evaluating the efficacy of the video systems
Participants initially completed a multiple-choice test (S1 File) which tested their baseline knowledge of a hip resurfacing procedure. The test was designed using a modified Delphi technique with advice and feedback sought from four expert orthopedic attending surgeons. The experimental group (respectively control group) were asked to carry out the following tasks:
- View a multiple POV (resp. single POV) video on hip resurfacing procedure composed with a tutorial video captured with several cameras (resp. a front camera) for the simulated sawbone and with a real hip resurfacing procedure as captured a group of cameras (resp. by the camera mounted to the handle of the surgical lamp) (as show on Fig 2). They were allowed to rewind and fast-forward the video as they wish. The viewing time was twenty minutes.
- Complete the same multiple-choice test again, followed by a Likert-scale subjective questionnaire (S2 File) designed using a modified Delphi technique to demonstrate self-perception of their confidence and satisfaction of the learning experience.
- Repeat the procedure on a sawbone by inserting a pin into the femoral head as demonstrated in the video, with their performance recorded by a camera (with their face or identical information not included in the video).
- Watch the video seen by the opposite group before answering a subjective questionnaire to compare their preferences.
Interface of the novel multiple POV video system, which allows the user to click and navigate through any desired viewpoint, and the equivalent for standard single POV video system.
The sawbone models were then CT scanned to evaluate the drilling accuracy within the femoral head. The CT data (in Digital Imaging and Communications in Medicine [DICOM] format) was obtained and imported into Mimics (Materialise NV, Leuven, Belgium) software for 3D reconstruction. On each 3D model the actual drilling trajectories were extracted and compared to an ideal virtual drilling path set by the operating surgeon in the video. The difference in the distance (mm) of points of pin insertion into the femoral head, and the axis angle deviation between the actual drilled trajectory and the ideal virtual drilling path were then measured.
Both single and multiple video systems included identical verbal commentary about the surgical procedure, including how to achieve an ideal drilling trajectory in the femoral head as demonstrated by the expert surgeon.
2 expert surgeons experienced in hip resurfacing surgery evaluated the participants’ performance using the modified global rating scale (GRS) (S3 File), which was designed using a modified Delphi technique, after watching their performance recorded in the video.
Outcomes
Primary outcome was the objective performance metrics of the deviation between participants’ trajectory and ideal virtual trajectory of pin insertion into the femoral head. Secondary outcomes were (i) the GRS scores, (ii) knowledge test scores and (iii) questionnaire scores.
Power study
Based on a previous pilot study (n = 5) modelled on non-inferiority/superiority trials, a minimum number of 9 participants were required in each cohort (mean scores of 7.2 [experimental group] vs. 5.5 [control group], SD 1.27, 1-β = 80%, α = 5%). In the absence of a figure quoted in current literature, our pilot study suggested that a difference of 30% in the objective test scoring can be construed as being clinically significant, which was agreed by 3 expert orthopedic surgeons.
Statistical analysis
All data were analyzed using SPSS Statistics version 19 (IBM, New York, USA). For normally distributed data an independent t-test for dependent variables was adopted, while for data that were not normally distributed, a Mann-Whitney U test was used. Results for testing were calculated as mean (SD) [95% CI] and as median ± IQR (95% Bonett Price CI) for the Likert-scale questionnaire. Results are also displayed using box-and-whisker plots and bar charts. Statistical significance was set as p<0.05.
Results
Demographics
30 participants were successfully randomized and completed the experiment. The median age for the participants was 22 (range 19–24) years and their median undergraduate year in medical school was year 4 (range 3–5). Subgroup analysis showed that 47% (n = 14) participants were male and 93% (n = 28) of them were right-handed.
Outcomes
Primary outcome results.
The differences in the distance between points of pin insertion into the femoral head, and the axis angle deviation between the actual drilled trajectory and the ideal virtual drilling path in both groups are shown in Table 1.
Secondary outcome.
The GRS scores of participants’ performance in the experiment group (mean score of 26.3) significantly outperformed the control group (mean score of 25.4; p = 0.046). The 10 questions in the knowledge test were divided into 3 subcategories: (i) 4 questions regarding spatial awareness (Q1, 2, 6, 10), (ii) 4 regarding operation details (Q3, 4, 5, 9) and (iii) 2 regarding sequence comprehension (Q7, 8). There was no significant difference between cohorts in the baseline test scores (for any category) before video-learning (Table 2), thus ensuring heterogeneity. Post-learning tests were repeated and are tabulated in Table 2. The 5-point Likert-scale questionnaire scores on satisfaction and acceptability (median ± IQR, 95% Bonett Price CI) in the experiment group (63±8, 59.3–66.7) was 21% significantly higher than the control group (52±7, 48.7–55.3; p = 0.002) as seen in Fig 3. After crossing over and both groups having experienced both video systems, feedback demonstrated that 80% found the multiple POV more helpful to learn the procedure compared to the single view, whereas 17% found it partially helpful and 3% found it not helpful.
Score values are mean (SD) [95% CI].
The box shows the upper and lower quartile. The line in the middle is the median. The whiskers show the upper and lower limits. Crosses represent outliers.
Discussion
Uses of surgical video simulation training
Simulation is already well-established as an effective training medium in the aviation industry [24, 25]. In medicine, camera-based procedures with a single POV can be simulated in a similar fashion. Surgical video learning is a comparatively cheaper and more widely used method for trainees to gain procedural knowledge. Computer-based video instruction has been shown to be an effective indispensable tool for teaching basic surgical skills in numerous studies [26–28]. It offers several advantages over traditional teaching in the operating room, such as accommodating a large number of observers without disrupting clinical flow [29].
A variety of methods are available for capturing video of surgical procedures, but the view is frequently obscured by theatre personnel. In recent years several commercially available head mounted cameras have been used to accurately record surgeons’ view without occlusion, with early positive results [21–23]. There are also reports of Google glass being used for live broadcasting surgeries from the surgeon’s POV, which could be used for surgical training or telementoring without compromising the pace of the operation or patient safety [30, 31]. However, the downside to these head-mounted cameras is a potentially unsteady image, and a single POV. For some surgical procedures, such as joint replacement, single POV is of limited benefit, delivering a working knowledge of the operation, but not helping to develop the choreographed skillset required to perform the procedure.
Modernizing surgical video training
In our study, we developed a novel multiple POV video system, which enabled trainees to visualize surgery from numerous viewpoints. Despite equivalent baseline test scores between the cohorts, the experiment group objectively and subjectively outperformed the control group, with an improved performance accuracy, better performance, post-learning test scores and attitude towards the learning experience. This may be attributable to a better appreciation of the technical intricacies of surgery in a 3D setting. Moreover, multiple POVs avoid loss of visualization secondary to camera occlusions. However, in terms of sub-category scores in the knowledge test post-video learning, there was no difference in sequence understanding scores between the two groups. This suggests that both video systems provided adequate information to learn the sequence of operative steps. When asked to perform part of the procedure using a sawbone simulation, the significantly lower angular deviation observed in the experiment group is an objective demonstration that the benefit of multiple POV translates into improved surgical performance, with a higher procedural accuracy. So, the null hypothesis was rejected, and with the exception of sequence comprehension and distance from optimal pin entry site, the multiple POV was found to be superior to the single POV video system.
A hip resurfacing procedure was chosen for our study for the following two reasons: (i) it is a relatively new procedure of which student or junior doctors are unlikely to have any former experience or knowledge, thus avoiding bias; and (ii) precise pin positioning in the femoral head (a key element to the success of the operation) requires visuospatial awareness and an appreciation of 3D planes from multiple angles.
Despite the advantages of the novel multiple POV, learning to operate using such a system cannot replace the clinical acumen gained from actual hands-on training. Nevertheless, our video system can act as a valuable adjunct to reinforce and enhance the learning experience. Use of this multiple POV video system may help to improve confidence and reduce anxiety before medical students and trainee surgeons move on to apprenticeship-based training in the operating room.
Whether the advantage of this multiple POV will be transferred into other surgical specialties still needs to be evaluated and is worth to be investigated in future studies.
Limitations
Firstly, the multiple POV system appeared to stimulate more interactivity by the candidates in terms of button pressing, which may possibly encourage the participants to watch more footage of the operation than with the single POV. This could have contributed towards the higher scores in the multiple POV group. Secondly, as proposed by the participants, the multiple POV video system does not permit interaction between trainees and instructors and there was a lack of zoom function. Additionally, the Likert-scale questionnaire and the knowledge test had not been previously validated, albeit no validated alternatives exist within the medical literature. However, they were validated by four experienced orthopedic surgeons. Finally, the correlation between objective assessment of visuospatial awareness and precision of femoral head pin positioning was not included in our study, which will be conducted in the due course.
Future prospects.
The benefit of learning surgical procedures using a multiple viewpoint over traditional single viewpoint video has been confirmed in our study. With advances in technology, the next step of our study is to immerse the trainee into a 3D reconstructed theatre setting in virtual world using the Oculus Rift VR headset (Oculus VR, California, USA). This virtual world will allow the trainee to observe an operation from any viewpoint with 3D perception.
Conclusion
We developed a novel multiple POV video system to enhance the learning experience of orthopedic operative steps. The multiple POV provided significant objective and subjective advantages over single POV. Whereas multiple POV video cannot replace the experience of learning from actual surgery, it did enhance and reinforce the technical skills and knowledge to improve confidence, competence and performance metrics.
Supporting information
S1 File. Questionnaire on knowledge of a hip resurfacing procedure.
This questionnaire evaluates the knowledge of a hip resurfacing procedure.
https://doi.org/10.1371/journal.pone.0209904.s001
(DOCX)
S2 File. Questionnaire to evaluate surgical video system.
This questionnaire evaluates the surgical video system.
https://doi.org/10.1371/journal.pone.0209904.s002
(DOCX)
S3 File. Modified GRS.
This questionnaire allows expert to evaluate the participants’ performance using the modified global rating scale, which was designed using a modified Delphi technique, after watching their performance recorded in the video.
https://doi.org/10.1371/journal.pone.0209904.s003
(DOCX)
S4 File. Dataset.
This file contains the dataset of the study.
https://doi.org/10.1371/journal.pone.0209904.s004
(DOCX)
References
- 1. Barnes RW, Lang NP, Whiteside MF, Halstedian technique revisited. Innovations in teaching surgical skills. Ann. Surg. 1989; 210: 118–121. pmid:2742408
- 2. Michelson JD, Simulation in orthopaedic education: an overview of theory and practice. J. Bone Joint Surg. (Am) 2006; 88: 1405–1411.
- 3.
Mendick R, Donnelly L, Kirk A, Medical blunders cost NHS billions. http://www.telegraph.co.uk/news/health/news/11733719/Medical-blunders-cost-NHS-billions.html, 2015 (accessed 16.08.06).
- 4. Chikwe J, Souza AC, Pepper JR, No time to train the surgeons. BMJ 2004; 328: 418–419. pmid:14976074
- 5. Giles JA, Surgical training and the European Working Time Directive: The role of informal workplace learning. Int. J. Surg. 2010; 8: 179–180. pmid:20144752
- 6. Bridges M, Diamond DL, The financial impact of teaching surgical residents in the operating room. Am. J. Surg. 1999; 177: 28–32. pmid:10037304
- 7. Farnworth LR, Lemay DE, Wooldridge T, Mabrey JD, Blaschak MJ, DeCoster TA, et al., A comparison of operative times in arthroscopic ACL reconstruction between orthopaedic faculty and residents: the financial impact of orthopaedic surgical training in the operating room. Iowa Orthop. J. 2001; 21: 31–35. pmid:11813948
- 8. Rafiq A, Moore JA, Zhao X, Doarn CR, Merrell RC, Digital video capture and synchronous consultation in open surgery. Ann. Surg. 2004; 239: 567–573. pmid:15024319
- 9. Cosman PH, Cregan PC, Martin CJ, Cartmill JA, Virtual reality simulators: current status in acquisition and assessment of surgical skills. ANZ J. Surg. 2002; 72: 30–34. pmid:11906421
- 10. Sugand K, Akhtar K, Khatri C, Cobb J, Gupte C, Training effect of a virtual reality haptics-enabled dynamic hip screw simulator: A randomized controlled trial. Acta. Orthop. 2015; 86: 695–701.
- 11. Akhtar K, Sugand K, Sperrin M, Cobb J, Standfield N, Gupte C, Training safer orthopedic surgeons. Construct validation of a virtual-reality simulator for hip fracture surgery. Acta. Orthop. 2015; 86: 616–621. pmid:25885171
- 12. Sadideen H, Hamaoui K, Saadeddin M, Kneebone R, Simulators and the simulation environment: getting the balance right in simulation-based surgical education. Int. J. Surg. 2012; 10: 458–462. pmid:22935356
- 13. Milburn JA, Khera G, Hornby ST, Malone PS, Fitzgerald JE, Introduction, availability and role of simulation in surgical education and training: review of current evidence and recommendations from the Association of Surgeons in Training. Int. J. Surg. 2012; 10: 393–398. pmid:22609475
- 14. Blyth P, Stott NS, Anderson IA, A simulation-based training system for hip fracture fixation for use within the hospital environment. Injury 2007; 38: 1197–1203. pmid:17659287
- 15. Atesok K, Mabrey JD, Jazrawi LM, Egol KA, Surgical simulation in orthopaedic skills training. J. Am. Acad. Orthop. Surg. 2012; 20: 410–422. pmid:22751160
- 16. Rambani R, Viant W, Ward J, Mohsen A, Computer-assisted orthopedic training system for fracture fixation. J. Surg. Educ. 2013; 70: 304–308. pmid:23618438
- 17. LeBlanc J, Hutchison C, Hu Y, Donnon T, A comparison of orthopaedic resident performance on surgical fixation of an ulnar fracture using virtual reality and synthetic models. J. Bone Joint Surg. (Am) 2013; 95; e60, S1-5.
- 18. Pedersen P, Palm H, Ringsted C, Konge L, Virtual-reality simulation to assess performance in hip fracture surgery. Acta. Orthop. 2014; 85: 403–407. pmid:24786902
- 19. Cosman PH, Shearer CJ, Hugh TJ, Biankin AV, Merrett ND, A novel approach to high definition, high-contrast video capture in abdominal surgery. Ann. Surg. 2007; 245: 533–535. pmid:17414600
- 20. Smith R, Day A, Rockall T, Ballard K, Bailey M, Jourdan I, Advanced stereoscopic projection technology significantly improves novice performance of minimally invasive surgical skills. Surg. Endosc. 2012; 26: 1522–1527. pmid:22234585
- 21. Matsumoto S, Sekine K, Yamazaki M, Funabiki T, Orita T, Shimizu M, et al., Digital video recording in trauma surgery using commercially available equipment. Scand. J. Trauma Resusc. Emerg. Med. 2013; 21: 27. pmid:23578301
- 22. Bizzotto N, Sandri A, Lavini F, Dall’Oca C, Regis D, Video in operating room: GoPro HERO3 camera on surgeon’s head to film operations—a test. Surg. Innov. 2014; 21: 338–340. pmid:24363275
- 23. Graves SN, Shenaq DS, Langerman AJ, Song DH, Video capture of plastic surgery procedures using the GoPro HERO 3+. Plast. Reconstr. Surg. Glob. Open 2015; 3: e312. pmid:25750851
- 24. Gateau T, Durantin G, Lancelot F, Scannella S, Dehais F, Real-time state estimation in a flight simulator using fNIRS. PloS One 2015; 10: e0121279.2.
- 25. Tamura A, Wada Y, Shimizu N, Inui T, Shiotani A, Correlation of climbing perception and eye movements during daytime and nighttime takeoffs using a flight simulator. Acta. Otolaryngol. 2016; 136: 433–438. pmid:26808614
- 26. Jowett N, LeBlanc V, Xeroulis G, MacRae H, Dubrowski A, Surgical skill acquisition with self-directed practice using computer-based video training. Am. J. Surg. 2007; 193: 237–242. pmid:17236854
- 27. Nousiainen M, Brydges R, Backstein D, Dubrowski A, Comparison of expert instruction and computer-based video training in teaching fundamental surgical skills to medical students. Surgery 2008; 143: 539–544. pmid:18374052
- 28. Xeroulis GJ, Park J, Moulton CA, Reznick RK, Leblanc V, Dubrowski A. Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery 2007; 141: 442–449. pmid:17383520
- 29. Nagengast ES, Ramos MS, Sarma H, Deshpande G, Hatcher K, Magee WP, et al., Surgical education through video broadcasting. J. Craniofac. Surg. 2014; 25: 1619–1621. pmid:25162553
- 30.
Ramachandran V, Doctor wearing google glass live streams surgery. http://mashable.com/2013/08/30/google-glass-surgery-live-stream/, 2013 (accessed 16.08.10).
- 31. Hashimoto DA, Phitayakorn R, Fernandez-del CC, Meireles O, A blinded assessment of video quality in wearable technology for telementoring in open surgery: the Google Glass experience. Surg. Endosc. 2016; 30: 372–378. pmid:25829065