https://orcid.org/0000-0002-1161-9648
Figures
Abstract
Background
During critical care procedural skills training (e.g., in intubation and pericardiocentesis) the appropriate supervision level is important to ensure correct use of techniques and guarantee patient safety. The appropriate teaching style should be selected to address residents’ learning behavior and foster their competence. The aim of this study was to explore the number of repetitions for given skills needed to achieve a specified supervision level and a specific teaching style.
Methods
This cross-sectional multicenter survey obtained data from residents and faculty of three multidisciplinary intensive care units (ICU) in Switzerland. Using a 4-point Likert scale, participants were asked to indicate the number of repetitions required to achieve the specified supervision level and teaching style.
Results
Among 91 physicians, the response rate was 64% (n = 59). Their median estimations of the numbers of skill repetitions needed to achieve the final fourth level of supervision and final fourth stage of teaching style were as follows: arterial catheter insertion: supervision level 32, teaching style 17.5; peritoneal paracentesis: supervision level 27, teaching style 17; central venous catheter insertion: supervision level 38, teaching style 28; lumbar puncture: supervision level 38, teaching style 21; endotracheal intubation: supervision level 100, teaching style 45; chest drain insertion: supervision level 27, teaching style 21.5; temporary pacemaker placement: supervision level 50, teaching style 19.5; percutaneous tracheostomy: supervision level 50, teaching style 29; pericardiocentesis: supervision level 50, teaching style 35. Comparison of repetitions between supervision level and teaching style revealed no difference at the first and second levels, except for endotracheal intubation at level 2 (p = 0.03). Differences were observed at the third and fourth levels of supervision level and teaching style (p≤0.04).
Citation: Zante B (2023) Impact of number of critical care procedural skill repetitions on supervision level and teaching style. PLoS ONE 18(1): e0280207. https://doi.org/10.1371/journal.pone.0280207
Editor: Samuele Ceruti, Clinica Luganese Moncucco, SWITZERLAND
Received: February 27, 2022; Accepted: December 22, 2022; Published: January 23, 2023
Copyright: © 2023 Bjoern Zante. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The author received no specific funding for this work.
Competing interests: The author have declared that no competing interests exist.
Abbreviations: SL, supervision level; TS, teaching style; ICU, intensive care unit; PGY, postgraduate year; CBME, competency-based medical education; EPA, entrustable professional activities; SD, standard deviation; CI, confidence interval
Introduction
Theory-driven concepts of medical education differ in their emphasis on the roles of learners (residents) and faculty in the process of learning/teaching [1–4]. Even the concept of competency-based medical education [5] and current critical care training programs pay insufficient attention to the interaction between residents and faculty [6–8]. However, it is particularly important that the appropriate supervision level is applied to ensure correct technique in the performance of skilled tasks and guarantee patient safety [9]. The teaching style should be adjusted according to residents’ individual learning behavior [10].
Depending on residents’ individual experience, different supervision levels may apply, as envisaged in ten Cate’s concept of entrustable professional activities independently of the time in training [11]. In brief, the different supervision levels are as follows: (1) practice skill under proactive, full supervision, as coactivity with faculty; (2) practice skill under reactive/on-demand supervision; (3) practice skill unsupervised; (4) resident supervises more junior colleagues [11].
The Situational Leadership Theory advanced by Hersey et al. (established in the mid-1970s) views effective leadership behavior as an “interplay among (1) the amount of guidance and direction a leader gives; (2) the amount of socioemotional support relationship behavior a leader provides; and (3) the readiness that individuals exhibit in performing a specific task” [12]. Based on this model, faculty should be able to adapt their teaching style to match the residents’ individual learning behavior, which in turn is dependent on the residents’ readiness (combination of ability and willingness; Fig 1).
Four different teaching styles: (1) directing style, (2) coaching style, (3) supporting style, and (4) delegating style are recommended depending on the individual learning behavior of the residents (Table 1) [13].
There is some evidence that the number of skill repetitions has an impact on supervision level and teaching behavior [9,10]. However, the data are limited [9,10,14,15].
This cross-sectional multicenter study was primarily conducted to assess the perception of critical care residents and the respective faculty as to what number of repetitions of critical care procedural skills is needed to achieve distinct supervision levels and for use of the appropriate teaching style. Secondary outcome measures were the individual experience regarding the critical care skills and the perceptions of the difficulty of the critical care skills and the comparison of the perceptions between residents and the faculty.
Materials and methods
Participants
The participants in this cross-sectional multicenter study were residents and faculty in multidisciplinary departments of intensive care medicine at three tertiary care teaching hospitals in Switzerland. The residents’ base specialties were anesthesiology, internal medicine, surgery (all subspecialties), special training in intensive care medicine, and others. Their duration of training in the ICU varied from 6 months or less to more than 12 months. All faculty members were specialists in intensive care medicine. All participants were sent an e-mail with an electronic link to the survey (S1 File).
Survey
The survey was conducted using an online tool (UmfrageOnline, enuvo GmbH, Zurich, Switzerland). The following demographic indices were documented: for the residents, gender, postgraduate year (PGY), duration of training in the intensive care unit (ICU), and base specialty; for faculty, gender and PGY. Furthermore, the residents noted their experience (number of repetitions performed) in the following critical care skills: chest drain insertion, percutaneous tracheotomy, pericardiocentesis, endotracheal intubation, central venous catheter insertion, arterial catheter insertion, lumbar puncture, temporary pacemaker implantation, and peritoneal paracentesis. Experience was classified as number of skill repetitions: zero, 1–5, 6–10, 11–20, 21–50, or >50 repetitions.
Residents and faculty were asked to categorize specified critical care skills according to their difficulty. On the assumption of a wide range of difficulty of the skills, a 9-point Likert scale was used (e.g., insertion of an arterial catheter vs. pericardiocentesis).
The participants were also requested to estimate the number of repetitions required to achieve the following four supervision levels, defined according to ten Cate’s concept of entrustable professional activities [11]:
- Practice skill under proactive, full supervision, as coactivity with faculty
- Practice skill under reactive/on-demand supervision
- Practice skill unsupervised
- Allowed to supervise others
Finally, the participants indicated the number of repetitions required to achieve the specified perceived teaching style. Teaching styles were defined according to Hersey et al.’s Situational Leadership Theory (Table 1) [13].
Ethics
The Ethics Committee of Bern (Kantonale Ethikkommission Bern) waived the need for ethics approval and the need to obtain consent for the collection, analysis, and publication of the data for this study (Req-2021-00079). Participation was voluntary and anonymous. This study adhered to the tenets of the Declaration of Helsinki.
Statistical analysis
Statistical analysis was performed using MedCalc 17.4 (MedCalc Software, Ostend, Belgium). The demographic data were analyzed descriptively. The Spearman rank correlation test was used to compare residents’ and faculty’s classification of skill difficulty and the Mann-Whitney U-test to compare residents’ and faculty’s estimation of skill repetitions according to supervision level and teaching style. Furthermore, comparison of estimated skill repetitions for the supervision level and the level of teaching style was performed. For multiple testing and to control the type-I error rate at the nominal level of 0.05, we applied the Hochberg procedure to correct all p-values that were derived from different models [16]. A two-tailed p-value <0.5 was considered to indicate a significant difference.
Results
Fifty-nine (64%) of 91 physicians responded to the survey. Thirty-three of the 59 participants (56%) were residents and 26 (44%) were faculty members. Overall, 97% (57/59) completed the assessment of skill difficulty, 80% (47/59) completed the assessment of supervision level, and 66% (38/59) completed the assessment of teaching behavior.
Demographics
Forty-five percent (15/33) of the participating residents were female, 55% (18/33) male. The mean PGY was 6.76 years (standard deviation [SD] 2.44, range 3–14 years). The residents’ distribution among base disciplines and their duration of ICU training are shown in Table 2. The residents’ experience regarding skill performance is expressed as number of repetitions (Table 3). Forty-two percent (11/26) of the faculty were female, 58% (15/26) male. The mean PGY for the faculty was 12.76±4.82 years.
Assessment of skill difficulty
The distribution of skill difficulty as assessed by residents and faculty is shown in Table 4. Spearman’s rank correlation indicated no differences between residents and faculty with regard to the ranking of skill difficulty: arterial catheter insertion: p = 0.68, 95% CI -0.35–0.16; central venous catheter insertion: p = 0.59, 95% CI -0.09–0.41; chest drain insertion: p = 0.91, 95% CI -0.23–0.29; endotracheal intubation: p = 0.59, 95% CI -0.13–0.38; lumbar puncture: p = 0.59, 95% CI -0.38–0.13; percutaneous tracheotomy: p = 0.59, 95% CI -0.41–0.1; pericardiocentesis: p = 0.59, 95% CI -0.11–0.4; peritoneal paracentesis: p = 0.69, 95% CI -0.34–0.18; temporary pacemaker placement: p = 0.94, 95% CI -0.27–0.25. Overall, residents and faculty rated the difficulty of these procedures in the following ascending order: (1) arterial catheter insertion, (2) peritoneal paracentesis, (3) central venous catheter insertion, (4) lumbar puncture, (5) endotracheal intubation, (6) chest drain insertion, (7) temporary pacemaker placement, (8) percutaneous tracheostomy, (9) pericardiocentesis.
Skill repetitions and supervision level
No differences were found between residents’ and faculty’s estimations of numbers of skill repetitions with regard to supervision level (S1 Table in S1 File).
Residents’ and faculty’s assessments of the number of repetitions required for the respective supervision levels are summarized in Table 5.
Skill repetitions and teaching style
No differences were found between residents’ and faculty’s estimations of numbers of skill repetitions with regard to teaching style (S2 Table in S1 File). Residents’ and faculty’s assessments of the number of repetitions required for respective teaching styles are summarized in Table 6.
Comparison of repetitions for supervision level and level of teaching style
The estimated number of repetitions required did not differ at the first and second levels of supervision or for the directing and coaching styles of teaching (p-values are given in Table 7), with the exception of endotracheal intubation on the second level and the coaching style (p = 0.03; Table 7). Differences were observed for the third and fourth levels of supervision and the supporting and delegating styles of teaching (p-values are given in Table 7).
Discussion
In this cross-sectional multicenter study we explore associations between on the one hand the number of repetitions of critical care procedural skills and on the other hand supervision levels and teaching styles. Based on critical care residents’ and faculty’s ratings for cut-offs between one level/style and the next, potential learning curves were generated.
The level of supervision applied to ensure correct technique and guarantee patient safety may vary according to the residents’ experience of respective procedural skills. This experience, combined with acquired knowledge, attitudes, and values, may reflect the competence of the residents. In competency-based training it is essential to realize that the focus should be on individual performance rather than time in training, acknowledging that “high-performing” and “low-performing” individuals can both attain the required degree of competence. Importantly, simply performing a large number of skill repetitions does not guarantee competence; here, individual feedback is key to improve the residents’ performance. Furthermore, summative entrustment decisions are needed to guide the residents as they pass through the different supervision levels (levels 1–4) [17] and may leads to improved entrustment levels [18]. This could be done using appropriate checklists [19,20] with individual assistance from the faculty [21].
Besides the conventional bedside training, simulation training may be beneficial for common procedural skills (e.g., central venous catheter insertion) and particularly for less common skills (e.g., pericardiocentesis) [9,22,23].
Only sparse data are available on the numbers of skill repetitions needed to achieve particular supervision levels [14,24]. A recent research synthesis for certain skills (central venous catheter insertion, lumbar puncture, peritoneal paracentesis, and thoracocentesis) concluded that experience does not ensure competence. However, the number of skill repetitions deemed to represent clinical experience was very low (repetition categories 0, 1–2, 3–6, 7–10, 11–20, >21); this may explain why only 10% of the residents passed the assessments with a mean of 48% correct checklist items [25]. Other studies have observed that higher numbers of repetitions ensure safe skill performance [26,27]. Central venous catheter insertion by an experienced physician (≥50 catheterizations performed) is half as likely to result in complications compared with an inexperienced physician (<50 catheterizations) [26]. Physicians experienced in thoracocentesis (>30 repetitions) may perform better than their inexperienced colleagues (<30 repetitions) [27]. However, self-assessed confidence may rise in both experienced and inexperienced trainees after training [28].
Lower numbers of repetitions required for unsupervised execution have been suggested for peritoneal paracentesis [24]. These differences in numbers of repetitions for various skills could be explained by different degrees of skill difficulty, as observed in this study. However, even for more difficult skills such as pericardiocentesis, repetitions may lead to a reduced supervision level [9]. Therefore, the number of repetitions needed to ensure correct technique and guarantee patient safety may depend on skill complexity [29,30].
The recommended number of repetitions for endotracheal intubations in training varies from 33 to 200 [30–32]. As recently shown, the learning curve appears to plateau at 100–200 endotracheal intubations, and this is associated with a significant increase in the lowest oxygen saturation of patients undergoing intubation [33]. This seems to represent a good example of the importance of performing an appropriate number of repetitions of critical care procedural skills in order to achieve correct technique, establish a routine, and guarantee patient safety [34].
Faculty’s teaching style may change according to the residents’ learning behavior and is related to their state of readiness, which in turn depends on their ability and their willingness to perform a skill (Fig 1 and Table 1) [13]. The ability to perform a skill may relate to knowledge, experience, and manual dexterity. These factors may influence the cognitive load and vary directly with the mental effort and inversely the cognitive capacity to perform a skill or task [35,36]. Willingness may be related to residents’ confidence, commitment, and motivation (Fig 1) [37–39]. Both ability and willingness drive learning behavior, which in turn may drive the faculty’s teaching style (Table 1). Therefore, the faculty members can select the teaching style that will best foster the competence of the individual residents [40].
Interestingly, at the lower levels of supervision and teaching style for novices and advanced beginners, no differences in the estimated numbers of repetitions required were observed (except for endotracheal intubation) (Table 7). At levels 3 and 4, corresponding to advanced and proficient residents, higher numbers of repetitions were observed for supervision than for teaching style (Table 7). This could be interpreted as meaning that despite close supervision by the faculty members, their teaching style could be more relaxed. In turn, this may be explained by different purposes and progress of supervision and teaching style: while supervision is required to ensure patient safety and correct technique, faculty’s teaching style addresses residents’ learning behavior. Moreover, it is tempting to speculate that this might be affected by the potential unfamiliarity of the residents with respective skills. More than half (Table 2: internists, surgeons, and others) of the residents start their training in intensive care medicine without profound knowledge and routine regarding the critical care skills. This might lead to the demand for more repetitions, especially to attain the higher levels of supervision and teaching style.
Limitations
This study has several noteworthy limitations. Because it was performed in only three ICUs in one country, the generalizability of its findings is limited. The small sample size and the low response rate mean that the results must be interpreted with caution (particularly in view of the high confidence intervals). Observational studies with larger sample sizes are needed to verify these findings. Furthermore, our data were self-assessed, with all the inherent limitations, and objective learning curves for respective skills must be generated. However, our findings may be useful for setting numbers of repetitions in future research on the supervision levels for critical care procedural skills. An additional focus should be on residents’ learning behavior and its underlying components, i.e., ability (knowledge, experience, and manual dexterity) and willingness (confidence, commitment, and motivation) with implications for the faculty’s teaching style.
Conclusion
It appears that faculty’s supervision level and teaching style should be graduated according to the numbers of repetitions of critical care procedural skills. Knowledge of the association of numbers of skill repetitions with supervision levels and/or teaching styles may support both residents and faculty in learner-centered education.
References
- 1. Levy IM, Pryor KW, McKeon TR. Is Teaching Simple Surgical Skills Using an Operant Learning Program More Effective Than Teaching by Demonstration? Clinical orthopaedics and related research. 2016;474(4):945–55. pmid:26369658
- 2. Willingham DB. A neuropsychological theory of motor skill learning. Psychological review. 1998;105(3):558–84. pmid:9697430
- 3. Shaker D. Cognitivism and psychomotor skills in surgical training: from theory to practice. International journal of medical education. 2018;9:253–4. pmid:30269109
- 4. Kovacs G. Procedural skills in medicine: linking theory to practice. The Journal of emergency medicine. 1997;15(3):387–91. pmid:9258796
- 5. Long DM. Competency-based Residency Training. Academic Medicine. 2000;75(12):1178–83.
- 6. Bion JF, Barrett H. Development of core competencies for an international training programme in intensive care medicine. Intensive care medicine. 2006;32(9):1371–83. pmid:16841214
- 7. Bateman RM, Sharpe MD, Jagger JE, Ellis CG, Solé-Violán J, López-Rodríguez M, et al. 36th International Symposium on Intensive Care and Emergency Medicine: Brussels, Belgium. 15–18 March 2016. Critical care (London, England). 2016;20(Suppl 2):94. pmid:27885969
- 8. Barrett H, Bion JF. An international survey of training in adult intensive care medicine. Intensive care medicine. 2005;31(4):553–61. pmid:15750798
- 9. Zante B, Schefold JC. Simulation training for emergency skills: effects on ICU fellows’ performance and supervision levels. BMC medical education. 2020;20(1):498. pmid:33298042
- 10. Zante B, Klasen JM. Learner-centered education: ICU residents’ expectations of teaching style and supervision level. BMC medical education. 2021;21(1):411. pmid:34330260
- 11. Ten Cate O. Nuts and bolts of entrustable professional activities. Journal of graduate medical education. 2013;5(1):157–8. pmid:24404246
- 12.
Hersey P, Blanchard KH, Johnson DE. Management of Organizational Behavior: Leading Human Resources. Uttar Pradesh, India: Pearson; 2019.
- 13.
Hersey P, Blanchard KH, Johnson DE. Management of Organizational Behavior: Leading Human Resources. 10th ed. Upper Saddle River, N. J.: Pearson; 2012.
- 14. Tariq M, Bhulani N, Jafferani A, Naeem Q, Ahsan S, Motiwala A, et al. Optimum number of procedures required to achieve procedural skills competency in internal medicine residents. BMC medical education. 2015;15:179. pmid:26493025
- 15. Fessler HE, Addrizzo-Harris D, Beck JM, Buckley JD, Pastores SM, Piquette CA, et al. Entrustable professional activities and curricular milestones for fellowship training in pulmonary and critical care medicine: report of a multisociety working group. Chest. 2014;146(3):813–34. pmid:24945874
- 16. Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika. 1988;75(4):800–2.
- 17. Ten Cate O, Schwartz A, Chen HC. Assessing Trainees and Making Entrustment Decisions: On the Nature and Use of Entrustment-Supervision Scales. Academic medicine: journal of the Association of American Medical Colleges. 2020;95(11):1662–9. pmid:32324633
- 18. Valentine N, Wignes J, Benson J, Clota S, Schuwirth LW. Entrustable professional activities for workplace assessment of general practice trainees. The Medical journal of Australia. 2019;210(8):354–9. pmid:30977150
- 19. McGaghie WC, Adams WH, Cohen ER, Wayne DB, Barsuk JH. Psychometric Validation of Central Venous Catheter Insertion Mastery Learning Checklist Data and Decisions. Simulation in Healthcare. 2021;16(6). pmid:33156260
- 20. Mankute A, Juozapaviciene L, Stucinskas J, Dambrauskas Z, Dobozinskas P, Sinz E, et al. A novel algorithm-driven hybrid simulation learning method to improve acquisition of endotracheal intubation skills: a randomized controlled study. BMC Anesthesiology. 2022;22(1):42. pmid:35135495
- 21. Gaubert S, Blet A, Dib F, Ceccaldi P-F, Brock T, Calixte M, et al. Positive effects of lumbar puncture simulation training for medical students in clinical practice. BMC medical education. 2021;21(1):18. pmid:33407416
- 22. Jagneaux T, Caffery TS, Musso MW, Long AC, Zatarain L, Stopa E, et al. Simulation-Based Education Enhances Patient Safety Behaviors During Central Venous Catheter Placement. Journal of Patient Safety. 2021;17(6). pmid:28984729
- 23. Okano H, Mayumi T, Kataoka Y, Banno M, Tsujimoto Y, Shiroshita A, et al. Outcomes of Simulation-Based Education for Vascular Access: A Systematic Review and Meta-Analysis. Cureus. 2021;13(8):e17188. pmid:34414052
- 24. Grabau CM, Crago SF, Hoff LK, Simon JA, Melton CA, Ott BJ, et al. Performance standards for therapeutic abdominal paracentesis. Hepatology (Baltimore, Md). 2004;40(2):484–8. pmid:15368454
- 25. Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Residents’ Procedural Experience Does Not Ensure Competence: A Research Synthesis. Journal of graduate medical education. 2017;9(2):201–8. pmid:28439354
- 26. Sznajder JI, Zveibil FR, Bitterman H, Weiner P, Bursztein S. Central vein catheterization. Failure and complication rates by three percutaneous approaches. Archives of internal medicine. 1986;146(2):259–61. pmid:3947185
- 27. Rasmussen KMB, Hertz P, Laursen CB, Arshad A, Saghir Z, Clementsen PF, et al. Ensuring Basic Competence in Thoracentesis. Respiration. 2019;97(5):463–71. pmid:30625480
- 28. Spencer TR, Bardin-Spencer AJ. Pre- and post-review of a standardized ultrasound-guided central venous catheterization curriculum evaluating procedural skills acquisition and clinician confidence. The Journal of Vascular Access. 2019;21(4):440–8. pmid:31692399
- 29. Nguyen BV, Prat G, Vincent JL, Nowak E, Bizien N, Tonnelier JM, et al. Determination of the learning curve for ultrasound-guided jugular central venous catheter placement. Intensive care medicine. 2014;40(1):66–73. pmid:23974524
- 30. Buis ML, Maissan IM, Hoeks SE, Klimek M, Stolker RJ. Defining the learning curve for endotracheal intubation using direct laryngoscopy: A systematic review. Resuscitation. 2016;99:63–71. pmid:26711127
- 31. Chichra A, Naval P, Dibello C, Tsegaye A, Mayo P, Koenig S, et al. Barriers to Training Pulmonary and Critical Care Fellows in Emergency Endotracheal Intubation Across the United States. Chest. 2011;140(4):1036A.
- 32. Bernhard M, Mohr S, Weigand MA, Martin E, Walther A. Developing the skill of endotracheal intubation: implication for emergency medicine. Acta anaesthesiologica Scandinavica. 2012;56(2):164–71. pmid:22060976
- 33.
Brown W, Janz DR, Russell D, Joffe EM, James DM, Vonderhaar DJ, et al. Effect of Operator Experience on Outcomes of Emergency Airway Management: The ICU Intubation Learning Curve. D25 CRITICAL CARE: HARD TIMES—RESUSCITATING MY PATIENT: FLUID, BLOOD AND OTHER STRATEGIES. American Thoracic Society International Conference Abstracts: American Thoracic Society; 2019. p. A5985-A.
- 34. Sheng AY, Clark A, Amanti C. Supervision of Resident Physicians. Emergency Medicine Clinics of North America. 2020;38(2):339–51. pmid:32336329
- 35. Young JQ, Van Merrienboer J, Durning S, Ten Cate O. Cognitive Load Theory: implications for medical education: AMEE Guide No. 86. Medical teacher. 2014;36(5):371–84. pmid:24593808
- 36.
Sweller J, Ayres P, Klayuga S. Cognitive Load Theory. New York: Springer-Verlag; 2012.
- 37. Taylor DCM, Hamdy H. Adult learning theories: Implications for learning and teaching in medical education: AMEE Guide No. 83. Medical teacher. 2013;35(11):e1561–e72. pmid:24004029
- 38. Lebreton M, Bacily K, Palminteri S, Engelmann JB. Contextual influence on confidence judgments in human reinforcement learning. PLoS computational biology. 2019;15(4):e1006973. pmid:30958826
- 39. Pelaccia T, Viau R. Motivation in medical education(). Medical teacher. 2017;39(2):136–40. pmid:27866457
- 40. Nieboer P, Huiskes M. The regulation of learning in clinical environments: A comment on ’Beyond the self’. Medical education. 2020;54(3):179–81. pmid:31912557