Introduction

The role of assessment in shaping student behaviors has been well-documented in medical education literature, including driving a performance-mindset with significant impacts on learning [1–3]. This is particularly true for high-stakes assessments such as the United States (U.S.) Medical Licensing Examinations (USMLE). For example, in the U.S., as competition for residency positions from both domestic and international medical students has amplified, program directors increasingly use performance on USMLEs for interview selection, and this has had a significant impact on student behaviors approaching these exams [4, 5]. These exams were not developed for this purpose; consequently, extrapolation of this assessment data to success in residency lacks strong evidence [6–9]. Historically, more emphasis was placed on USMLE Step 1 compared to Step 2 CK, with many fields using score thresholds to filter applicants [10–12]. This has resulted in what many have called Step 1 “exam mania” [3, 10, 12], as students spend a large portion of their studies engaged in a parallel curriculum developed by third-party resources, and wellbeing is sacrificed in the face of enormous pressure to perform well.

For those who do not obtain their desired score on the first USMLE Step 1, Step 2 CK traditionally offered an opportunity to demonstrate improvement. While several specialties have already been relying on Step 2 CK scores more heavily [5], this phenomenon is only likely to increase given the recent change by the National Board of Medical Examiners (NBME) to pass/fail scoring for Step 1 [13–15]. Concerns from various stakeholders have highlighted a transition of “exam mania” from Step 1 to Step 2 without addressing the underlying systemic issues that led to exam mania in the first place, namely the predominant role that high-stakes exams increasingly play in medical education. Yet, relatively few studies have focused on student behaviors and attitudes towards Step 2 CK. Now, in the era of pass/fail Step 1 and with the growing importance of Step 2 CK, this gap is of particular relevance and may help us understand the processes that medical students undergo when approaching high-stakes examinations more broadly.

For USMLE and other high-stakes exams, much of what we know about preparation has been explored through the parallel curriculum [3, 16, 17], consisting of external resources used to maximize academic performance in lieu of materials developed by medical school faculty. This curriculum develops out of desire, above all else, to perform well on USMLE, and is a direct outgrowth of “exam mania” [3, 16, 17]. Yet, less attention has been paid to the educational systems that contribute to the phenomenon of “exam mania” in the first place.

Originally described in the medical education literature by Frederic Hafferty in 1994, formal, informal, and hidden curricula heavily influence the medical learning environment (Table 1) [18, 19]. The formal curriculum is offered and endorsed by the institution (e.g., syllabi and learning objectives). The informal curriculum represents interpersonal learning (e.g., through casual conversations between students and mentors). The hidden curriculum represents the norms and values embedded in organizations and their practices that drive unintentional or subliminal teaching [19]. This includes how organizations evaluate student performance, how funding and resources are allocated, the proportion of time spent on particular curricular topics, and how language is employed to describe the curriculum and expected competencies of learners. Most research on the hidden curriculum focuses on clinical experiences [20]. In contrast, here we apply the hidden curriculum as a conceptual framework to understand student behaviors and attitudes approaching high-stakes examinations—in this case, the USMLE.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Curricular frameworks found within undergraduate medical education.

https://doi.org/10.1371/journal.pone.0279911.t001

Using focus groups of fourth-year medical students, we employ qualitative, constructivist methods to explore the complex decision-making processes and emotion work [21] students face while navigating preparation for high-stakes medical licensing examinations. Despite an initial focus on Step 2 CK, participants’ experience of “exam mania” regarding the Step 1 exam dominated the discussion, with students highlighting lessons learned from the informal and hidden curricula about the balance of resources, time, and mental energy to spend on these exams and their overarching effects on undergraduate medical education (UME). We focused predominantly on the informal and hidden curriculum as these often rely on implicit messaging that can subconsciously influence student decision-making, compared to the more explicit formal curriculum. Our work comes at a critical time as the U.S. considers the role Step 1 and Step 2 CK will continue to play in the UME-graduate medical education (GME) transition in the new era of a pass/fail Step 1, and how “exam mania” continues to affect medical student decision-making and emotion work more broadly.

Materials and methods

Six focus groups of fourth-year students were conducted at the University of Illinois College of Medicine (UICOM)–Chicago campus from November 2018-February 2019. Of note, this study was conducted prior to the transition of USMLE Step 1 to pass/fail scoring, which occurred on January 26, 2022. We supplemented these focus groups with content analysis of educational materials to contextualize our understanding of curricular offerings related to USMLE and to enhance the rigor of our findings [22]. We chose focus groups, rather than one-on-one interviews, as our primary data source given our interest in dialogue generated between students to highlight experiential similarities and differences and because focus groups are useful to provide deep insight into complex perspectives [23–26]. Sessions were conducted until sufficient information power was obtained, i.e., until we had enough rich data relevant to our study aim to address our research question [27, 28]. This study was approved by the UICOM-Chicago institutional review board (Protocol #2017–0482) and adhered to the Standards for Reporting Qualitative Research (SRQR) recommendations [29]. The need for informed consent was waived by the ethics committee.

Focus groups were facilitated by the senior author (LEH), a medical educator with expertise in qualitative methods, and each session was assisted by senior medical students with lived experience (JRG, SMR, HR) and national leadership positions related to USMLE (JRG). Our team also included a faculty administrator with experience in residency selection, career advising, and admissions (TT). This diverse research team provided a variety of perspectives about the research topic during all phases of the research study, which was particularly useful in ensuring thorough data collection and rigorous analysis. We used purposeful sampling of students from the entire class following initial recruitment via the class listserv. Eligible participants were current fourth-year students who had already taken Step 2 CK and included a broad representation of intended specialty choices. Focus groups were conducted in familiar classrooms; participants were provided lunch during the session and entered a lottery for a $25 gift card. We developed a semi-structured facilitator guide with open-ended questions that focused on (1) information and strategies about when to take Step 2 CK and how previous academic factors such as Step 1 influence this, and (2) emotions related to Step 2 CK, including preparation, advice, and outcome (S1 Appendix). Sessions were one-hour long, audio-recorded, transcribed, and de-identified. Data was entered into NVivo (Version 12.4.0, QSR International Inc., Burlington, MA, USA).

We used constructivist qualitative methods to analyze results [30, 31]. Specifically, rather than beginning with a hypothesis, we reviewed our data and noted salient trends using a theory-informing, inductive study design [32]. In other words, after several iterative rounds of data collection and analysis, we identified the concepts of informal and hidden curricula as a relevant feature of our data to shape final interpretations. Each transcript was read and coded independently by at least two authors (JRG, SMR, HR) using open-and-focused coding [33]. The team met to finalize codes, identifying whether quotes best described formal, informal, or hidden curricula. We explicitly selected these conceptual frameworks as, unlike the parallel curriculum, they have not previously been explored in the context of high-stakes examinations. Any disagreements were discussed until consensus was reached. Once codes were finalized, two authors (JRG, SMR) drafted analytic memos to illustrate dominant themes. These themes were discussed with the senior author (LEH), and overlapping concepts were merged into integrative memos based on the relationship of themes to one another [33].

Results

Twenty-three students participated in six focus groups (median 4–5 students per session). 15 (65.2%) participants were women and 8 (34.8%) were men. The most common intended specialties included emergency (21.7%) and internal medicine (17.4%).

Overall, we found that formal, informal, and hidden curricula collectively contribute to “exam mania” as medical students approach high-stakes licensing examinations. Students are driven towards the belief that these high-stakes examinations are the most important factor in medical school, and all other examinations and experiences are secondary.

Formal curriculum

Previous student-led efforts to improve USMLE preparation both within and parallel to the formal curriculum have been published [34, 35]. Notably, the concept of formal curriculum could also relate to the USMLE exams themselves, although typically it has been reserved for content offered specifically by the institution, which may or may not align with high-stakes licensing examinations. Analysis of offerings at UICOM related to Step 1 and Step 2 CK revealed that most resources provided by the administration focus on Step 1 (Table 2). Most are optional or recommended, but not part of the formal curriculum of the institution. The misalignment of formal curriculum and Step 1 content encouraged pursuit of a parallel curriculum, a phenomenon observed with other high-stakes examinations worldwide [3]. However, focus group participants noted that formal activities and resources during the clinical phase of education more frequently aligned with the content of Step 2 CK, resulting in decreased drive to engage in parallel curricula for this exam.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Content analysis of parallel curricular materials offered to medical students at A single institution for preparation for USMLE examinations.

https://doi.org/10.1371/journal.pone.0279911.t002

Discussion

High-stakes assessments create unintended, yet potentially damaging pressures on medical trainees and have a significant impact on the medical education community and result in “exam mania.” While these phenomena may be familiar to medical students and administrators, we lack high quality, empirical data to support these observations. The USMLE is one such high-stakes examination taken by both U.S. and international medical students worldwide. Here, we provide data showing that the formal, informal, and hidden curricula unite to send a single message to medical students: high-stakes exams are the most important part of medical education. Indeed, the formal, informal, and hidden curricula are always at play together to create an educational environment that influences student behaviors. The prioritization of exam scores has important consequences on professional identity, career aspirations, and interpersonal interactions, in addition to the outgrowth of parallel curricula with additional financial burden placed on trainees. The medical student learning environment is filled with conflicting messages about what is most important, and as medical schools lose control over their own curricula as they seek to prepare students for these external exams, trust in the education they provide may erode [8].

These consequences contribute to emotional and intellectual exhaustion, resulting in burnout as students are beginning their careers. Increased burnout amongst physicians is correlated with lower empathy, and negatively impacts professionalism and professional identity [44]. Informal and hidden curricula around high-stakes examinations can increase feelings of guilt and potentiate burnout. Institutions should adopt new strategies to improve preparation and readiness for these exams, in particular the Step 2 CK exam in the era of pass/fail Step 1. At the same time, institutions, including residency program directors, must not permit this “exam mania” to simply transfer to Step 2 CK. Interventions in the learning environment can improve student well-being, and national efforts are currently underway to address this [45, 46]. In addition, addressing issues related to interpersonal support while preparing for exams may also prevent burnout [47]. Finally, the phenomena identified here related to high-stakes standardized examinations are part of a larger discussion related to the UME-GME transition, and indeed recent efforts led by the Coalition for Physician Accountability (CPA) have sought to address this wide-reaching problem by bringing in voices from various stakeholder organizations [48]. For example, final recommendations from the CPA’s UME-GME Review Committee (UGRC) released in August 2021 have established guidelines for addressing inequities in the UME-GME transition, reducing financial burden on applicants, and cautioned against an “assessment rat race” [49, 50].

To our knowledge, this is the first study to apply the theories of formal, informal, and hidden curricula as a conceptual framework to understand student approaches to high-stakes licensing exams (Table 1). We provide the student voice with insight into decision-making and emotion work approaching these exams. Our data also offers valuable guidance for the next phase of USMLE where Step 1 is pass/fail but the role of Step 2 CK remains uncertain [14]. Future studies should re-examine perspectives after several years of pass/fail scoring, especially if the emphasis does shift to Step 2 CK as many have predicted. More broadly, our findings extend beyond U.S. allopathic medical students, including osteopathic medical students who routinely take USMLE in addition to the Comprehensive Osteopathic Medical Licensing Examination of the U.S. (COMLEX-USA) [51, 52] and international medical graduates (IMGs) who rely on USMLE performance to distinguish themselves and secure residency positions in addition to passing rigorous board examinations in their home countries [53, 54]. This is a time for the medical education community to rise to the challenge and offer innovative approaches to curriculum and assessment.

Our study has some limitations. We conducted our study at a single institution, and despite reaching sufficient information power [26, 27], the number of participants involved was modest; however, these smaller groups facilitated increased depth and breadth of discussion amongst our diverse participants. While our approach utilized students from our institution as a case study to provide data on broad trends in medical education related to high-stakes examinations, future studies may seek to involve participants from multiple institutions or investigate how intended specialty influences perspectives. Additional studies may also seek to investigate perspectives of both U.S. osteopathic medical students and IMGs, as well as how foreign high-stakes exams differ between those routinely taken in the U.S.

In the context of U.S. medical education, the transition of Step 1 to pass/fail scoring will have significant effects on student approaches to high-stakes licensing exams. Some suggest that the competitive pressures and “exam mania” will simply shift downstream to Step 2 CK [14, 15], making our study even more relevant. It is important to recognize how informal and hidden curricula influence these exams. Institutions should pay careful attention to the messages sent by informal and hidden curricula on the importance of high-stakes licensing exams, and develop formal curricula to ensure that students have adequate preparation and opportunities beyond test scores to demonstrate their capability.

Conclusions

The unintended pressures that high-stakes examinations place on medical students can have profound effects on professional identity and drive increased burnout amongst trainees. As the U.S. transitions to the new era of pass/fail Step 1 and contemplates what role Step 2 CK will play, we must pay careful attention to preventing these unintended, but highly influential effects on medical student decision-making and emotion work.

Supporting information

Acknowledgments

The authors would like to thank Kenji R. Kobayashi, MD, and Ariana Melendez, MD, who were involved in the initial conversations for this study and helped provide valuable insight. We thank Nikita Williams for her assistance in transcription of focus group recordings. We would also like to thank Raymond H. Curry, MD, Senior Associate Dean for Educational Affairs, for his support of and valuable feedback on this study.