Sample and setting

The consolidated criteria for reporting qualitative research (COREQ) were followed in the reporting of this study [14]. The completed checklist is found in S1 Table. Data was collected from September 2020 to July 2021 through semi-structured individual interviews with junior doctors. Purposive and convenience sampling was used to recruit house officers and residents from different surgical and medical specialties such as Orthopedic Surgery, Internal Medicine and Ophthalmology. To examine the extent to which the curricula have evolved over the years, junior doctors with different years of clinical experience were recruited; mainly, those with one to five years of experience and those with six to ten. Doctors who were trained in different countries were also recruited, as comparing the views of those trained locally and overseas was designed to help us identify the strengths and gaps of Singapore’s medical curricula in preparing students for the digital age. The doctors were recruited through email announcements, department heads and snowballing technique. Waiver of review was granted by SingHealth Centralised Institutional Review Board (Reference Number: 2020/2880).

Data collection

Participants read a study information sheet and had their questions answered before email consent was documented and basic demographic data was collected. All the interviews were conducted in English by researcher HZ over Zoom. Each interview lasted approximately 30 minutes. The interview questions addressed digital technologies listed by Forbes as the latest technology trends that would transform medicine and healthcare in 2020, which included AI, ML and robotics [15], their medical school curriculum and their exposure to any new forms of technology in clinical practice (S2 Table). These were formulated based on the research questions established for this study, as stated in the Introduction, as well as consensus with the team.

During the interview, to avoid the pitfall of under-reporting where the participants’ ability to recall their medical school curricula accurately would limit their responses, we mentioned digital technologies that may be relevant to clinical practice based on our literature review of technological trends that will transform healthcare, as well as search of courses offered by the schools as listed on the schools’ websites. For example, for participants who graduated from National University of Singapore’s Yong Loo Lin School of Medicine (YLL), we asked about their exposure to informatics, if any. This question proved useful as it provided us with insights on the coverage of courses related to digital technologies in the curricula. Mentioning some digital technologies was also necessary because most participants were uncertain about what constituted ‘digital technologies.’ Hence the need to provide them with relevant examples.

In reviewing issues of reflexivity in qualitative research [16], we were made aware of potential researcher bias that might be present due to the established professional relationship between the Principal Investigator and research participants. The interviewer, having no prior relationship with any of the participants, was an important measure to counter the threat of bias. To prevent this from influencing our research, we assigned code identifiers to each participant to ensure anonymity. Codes that begin with ‘HO’ refer to house officers, ‘JR’ refer to junior residents while ‘SR’ stand for senior residents. These acronyms are followed with the specialty that the participants were posted to at the time of interview, and the country where they took their medical degree.

Data analysis

The interviews were transcribed verbatim by a transcriber. The transcripts were reviewed by HZ and Principal Investigator FKY to ensure transcription accuracy. They were analyzed using NVivo 12 concurrent with conducting other interviews until data saturation was reached. HZ and FKY read the transcripts independently and adopted an inductive thematic analysis approach when evaluating the data to draw common and shared meanings among participants [17]. Thematic analysis proved relevant because it involves ‘more than description and categorization, but not extending so far as to develop theory’ [17]. Coding frameworks and themes were developed iteratively using Braun and Clarke’s (2006) six-step process: familiarizing with the data, generating initial codes, searching for key themes, reviewing themes, defining and naming themes and producing the report [18]. HZ and FKY coded the data independently before resolving discrepancies in assessment through consensus.

Data from junior doctors trained in different specialties and who graduated from medical schools were collected to ensure accuracy of results. Credibility was maintained by cross-checking the information provided by the respondents with other key stakeholders. The latter included deans and vice deans of education of local medical schools, clinical educators and other senior consultants working in Singapore’s public healthcare institutions including hospitals and national specialty centers.

Lack of flexibility and dynamism within the curriculum

There was the sentiment that there is a disconnect between contemporary practice and the medical school curriculum, which some respondents such as SR-07 and HO-10, quoted below, perceived as lacking in flexibility and dynamism.

In medical school, we were given a set of rules to follow but in real world medicine, that’s never the case. Sometimes, patients need flexible, individualized treatment. I think the problem lies in medical school being too didactic. Students are not taught how to be flexible and innovative. [SR-07, General surgery, Australia]

In medical school, a lot of the learning is theoretical. Everything is very safe, very structured, and you seem to target patients’ symptoms and medical issues one-by-one. But when you are in the wards, it is different; your patient comes in with multiple problems. Things are more dynamic. [HO-10, General medicine, Australia]

The excerpts prove the need to disrupt traditional educational structures and to strengthen the link between education and practice. The sharing by HO-10 also reinforces the importance of the medical school curriculum to remain relevant with contemporary healthcare needs. This is especially crucial given the growing healthcare needs of Singapore’s ageing population with multiple morbidities.

Dated learning style

Other respondents reiterated the lag between the existing curriculum and evolving trends in clinical settings by highlighting the huge emphasis placed on rote memorization of medical information. On the other hand, digital competencies such as interpreting data on health-related smartphone applications and searching for relevant medical information on digital platforms were not emphasized as much, as cited below:

Medical school education was a bit archaic where we just memorized everything. In reality, a lot of patient management is quite automated, like choosing appropriate treatment, what to order and how to monitor patients. So, there are already structured protocols to consider… Healthcare apps could be useful in tracking the progress of some patients but then, how do you interpret the data? [JR-11, Rheumatology, Singapore]

Rote memorization of a whole list of things is less useful nowadays. It is useful to know the broad principles but realistically, if you do not know, you can still go back to searching it. So what ought to be emphasized more is knowing how to look for information, for example, being able to digest relevant papers and know the implications for clinical practice. [JR-09, Neurology, UK]

Skills in interpreting data from health apps constitute part of broader competencies needed for digital health, a field of medicine that broadly constitutes technologies such as electronic health records (EHRs), telemedicine, wearable devices, health information technology (IT) and personalized medicine [21]. Indeed, studies have shown that medical students, medical school faculty and leaders perceive practical training in data management including big data analysis, data sharing and data security as useful for students [2, 4, 22]. Being equipped with knowledge of the ethical and legal aspects of using digital health technologies as well as communication skills in advising patients to use digital health tools are also considered important [4]. However, just like the medical faculties in countries such as Germany [4], courses on digital health vary in scope and content across Singapore’s core medical school curricula.

Furthermore, skills in searching for relevant medical information has been recognized as important in the need to obtain trustworthy and validated information for patients as well as to reduce burnout among doctors that stem from information overload [23]. To equip students with these skills, training in new content areas such as metacognition, data science, informatics and AI are needed [23, 24]. The need for medical schools to reduce emphasis on memorization of medical information in order to devote more time for these skills has been acknowledged by scholars such as Cutrer et al (2021), Paranjape et al (2019) and Park et al (2019). However, while schools in other developed countries such as the USA have begun to adjust aspects of educational programming in the acknowledgment of rapid expansion of information [24], there remains a lack of scaffolding by medical schools in Singapore to explicitly teach students how to leverage information resources for clinical care.

Limited access to EHRs

Although students were introduced to EHRs used in the hospitals, their lack of access to those used in actual clinical settings interfered with their acquisition of patient charting and medication skills as well as smooth transition to housemanship:

During the Student Internship Programme (SIP), we learnt about the computer systems but our accounts were different from the house officers’. So, there were a lot of things we could not do, like ordering medication. Our notes were also slightly different. There were some we could not access, so we could not write referral letters and all that. When I first started housemanship, I was struggling with all the different software- they are all very different. Took me weeks to get familiarized. [HO-02, Orthopaedics, Singapore]

It would be helpful if we were exposed to the records early enough and given the same account as the house officers’ during the SIP. It would also be good to have a more streamlined process or portal across all hospitals where we use the same software. [HO-05, Paediatrics, Singapore]

The excerpts reflect the importance of providing an authentic learning environment for students in order to enhance practice satisfaction and better prepare them for future clinical practice. Indeed, studies have shown that when students experience firsthand the utility and challenges of using health IT, they are able to hone their clinical decision-making skills, initiate quality improvement and interpret lab results found in patients’ records, among others [25]. Additionally, creating a single type of EHR across multiple training sites would also help to reduce training burden for students.

Gradual uptake of digital technologies in the healthcare sector

Some interviewees also attributed their sense of inadequacy in utilizing digital technologies to the perceived gradual uptake of emerging technologies in healthcare:

We have only gone digital in terms of x-rays, results and documentation. Even so, it took us 20 over years. It is a very slow process. A lot of the things we are doing are actually still very old. [JR-10, Internal Medicine, Singapore]

Others pointed out that the acceleration of digital technology uptake in Singapore’s healthcare system would only be possible with a strong IT infrastructure, which seems to be lacking in the current system:

My main gripe is IT support. When it’s downtime, it’s really disruptive and nobody knows how to remedy. There was once when my account was just hanging for a good six hours. That also means the patients are just lying in recovery without any plans and nobody knows what to do with them. So if you want to be advanced, the IT system has to be reliable and advanced as well. [SR-03, Orthopaedics, Singapore]

The excerpt by JR-10 reiterates our observation that despite Singapore’s digital competitiveness, the healthcare sector is still lagging behind in terms of technological uptake, especially in the realm of clinical care. While reasons for the lag in digital transformation of healthcare may include complexities of the healthcare system and the high stakes involved when technology is adopted in critical decision-making [26], a lack of active adoption of basic technology is also a major barrier to the uptake of more advanced forms of technology.

Additionally, SR-03’s sharing highlights the importance of strengthening the IT infrastructure, as gaps in the infrastructure will interfere with patient care. Studies have also reported that inefficiencies in the healthcare’s IT system have caused burnout and job unsatisfaction among junior doctors [27]. On the other hand, a fully integrated IT system would improve access to information for patients, carers and healthcare professionals [28]. Hence, limitations in IT infrastructure prove to be another barrier that need to be addressed effectively by relevant stakeholders.

Lack of an ecosystem that promotes the culture of innovation

Some respondents opined that there is a lack of ecosystem within local healthcare institutions that promotes a culture of innovation among medical trainees. They attributed this to the lack of time brought about by a heavy workload and of an open mindset towards medical innovation in general, as exemplified by JR-10 and SR-01, respectively:

Our healthcare system overvalues efficiency and ability to take a heavy workload, so, most of the time, junior doctors are just stuck doing work. The system needs to be less of a pressure cooker so that doctors will have time to explore other interests (beyond clinical work). [JR-10, Internal Medicine, Singapore]

When I was trying to put up a grant for a medical innovation project, I met with some resistance from those in the senior level who questioned why I was doing that project, who was going to access the online portal I wanted to create. They believed the patients and doctors are not going to use it, but I think change has to start from somewhere. If we are willing to start making a change, that is how things will advance [SR-01, Obstetrics and Gynaecology, Ireland].

The above excerpts demonstrate that while there may be interest among medical trainees to engage in medical innovation projects, several systemic barriers remain. There has to be a mindset shift in terms of how the healthcare system treats its manpower resources, as it currently appears to tie the worth of labour to productivity, which may inadvertently reduce the creativity and diversity of talents within the healthcare workforce. The attitudes of healthcare leaders towards projects involving technology or innovation should also change given their role in developing and maintaining the governance and standards of doctors. Moreover, broader digital technology initiatives by medical schools and healthcare institutions should trickle down to the core undergraduate curricula, as current sustained efforts to train budding physicians in digital technologies are largely limited to postgraduate programs [29].

Lack of guidance from qualified and available mentors

Those who were interested in the specifics of digital technologies pointed to the lack of perceived expertise and guidance from seniors who were involved in technological innovation projects:

When I asked a Consultant for details on how a triage that involved AI worked, all she could tell me was, there was some AI involved. This is what happens when the average doctor who may not be that interested in these technologies, work on such projects. [JR-10, Internal Medicine, Singapore]

Some respondents also highlighted that their limited training in digital technologies used in the wards, coupled with a lack of proper guidance, proved challenging in their clinical practice, as illustrated by SR-01 below:

We were not trained to perform bedside ultrasounds so when I first started out in O&G, whenever a patient came to the emergency clinic, if it was acute abdomen, we would send them to the radiology department or reschedule for them to come back to outpatient clinic where they can get a formal scan… and when I was a medical student in Ireland, we weren’t really allowed to perform any procedures, like setting IV cannulas for patients. So, when I first started out as a HO, I absolutely had no idea how to set a plug. I had a nurse to teach me that. It’s often difficult to find a senior to supervise you because they will be really busy. [SR-01, Obstetrics and Gynaecology, Ireland].

The above excerpts reiterate the need for a mentoring system and formal training in digital technologies for trainees. In addition to equipping trainees with the necessary competencies, clinical educators and mentors themselves have to be knowledgeable in these fields. The availability of qualified mentors would also help to foster a culture of innovation among trainees.

The findings show that the gaps in the training of digital technologies within the medical school curricula are largely intertwined with those in actual clinical settings and should not be addressed in silos. Specifically, they demonstrate how medical education, the clinical setting and the healthcare system work as a continuous feedback loop that inform and enhance one another. In so doing, the study reiterates the importance of multiple stakeholders in strengthening the medical school experience.

Encourage a shift in assessment of competencies and adaptive learning

In response to concerns about the lack of flexibility in the curriculum, medical schools need to reexamine the way in which students’ clinical competence is assessed. Currently, students are largely tested on their theoretical knowledge, and medical examinations are heavily focused on the basic sciences. A redesign of the curriculum could involve curriculum expansion and a review of assessment strategies. For example, there should be an increase in the assessment of creativity, critical thinking skills and concepts that would contribute to students’ understanding of digital technologies. Students should also be tested on the fundamentals of data science and topics related to digitalization in their final examinations.

Additionally, to address the concern about the lack of dynamism in the curriculum, medical schools need to equip students with adaptive skills. Cutrer et al (2021) have highlighted the value of a curriculum that gears learners to learn and innovate in response to practice challenges rather than one that focuses primarily on the delivery of content [24]. For example, at Harvard Medical School, which has reorganized its entire curriculum using a case-based collaborative learning model as opposed to a performance-oriented culture, students are valued for their reasoning skills rather than just for correct answers [24]. At Vanderbilt University School of Medicine, students are required to identify knowledge gaps in the course of patient care and present their findings to a clinical team that will assess their skills in information appraisal and application [24].

Provide dedicated training in digital competencies

To ensure that the school curricula remain up-to-date and are aligned with the needs of the healthcare system, a dedicated training in digital competencies for clinical practice is needed. One way is to reduce the duration on basic sciences education, especially those that require students to memorize a barrage of medical information, and increase the time spent on educating them about the fundamentals of digital technologies instead. Research have shown that longitudinal curricula that span the duration of medical school would expose students to a wider range of skills [7, 4]. For instance, Hersh et. al. (2017) highlight that an undergraduate medical curriculum that includes a comprehensive set of competencies in clinical informatics would expose students not only to EHRs but also to telemedicine and personalized medicine as well as other useful skills in clinical decision support and quality improvement [7]. Likewise, Foadi et. al. (2021) show how integrating digital competencies longitudinally into the curriculum can equip students with skills such as handling of medical data, knowledge of digital infrastructure of the health system, application of data in patient care and preventive medicine, ethical and medico-legal basics and transformation processes in medicine brought about by digitalization [4].

To maximize effective implementation of the aforementioned skills, educational systems, program structures and objectives need to be revised in order to create new learning outcomes [7]. A systematic curricular reform would also require medical schools and healthcare professionals to incorporate digital competencies in their ongoing dialogue on medical education [42]. During clinical rotations, medical experts should consciously introduce students to relevant applications of digital technologies in practice.

Expand students’ access to EHRs

To increase students’ access to EHRs, a concerted effort of multiple stakeholders is needed. At the regulatory level, medical innovators should create EHRs that are the same as those used in actual clinical settings while recognizing the need for protection of confidentiality and regulation of access for specific purposes. For instance, deidentified patient data can be used to encourage students to practice writing orders, entering notes, reviewing data and formulating a care plan, as has been adopted in the US [33]. At the institutional level, schools should devise policies on the placement of students’ notes that, while compliant with all regulations, balance the need for patient privacy, liability concerns and the need for students to become competent with EHRs [33]. At the national level, a move towards interoperability of systems that allow users to share data would facilitate students’ adaptation to new systems in different healthcare settings [33].

Optimize digital technologies in healthcare

To address the perceived lag in the uptake of digital technologies in healthcare, hospitals and policymakers would first need to ensure that technological equipment and processes in clinical settings are up-to-date and operating at maximum efficiency. Hospitals would also benefit from obtaining medical innovators’ advice on ways to optimize digital technologies in diagnosis, treatment and overall patient care as well as reduce inefficiencies in poorly designed systems. A technology assessment committee could also be formed to develop clear policy guidelines that would enable medical students and healthcare professionals to utilize digital technologies safely and effectively without compromising patient care.

Foster a culture of innovation

In response to the feedback on the lack of an ecosystem that promotes the culture of innovation among medical trainees, we recommend a better balance between the provision of high-quality care to patients and a more efficient management of manpower resources in healthcare institutions to allow time for doctors to pursue projects that involve digital technologies. Protected time to mentor trainees in medical innovation projects should also be offered to qualified candidates. For students interested in medical innovation projects, healthcare institutions could offer them dedicated training during a gap year in their clinical years. Additionally, relevant recognition in the form of awards or opportunities for collaboration with clinicians involved in digital technology projects should be given to trainees who have demonstrated sufficient potential.

Provide mentorship to students

Lastly, to address trainees’ concern about the lack of guidance from qualified seniors, strong mentorship from competent medical educators is necessary. Interdisciplinary collaboration and networking among educators could help address the need for combined expertise in digitalization, medical education and medical science [36]. This would help to ensure the inclusion of competence from the widest range of specializations [36]. For doctors with little or no experience with digital technologies, they could undergo training via Continual Medical Education (CME) programs and learn from those outside of the medical community. All in all, proper mentorship from qualified educators is important as studies have shown that technological savviness among younger physicians does not necessarily translate into skills transferability to work with patients and in the professional medical setting [37, 43].