Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Telemedicine experience of NYC Internal Medicine residents during COVID-19 pandemic

  • Chia-Yu Chiu ,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Software, Validation, Writing – original draft

    Affiliation Department of Internal Medicine, Lincoln Medical Center, New York City, New York, United States of America

  • Amara Sarwal,

    Roles Investigation, Writing – original draft

    Affiliation Department of Internal Medicine, Lincoln Medical Center, New York City, New York, United States of America

  • Muzamil Jawed,

    Roles Data curation, Formal analysis, Methodology

    Affiliation Department of Internal Medicine, Lincoln Medical Center, New York City, New York, United States of America

  • Venkata Sireesha Chemarthi,

    Roles Data curation, Investigation, Methodology

    Affiliation Department of Internal Medicine, Lincoln Medical Center, New York City, New York, United States of America

  • Nehad Shabarek

    Roles Conceptualization, Resources, Supervision, Validation, Visualization, Writing – review & editing

    Affiliation Department of Internal Medicine, Lincoln Medical Center, New York City, New York, United States of America



The COVID-19 pandemic challenged the resilience of public health, including diagnostic testing, antiviral development and transmission prevention. In addition, it also affected the medical education of many residents and learners throughout the country. Historically, physicians undergoing their residency training were not involved in telemedicine. However, in response to the challenges faced due to COVID-19, the Accreditation Council for Graduate Medical Education (ACGME) released a provision in May 2020 to allow residents to participate in telemedicine.


Lincoln Medical Center, located in the South Bronx of New York City, currently has 115 Internal Medicine residents, and telemedicine clinic visits have been conducted by residents since June 2020. An anonymous 25-question survey was sent to all Internal Medicine residents between August 8, 2020 to August 14, 2020.


Of 115 residents, 95 (82.6% of the residents) replied to this questionnaire. Residents revealed feeling less confident in managing chronic diseases through telemedicine visits. The survey also shows that 83.1% of respondents prefer in-person visits during their training, 65.3% feel that the telemedicine experience will affect their future career choice, and 67.4% would prefer less than 50% of visits to be telemedicine in their future careers.


The purpose of the new ACGME rules allowing telemedicine was to prevent the undertraining of residents and maintain health care for the patient during the COVID-19 pandemic. This affects residency training and the experiences of residents, which in turn can influence their future career plans.


The term “Telemedicine” was first introduced and described as “delivery of medical care without the usual patient-physician confrontation.” in 1969. Later on, the term “Telehealth” extended the scope of telemedicine by incorporating a “broader set of activities, including patient and provider education in addition to patient care” [1]. However, the dynamic change in technology allowed for the evolution of telemedicine and currently telemedicine not only includes telephone calls, but also video chats and contact via the internet [2]. In 2010, the World Health Organization came up with a broad description of telemedicine, “The delivery of health care services, where distance is a critical factor, by all health care professionals using information and communication technologies for the exchange of valid information for diagnosis, treatment and prevention of disease and injuries, research and evaluation, and for the continuing education of health care providers, all in the interests of advancing the health of individuals and their communities” [3].

On March 18th 2020, the Accreditation Council for Graduate Medical Education (ACGME) issued a letter to the Graduate Medical Education (GME) community entitled “ACGME Response to the Coronavirus (COVID-19)” which immediately permitted residents/fellows to participate in patient care via telemedicine during the COVID-19 pandemic [4]. On March 20th 2020, ACGME issued a “Clarification regarding Telemedicine and ACGME Surveys” emphasizing that “appropriate level of supervision is in place for all residents/fellows based on each resident’s/fellow’s level of education/training and ability, as well as patient complexity and acuity” for these telemedicine visits [5].

In response to the COVID-19 pandemic, ACGME emphasized a three-stage guideline to maintain medical education and patient care: “Stage 1: Business as usual”; “Stage 2: increase clinical demands guidance”; and “Stage 3: pandemic emergency status guidance” [68]. In “Stage 1: Business as usual”, direct supervision in telemedicine is defined as “the supervising physician and/or patient is not physically present with the resident and the supervising physician is concurrently monitoring the patient care through appropriate telecommunication technology.” It also allows the resident/fellow to conduct a patient encounter remotely and then discuss the case with the supervising faculty member, also through remote means [6, 9]. “Supervision may include post-hoc review of resident-delivered care with feedback” was also mentioned at ACGME Common Program Requirements (Residency) [9].

Although telemedicine provides the flexibility for patient care during such a time, the feedback from trainees is limited. The purpose of this survey is focused on the trainee response of telemedicine and how confident they were with their medicine practice through this new technique.


Study group

Lincoln Medical Center, located in the South Bronx of New York City, is a full-service medical center and teaching hospital. The Internal Medicine Residency program currently has 115 residents, and telemedicine clinic visits have been conducted by residents since June 2020. Prior to scheduling a patient, medical records are reviewed by attending physicians to decide if patients qualify for telemedicine visits or require an in-person appointment. Within the department of Internal Medicine, telemedicine visits completed by residents occur in Endocrinology, Geriatric, Nephrology, Neurology, Gastroenterology, Cardiology and General Medicine clinics. All telemedicine during that study period is follow-up visit instead of new patient visit. At Lincoln Medical Center, residents conduct telemedicine visits exclusively through telephone visits rather than video calls during the study period. In order to assess the quality and feedback of these visits, an anonymous 25-question survey was sent to all Internal Medicine residents between August 8, 2020 to August 14, 2020.

Questionnaire design

There are six ACGME Core Competencies which have been used widely to evaluate resident competencies during residency training, including “Provide Patient-centered Care”, “Medical Knowledge”, “Practice-based Learning and Improvement” “Interpersonal and Communication skills”, “Professionalism” and “System-based practice/Work in interdisciplinary teams” [10].

Our questionnaire was designed to target the assessment of these six ACGME core competencies (S1 Data). Questions 1 to 4 are questions related to demographical information, including gender, postgraduate training year and prior COVID-19 experience of telemedicine inside/outside United States. Questions 5, 6, 7, 9, 17 and 21 focus on “Provide Patient-centered Care”. These questions are resident-oriented rather than patient-oriented, which is unique as most studies in literature focus on asking patients directly about their satisfaction. We ask residents how they perceived their patient’s experience to be when encountered virtually. Question 8 is asking how many phone call attempts were made if the patient did not pick up the phone call on the first try. This is important as there are no regulations in regards of how many attempts need to be performed. Questions 10 to 14 focus on “Resident confidence.” We chose some of the common diseases managed in the outpatient setting, including hypertension, congestive heart failure, diabetes, chronic obstructive pulmonary disease/asthma, and chronic kidney disease. We used these questions to assess how confident the trainees are in practicing their medical knowledge through telemedicine. Questions 15 and 16 are testing “System-based practice/Work in interdisciplinary teams”. Traditionally, in-person visits allow for patients to receive appointments for imaging studies and blood work prior to the completion of their visit. However, with telemedicine, the residents need to coordinate with nursing or clerical staff in order to schedule the imaging studies or blood work, and then one of the team members need to contact the patient again once those dates are confirmed. Questions 18 to 20 focus on “Practice-based Learning and Improvement” by asking trainees about their clinical experience and level of supervision trough telemedicine. Questions 23 to 25 assess the impact of telemedicine experience on internal medicine residents’ career plan.

Statistical analysis

Statistical analysis was completed by using software MedCalc (version 14.12.0; Ostend, Belgium). The Chi-square test was used to examine (1) association between number of telemedicine visits during residency and career decision, and (2) association between number of telemedicine visits during residency and acceptance of the telemedicine practice as a permanent part of a future career. P <0.05 was considered indicative of a statistically significant difference.


Of 115 residents, 95 (82.6% of the residents) replied to this questionnaire. Before the COVID-19 pandemic, 5.3% (5/95) of residents had telemedicine experience outside the United States and 8.4% (8/95) had telemedicine experience within the United States (Table 1).

Table 1. Telemedicine experience of Internal Medicine Residents: Demographics, practice characteristics and career choice (n = 95).

The survey shows that 81% (77/95) of residents will attempt to phone call 2 to 3 times if the patient did not pick up initially while 13.7% (13/95) will only attempt once. Currently, 42.1% (40/95) of trainees are able to conduct 4 to 5 telemedicine visits during 3.5 hours of clinic session while 45.3% (43/95) are able to perform 1 to 3 telemedicine visits. It is worth mentioning that, in our facility, each service distributes telemedicine visits differently. Some services give residents the whole clinic session to do telemedicine visits while some services mix telemedicine and in-person visits during the same clinic session. In the latter clinical scenario, residents need to find spare time between in-person visits to conduct the telemedicine visits.

Compared to in-person visits, a majority of residents responded that they received less clinical experience, less interaction with attendings and less supervision when engaging in telemedicine (Table 2). In addition, residents revealed feeling less confident managing chronic diseases through telemedicine visits in general. Only 2.1% (2/95) of trainees felt the patient received the same level of care when comparing telemedicine visits with in-person visits. None of the residents felt that the patients were always comfortable to discuss their medical condition through telephone and 73.6% of residents (70/95, citing their answer as always/often/sometimes) agreed that telemedicine visits increase the chance of patients lost to follow up. Medication reconciliation and language barrier were two main issues cited by residents.

Table 2. Telemedicine experience of Internal Medicine Residents: Resident satisfaction (n = 95).

This survey showed that 65.3% (62/95) of residents felt that the telemedicine experience will affect their career choice, while 26.3% (25/95) of residents did not feel affected by this experience. However, 83.1% (78/95) trainees preferred in-person clinic visits during their training and 67.4% (64/95) would prefer less than 50% of visits to be telemedicine in their future careers (Table 1).

We used Chi-Square test to examine (1) association between amount of telemedicine during residency and career decision as well as (2) association amount of telemedicine during residency and acceptance of the practice telemedicine when become attending physician. There were no statistically significant differences noted (p = 0.46 and p = 0.38, respectively). The amount of telemedicine conducted by residents seems to not affect career decisions of residents nor acceptance of the practice of telemedicine.

The result of this survey (Tables 1 and 2) and introduction of ACGME telemedicine regulation were presented to all faculty members and residents at a departmental conference. We emphasized that all faculty members should review the cases prior to appointments, in order to avoid complicated cases which are not suitable for telemedicine. We discussed “ACGME Common Program Requirements (Residency)” with all faculty and emphasized the supervision of all residents during telemedicine encounter [9].

Residents also received notification when the lab/imaging order was not completed by patients through the electronic medical system, EPIC, in a timely manner. Residents are able to send messages to the clinic nursing team to facilitate sending a reminder to patients.


The COVID-19 pandemic affected physician training tremendously, in all specialties, not only limited to Internal Medicine. Fortunately, we live in an age where technology is easily accessible, and sharing of information, such as conference and educational material, can be done in a timely manner. The effects of COVID-19 within the health care system have been separated into several fields such as patient care, medical education, and telemedicine [1113]. Telemedicine provides continuity of care for patients while being able to avoid leaving their home, and has been used widely in the United States [14, 15]. ACGME promptly permitted resident involvement in telemedicine in an attempt to minimize education disruption. This will be permitted through the entire 2020–2021 academic year and further extension of this permission is expected.

On the other hand, ACGME acknowledged that “Traditional time-based or volume-based measures may not be fully achievable during this period. Educational experiences may be modified or disrupted through alternative forms of education, such as virtual learning, deployment to another clinical rotation or activity, or by missing a traditionally required rotation” [16]. The integration of telemedicine into residency training was not smooth, and several unanswered questions remain. First, if telemedicine will be a permanent component to residency training programs, the learning goals and expectations of telemedicine need to be established. For example, how much telemedicine clinic experience is appropriate for a residency program, specifically the number of patients a resident should see during one clinic session and if clinic sessions should be exclusively dedicated to telemedicine visits versus a hybrid clinic session with a mixture of in-person and telemedicine visits. Second, even though telemedicine supervision is defined by ACGME, every institution should publish its own guidelines and monitoring methods to ensure the resident receives adequate supervision and safety during training.

In this survey, 65.3% (62/95) of trainees reported telemedicine experience affected their career decision and 67.4% (64/95) of trainees would prefer less than 50% of telemedicine in their future practice. It seems like the telemedicine experience in our program made residents unwilling to practice telemedicine after graduation, but it is worth to know that 26.3% (25/95) reported that the telemedicine experience made no difference on their career decision. Potentially, high volume of telemedicine visits within 3.5 clinic hours could be the most important factor that made our residents dislike telemedicine. However, there is no statistically significant difference on the result. The appropriate patient volume of telemedicine during residency training remains unclear and further evaluation is needed to see how the telemedicine experience during residency affects resident career decision.

Interestingly, even though telemedicine was not common in residency training prior to this pandemic, the United Stated Medical License Examination (USMLE) step 2 Clinical Skills (CS) already introduced telemedicine to examinees [17]. Among 12 cases on the CS exam, examinees potentially will face 1 case of a telephone encounter, usually the case scenario will be the patient’s family phone call for their newborn child. We believe that our residents are not naïve to telemedicine because they practiced telemedicine when preparing for the CS exam. However, CS exam was currently suspended and replaced by 5 pathways due to COVID-19 pandemic.

There are several limitations to this survey. First, this is a single center survey solely within the department of Internal Medicine, and therefore may not be representative of residents in other specialties or other institutions. Second, telemedicine is a new concept to our facility and residents. More time may be needed to let trainees and trainers adapt to the new educational system. However, this survey does help to shed light on the areas that need to be further developed. Third, several questions in this survey did not fit only one domain of ACGME Core Competencies and not every domain of ACGME Core Competencies is covered by our 25-question survey. Standardized surveys will be needed for quality improvement measurement. Fourth, although this survey mainly focusses on trainees, lack of patient input limits the correlation that can be made between trainee perception of patient experience versus patients’ report of their experience. Fifth, telephone visits are only one type of telemedicine and residency training may benefit from other telecommunication technique.

Telemedicine has a large effect on residency training in terms of not only patient care, but resident education and career choice. Since telemedicine will likely become the norm in the foreseeable future, long term monitoring to ensure adequate resident training is warranted and follow up is needed to monitor how telemedicine affects residents’ learning experience or ultimate career plan.

Supporting information

S1 Data. Telemedicine experience of Internal Medicine resident in NYC in COVID-19 era.



We would like to acknowledge the Internal Medicine residents at Lincoln Medical Center who participated this survey.


  1. 1. Bashshur Rashid, Shannon Gary, Krupinski Elizabeth, Grigsby Jim. The taxonomy of telemedicine. Telemed J E Health. Jul-Aug 2011;17(6):484–94. pmid:21718114
  2. 2. Sood Sanjay, Mbarika Victor, Jugoo Shakhina, Dookhy Reena, Charles R Doarn Nupur Prakash, et al. What is telemedicine? A collection of 104 peer-reviewed perspectives and theoretical underpinnings. Telemed J E Health. 2007 Oct;13(5):573–90. pmid:17999619
  3. 3. WHO Global Observatory for eHealth. (‎2010)‎. Telemedicine: opportunities and developments in Member States: report on the second global survey on eHealth. World Health Organization.
  4. 4. Nasca Thomas J. ACGME Response to the Coronavirus (COVID-19). Accreditation Council for Graduate Medical Education. 2020 March 18
  5. 5. Nasca Thomas J. ACGME Response to COVID-19: Clarification regarding Telemedicine and ACGME Surveys. Accreditation Council for Graduate Medical Education. 2020 March 20
  6. 6. Stage 1: Business as usual. Accreditation Council for Graduate Medical Education.
  7. 7. Stage 2: increase clinical demands guidance. Accreditation Council for Graduate Medical Education.
  8. 8. Stage 3: pandemic emergency status guidance. Accreditation Council for Graduate Medical Education.
  9. 9. ACGME Common Program Requirements (Residency). Accreditation Council for Graduate Medical Education.
  10. 10. Quinn Doris C, Bingham John W, Garriss G Waldon, Dozier E Ashley. Residents Learn to Improve Care Using the ACGME Core Competencies and Institute of Medicine Aims for Improvement: the Health Care Matrix. J Grad Med Educ. 2009 Sep;1(1):119–26. pmid:21975718
  11. 11. Ohannessian Robin, Tu Anh Duong Anna Odone. Global Telemedicine Implementation and Integration Within Health Systems to Fight the COVID-19 Pandemic: A Call to Action. JMIR Public Health Surveill. 2020 Apr 2;6(2):e18810. pmid:32238336
  12. 12. Dedeilia Aikaterini, Marinos G Sotiropoulos John Gerrard Hanrahan, Janga Deepa, Dedeilias Panagiotis, Sideris Michail. Medical and Surgical Education Challenges and Innovations in the COVID-19 Era: A Systematic Review. In Vivo. 2020 Jun;34(3 Suppl):1603–1611. pmid:32503818
  13. 13. Ahmed Hanad, Allaf Mohammed, Elghazaly Hussein. COVID-19 and medical education. Lancet Infect Dis. 2020 Jul;20(7):777–778 pmid:32213335
  14. 14. Preparedness Tools for Healthcare Professionals and Facilities Responding to Coronavirus (COVID-19). Centers for Disease Control and Prevention. 2020 July 29.
  15. 15. Hollander JE, Carr BG. Virtually Perfect? Telemedicine for Covid-19. N Engl J Med. 2020;382(18):1679–1681. pmid:32160451
  16. 16. Guidance Statement on Competency-Based Medical Education during COVID-19 Residency and Fellowship Disruptions. 2020 Sep 22. Accreditation Council for Graduate Medical Education
  17. 17. Step 2 CS Content Description and Information. United State Medical Licensing Examination.