https://orcid.org/0000-0002-0448-6349
Figures
Abstract
Objective
To evaluate the use of a COVID-19 app containing relevant information for healthcare workers (HCWs) in hospitals and to determine user experience.
Methods
A smartphone app (Firstline) was adapted to exclusively contain local COVID-19 policy documents and treatment protocols. This COVID-19 app was offered to all HCWs of a 900-bed tertiary care hospital. App use was evaluated with user analytics and user experience in an online questionnaire.
Results
A total number of 1168 HCWs subscribed to the COVID-19 app which was used 3903 times with an average of 1 minute and 20 seconds per session during a three-month period. The number of active users peaked in April 2020 with 1017 users. Users included medical specialists (22.3%), residents (16.5%), nurses (22.2%), management (6.2%) and other (26.5%). Information for HCWs such as when to test for SARS-CoV-2 (1214), latest updates (1181), the COVID-19 telephone list (418) and the SARS-CoV-2 / COVID-19 guideline (280) were the most frequently accessed advice. Seventy-one users with a mean age of 46.1 years from 19 different departments completed the questionnaire. Respondents considered the COVID-19 app clear (54/59; 92%), easy-to-use (46/55; 84%), fast (46/52; 88%), useful (52/56; 93%), and had faith in the information (58/70; 83%). The COVID-19 app was used to quickly look up something (43/68; 63%), when no computer was available (15/68; 22%), look up / dial COVID-related phone numbers (15/68; 22%) or when walking from A to B (11/68; 16%). Few respondents felt app use cost time (5/68; 7%).
Conclusions
Our COVID-19 app proved to be a relatively simple yet innovative tool that was used by HCWs from all disciplines involved in taking care of COVID-19 patients. The up-to-date app was used for different topics and had high user satisfaction amongst questionnaire respondents. An app with local hospital policy could be an invaluable tool during a pandemic.
Citation: Helou RI, Waltmans–den Breejen CM, Severin JA, Hulscher MEJL, Verbon A (2022) Use of a smartphone app to inform healthcare workers of hospital policy during a pandemic such as COVID-19: A mixed methods observational study. PLoS ONE 17(1): e0262105. https://doi.org/10.1371/journal.pone.0262105
Editor: Surbhi Leekha, University of Maryland School of Medicine, UNITED STATES
Received: May 11, 2021; Accepted: December 16, 2021; Published: January 5, 2022
Copyright: © 2022 Helou et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work has been made possible by a grant from Joint Programming Initiative on Antimicrobial Resistance (JPIAMR2017-045) https://www.jpiamr.eu/ The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In the first months of the COVID-19 pandemic, and even more thereafter, an overwhelming number of publications appeared on COVID-19. Especially in the first months, clinically relevant new findings were published on a daily basis while healthcare workers (HCWs) and hospital management struggled with an enormous influx of COVID-19 patients in the hospitals. Guidelines which assist HCWs in clinical decision making have shown to have a beneficial effect on therapy, length of hospital stay, mortality and overall patient safety [1, 2]. However, for COVID-19, these had to be changed regularly and often on a daily basis. Since clear, centrally directed communication during a healthcare crisis is key, hospitals should ensure correct and current policy is disseminated in a concise manner to all HCWs [3, 4]. HCWs have to be able to rely on the latest information provided, thereby ensuring good quality outbreak response.
Information technology such as clinical decision support systems (CDSS) can positively impact healthcare by facilitating diagnostic [5], therapeutic [6] and prognostic [7] processes. CDSS is software that facilitates clinical decision making, either by comparing individual patient characteristics with a clinical knowledge database or by providing relevant information which is then combined with the knowledge of the clinician [8]. Such systems also have shown value in the COVID-19 outbreak management setting [9–11]. A multitude of smartphone applications (apps) for patients have been developed and evaluated during the COVID-19 pandemic with focus on contact tracing [12, 13], self-testing [14], self-reporting [15] and education [16]. While these apps can help inform the public, professionals, and policy-makers, they are dependent on user uptake and continued engagement. In addition, ethical issues must be addressed to enable professionals and policy-makers to effectively use the data collected while protecting the privacy of app users. Few apps specifically for HCWs, some including CDSS, have been studied and these were limited to instruction videos [17], policy documents [18] or the addition of COVID-19 content to an existing platform containing an array of clinical medical content [19]. Although apps may have high potential to distribute the latest vital hospital information and policy on COVID-19 amongst HCWs, few studies evaluated the use of such apps focusing solely on COVID-19. The aim of this study is to evaluate the use of a COVID-19 app containing relevant information (such as local infection prevention guidelines, clinical guidelines, instructions on the use of personal protective equipment (PPE) and frequently asked questions) for HCWs in hospitals and to determine user experience.
Methods
Setting
This study was conducted in the Erasmus MC University Medical Center (Erasmus MC), Rotterdam, The Netherlands, an academic, tertiary care hospital with approximately 900 beds and 14,359 HCWs including 2,512 nurses and 935 medical specialists. The overall number of COVID-19 admissions in the hospital was monitored by the central hospital administration. Updates in local policy of the Erasmus MC were published on the intranet of the hospital, and communicated via the heads of the departments.
Eligibility criteria
All HCWs of the hospital with an institutional e-mail address were eligible to use the app.
Intervention
App.
The clinical decision support platform Firstline [20], was adapted to local conditions with exclusively SARS-CoV-2 / COVID-19 content from the Erasmus MC: the COVID-19 app, using Firstline’s cloud-based content management system (CMS). The mobile COVID-19 app was available for iOS and Android. In order to avoid inappropriate off-site usage of locally-customized guidelines, potential users of the mobile COVID-19 app were required to authenticate using their institutional email address.
Content.
Content included local infection prevention guidelines, clinical guidelines (diagnostic and therapeutic, including drug monographs), instructions on the use of personal protective equipment (PPE) such as face masks, goggles, face shields and medical gowns, COVID-19 related phone numbers of the hospital which could be dialed instantly and frequently asked questions. All content was non-confidential and available to all HCWs. App updates coincided with updates in infection prevention and clinical guidelines, which sometimes took place on a daily basis. Updates were created and reviewed in a restricted environment and were only released to end users upon approval. Therefore, updates in the app may have been published several minutes later than the content on the hospital’s intranet. Additionally, updates were most frequently uploaded on the intranet and in the app in the evening. To ensure that users could easily access the latest updates, the top page of the COVID-19 app -named ‘latest updates’- was dedicated for this purpose with a chronological overview.
Recruitment.
A newsflash and an article about the COVID-19 app and its purpose was published on the intranet of the hospital on March 31, 2020. After one day on the front page the article moved down in the chronological newsfeed. A link in the article redirected potential app users either to iTunes (Apple) or the Play store (Google). Five weeks after app launch, a second article was published on the intranet informing HCWs about the app and its use in the first month (May 6, 2020). Six weeks after app launch, users were asked to fill out a questionnaire to evaluate their user experience with the app in a sticky message on the app home screen with a link to the online questionnaire. This message was shown until the end of the study on July 8, 2020. Simultaneously, a push message was sent to all app users informing them about the questionnaire study and asking for their participation. Additional e-mails were sent to app user groups (based on role) with a relatively low response rate during preliminary analysis.
Outcomes.
Primary outcomes were 1) process indicators -including number of app downloads, session time, and accessed pages- which were anonymously collected and made available on a secure dashboard by Firstline and 2) user experience, focusing on barriers and facilitators of app use and use in general during the COVID-19 pandemic.
Questionnaire.
User experience was evaluated in an online questionnaire using LimeSurvey v2.06, an open-source online survey software. The questionnaire contained 23 questions and was developed by the authors based on interviews from a pilot study with an antibiotic stewardship app [21] and available literature [22–25]. In this pilot study 18 purposefully sampled potential users from several departments in three different countries (Sweden, Switzerland and The Netherlands) were interviewed. After two weeks of app use each user was interviewed about their experiences. Additional users were interviewed until information saturation was reached. The interview questions were based on the Flottorp framework, which is designed to evaluate barriers and facilitators of an intervention [22]. For analysis of the interviews the software Atlas.ti was used. Based on the Flottorp framework, the findings of the interviews and additional literature, the current survey was developed. The preliminary data has not yet been published as it is part of the abovementioned trial. Questions of the COVID-19 app survey focused on content and app use in general, especially during a busy period caused by SARS-CoV-2 / COVID-19 (Tables 3, 4 and 5, S1 Appendix).
Data management and analysis
During the installation process of the app, users were informed that user analytics would be collected and anonymously analyzed on group level. Additionally, users were asked to select their role in the hospital. Data of process indicators was stored in Data Studio, Google. Responses on the questionnaire were stored in LimeSurvey which was hosted on secure, password-protected hospital servers. In the questionnaire basic demographic information such as age, gender, role and department was asked. Data obtained from the questionnaire was collected and analyzed anonymously. Descriptive analyzes and Fisher’s exact test to assess questionnaire responses association were performed using IBM SPSS Statistics 25.0 for Windows (IBM, Armonk, NY, USA).
Results
COVID-19 admissions
The first COVID-19 patient was admitted to our hospital at the end of February 2020. After a peak of admissions in (mainly the second half of) March 2020 with 135 patients with a positive SARS-CoV-2 PCR, the number of patients with a positive SARS-CoV-2 PCR declined in April (111), May (20), June (5) until July 8 (1). In total, 272 patients with a positive SARS-CoV-2 PCR were admitted until July 8, 2020. Similarly, on a national level COVID-19 admissions in February 2020 were few (9), peaked in March 2020 (6373) and decreased in the following months (4845 in April, 904 in May, 205 in June and 166 in July) [26].
User analytics
The COVID-19 app was introduced on March 30, 2020. From the introduction until the end of study app content was updated 156 times. The total number of users from March 30, 2020 to July 8, 2020 was 1,168 of the hospital’s 14,359 HCWs (8.1%) which included 27.9% of all medical specialists and 10.3% of all nurses. COVID-19 app users were predominantly in the role of medical specialist (22.3%), nurse (22.2%), resident (16.5%), management (6.2%), medical student (4.0%), fellow (2.1%), infection prevention expert (0.4%) and other (26.5%) (Fig 1, S2 Appendix). The group ‘other’ consisted primarily of technicians (such as radiologic, laboratory, operating room, medical, IT, anesthetic, maintenance and pharmacy technicians) (38.4%), administrative professionals (30.8%), clinicians (16.1%), and research and education professionals (9.0%). The number of active users varied during the study period from 128 (March), 1017 (April), 594 (May), 487 (June) to 368 (July). The total number of sessions was 3903 with peak accessions at nine o’clock in the morning (Fig 2, S1 Fig). Nurses had the highest number of sessions throughout the day (Table 1).
The average session time was 1 minute and 20 seconds and most users used the app for 2–4 sessions during the study period (Table 2).
Overall, the most frequently accessed advice was information for HCWs such as when to test for SARS-CoV-2 (1214), latest updates (1181) and the COVID-19 telephone list (418). An advice can contain several screens with information. The most accessed screens were HCWs with possible COVID-19 symptoms (596), no / unknown unprotected physical contact with COVID-19 patients (472) and HCW without fever (461) (S3 Appendix). Among nurses, medical specialists and residents the most frequently accessed advice was information for HCWs such as when to test for SARS-CoV-2, latest updates and information regarding patients.
User experience
The initial app push message to invite users to respond to the questionnaire yielded 11 respondents. Three subsequent e-mails were sent as a reminder and yielded an additional 24, 22 and 14 respondents, respectively.
Of the COVID-19 app users, 71 users with a mean age of 46.1 years from 19 different departments completed the questionnaire of which three users partially completed the questionnaire. Most respondents were female (53/71; 75%), and nearly all respondents agreed that use of smartphone apps can contribute to their work (68/71; 96%) and indicated to already use different apps for their work on a daily (21/71; 30%) or weekly (18/71; 25%) basis (Table 3).
Questionnaire respondents considered the COVID-19 app clear, easy-to-use, fast, and useful (Table 4). One percent (1/70) of the respondents did not trust the app.
The most frequent reasons to use the app were checking the latest SARS-CoV-2 / COVID-19 updates, infection prevention and control policy, looking up/dial telephone numbers and clinical advice. Sixty-seven users were already partially familiar (49/68; 72%) or familiar (18/68; 26%) with the COVID-19 app content. Hyperlinks to the source documents were available, especially if the information was too elaborate to show in the app. Sixty-eight users rated the presence of hyperlinks as neutral (30/68; 44%), annoying (25/68; 37%) or did not encounter the hyperlinks (13/68; 19%). The most stated reasons to use the COVID-19 app were a quick information check and while ‘on the go’, such as when no computer was available. Furthermore, 63% (43/68) of the users found that use of the COVID-19 app did not save nor cost time, 29% (20/68) considered use saved time and 7% (5/68) said it cost them time (Table 5).
Eighty-seven percent (62/71) of all questionnaire respondents thought that app use was moderately or fully culturally accepted in their work environment. However, none of the nurses in age group 20 to 29 (0/5) felt app use was fully accepted compared to 76% (13/17) of nurses in the other age groups. Age group was associated with the perception that app use was culturally accepted in the work environment (p<0.01). For example, most respondents in the age group 20–29 felt this was moderately accepted, while the majority of respondents of all other age groups felt it was fully accepted (S4 Appendix). Additionally, gender was associated with the perception that app use was culturally accepted in the work environment (p<0.01). None of the female respondents felt it was not accepted. Contrarily, male respondents did feel this way (S3 Appendix). There was a relationship between role and whether respondents would use the COVID-19 app adjacent to patients (p<0.01). None of the residents found this unprofessional, although medical specialists did (S4 Appendix). App use adjacent to patients was no problem for half of the respondents (36/68; 53%), a quarter had doubts to do so (19/68; 28%) while others, predominantly nurses (7/8), felt it was not right (8/68; 12%) and some thought it was unprofessional (5/68; 7%). Colleagues had no negative influence on app use and most respondents would (38/68; 56%) or might (27/68; 40%) recommend the COVID-19 app to their peers. Only 3% (2/68) of the respondents would not use a similar app with information and guidelines if it was to be introduced in our hospital in the future.
Discussion
Here, we demonstrated that a smartphone app can be a very useful tool to disseminate up-to-date information and hospital policy to HCWs during a healthcare crisis such as the COVID-19 pandemic. Our COVID-19 app was implemented in a short period of time, was adopted easily by HCWs and cost little. Over the course of a 3⅓ month period, even after the peak of admitted patients, the app had 1168 users and 3903 unique sessions. The app was primarily used to access hospital policy regarding information for HCWs, such as when to test for SARS-CoV-2, latest updates, the SARS-CoV-2 / COVID-19 guideline and the COVID-19 telephone list. Considering the novelty of the virus and disease and changing insights it is not surprising information for HCWs regarding when to test for SARS-CoV-2 was accessed the most. Respondents of the questionnaire reflected the female:male ratio (75:25) of HCWs in our hospital (72:28). Respondents were positive about the influence apps had on their work and experienced an overall high user satisfaction using the COVID-19 app. One of the advantages was that the app can be used to quickly check information, while ‘on the go’ or in meetings, and predominantly had no effect on time or saved the user time. Additionally, previous non-users of apps seem to have a positive attitude towards this new technology for their work and are willing to adapt apps like the COVID-19 app.
Three months after its launch our app was downloaded by 1168 users, which is more than most other studies have reported for apps containing hospital policy. The dynamic work situation during the COVID-19 outbreak could explain the relatively high number of downloads of our app. The number of total app downloads in other studies varied from 53 times after 3 months, 1233 times in 4 months, 2013 times after 10 months, 990 times after 12 months to 3056 times after 14 months [19, 23, 25, 27, 28]. However, total app sessions (3903) seemed below average compared to other studies which ranged from 9259 sessions over 14 months, >16,000 sessions over 10 months to 18,860 sessions over 12 months and could be due to the short study period and novelty of the tool within the hospital [25, 28, 29]. Additionally, the app was introduced after the peak in COVID-19 admissions in the hospital and showed a corresponding decline in monthly use, similar to another study [16]. One study that was conducted during the start of the pandemic saw an increase in users, whereas our study conducted after the peak saw a decrease in COVID-19 app users, similar to the decrease in accessing COVID-19 related content in another study [18, 19]. We did not consider or intend the COVID-19 app to replace communication of the hospital policy via the intranet. However, since smartphones and apps are used extensively we aimed to augment the main channel of communication to increase the chance that SARS-CoV-2 / COVID-19 hospital policy reached HCWs. An article on the COVID-19 app was published on the intranet. Similar to survey respondents of our COVID-19 app, other studies found HCWs experienced their studied app as positive, easy and useful [17, 18].
Due to the very frequent updates in the COVID-19 app content we decided not to send push notifications to inform users of an update. This may have been a missed opportunity to engage more users as shown in another study [19]. Adherence to the COVID-19 content information in the app as well as the impact of the app on outcomes in patient care remain unclear and could be the focus of future research [18, 19].
Our COVID-19 app was used by all kind of HCWs. Indeed, hospitals are not only staffed with clinical workers such as physicians and nurses but also with other HCWs including technicians, administrative professionals and research and education professionals. During the COVID-19 crisis extra effort was asked of every HCW. Many HCWs got their COVID-19 information, such as risk of acquiring and transmitting SARS-CoV-2, from social media which may lead to poorer understanding or misunderstanding on several topics [30, 31]. Additionally, unclear and frequently changing guidelines can cause confusion amongst HCWs [32]. Therefore, hospital management needs to ensure that all HCWs receive accurate and up-to-date information in a timely fashion. In this technological era a smartphone app is an accessible, easy-to-use and an universal tool to facilitate dissemination of information.
Remarkably, none of the nurses in the youngest age category felt app use was fully accepted in their work environment contrary to the older age groups. This may be associated with a junior position of these HCWs who might worry that others perceive their smartphone use as not work-related [33]. Age group and gender were associated with the perception that app use was culturally accepted in the work environment. Additionally, a new group of app users who previously did not use apps for their work were enthusiastic to use our app during the COVID-19 crisis (13/71; 18%). Most would use or consider use of a similar future app which illustrates the willingness to adapt a new way of working in order to combat a new, not fully understood virus and disease.
Most questionnaire respondents felt app use for work was somewhat (12/71; 17%) or fully (50/71; 70%) accepted in their work environment and did not think it would be a problem to use the app adjacent to patients if necessary. Role was associated whether respondents would use the COVID-19 app adjacent to patients. Seven percent (5/68) of the respondents considered app use adjacent to patients unprofessional and 12% (8/68) stated that it did not feel right. Although we do not know the underlying reasons for this, as this was not explored, there are several possibilities. Mobile phones are known to be contaminated with a variety of pathogens [34]. Mobile phones in the hospital should therefore be used with application of basic infection prevention measures, but this may not be known by HCWs which might have made them reluctant to use the app. Additionally, HCWs might think they give the impression using their phone for leisure in front of the patient. Similarly, other studies showed that HCWs may feel uncomfortable (20%) using an app near colleagues or patients [23, 33]. Increasing awareness of the use of smartphone apps as medical tools remains necessary to allow them reaching their full potential in healthcare institutions.
Strengths and limitations
A strength of this study was that we evaluated the use of a relatively novel tool in a hospital during a healthcare crisis, which has been studied very limited. A possible limitation is the short period of this study which started after the peak and during the decline of COVID-19 in our hospital, and this has to be weighed against the corresponding onboarding, plateau phase and decline in app use. HCWs also may have been already familiarized with hospital’s intranet as the go-to source of SARS-CoV-2 / COVID-19 information at the start of the study. Data on how often hyperlinks to source documents were clicked were not available, and we could not determine if app users had any interest in the hyperlinks. Additionally, the questionnaire used in this pilot study was based on preliminary data from an ongoing study. A limited number of app users from all categories of HCWs participated in the questionnaire and users with a positive experience might be more inclined to do so, leading to a bias in appreciation of the app.
Conclusions
In a healthcare crisis HCWs need to be able to access an up-to-date hospital policy at all times in order to fulfill their jobs effectively and correctly. Nowadays, most HCWs carry a smartphone providing an ideal medium to receive critical information through apps during a pandemic such as COVID-19. Short implementation time makes it an excellent hospital resource to have in dire times. Our COVID-19 app proved to be a relatively simple yet innovative tool that was used by HCWs from all disciplines involved in care of COVID-19 patients. The up-to-date app was used for different topics and had a high user satisfaction amongst questionnaire respondents. The short implementation time makes it an attractive tool to disseminate information on topics such as infection prevention and control, use of PPE and patient care. An app with local hospital policy can be a invaluable tool during a pandemic.
Supporting information
S1 Appendix. Questionnaire in English and Dutch.
https://doi.org/10.1371/journal.pone.0262105.s002
(DOCX)
S2 Appendix. User analytics–number of users per role.
https://doi.org/10.1371/journal.pone.0262105.s003
(DOCX)
S3 Appendix. User analytics—top most frequently accessed advice.
https://doi.org/10.1371/journal.pone.0262105.s004
(DOCX)
S4 Appendix. Association between demography and survey responses.
https://doi.org/10.1371/journal.pone.0262105.s005
(DOCX)
References
- 1. Costantini E, Allara E, Patrucco F, Faggiano F, Hamid F, Balbo PE. Adherence to guidelines for hospitalized community-acquired pneumonia over time and its impact on health outcomes and mortality. Intern Emerg Med. 2016;11(7):929–40. pmid:27098057
- 2. Dondo TB, Hall M, Timmis AD, Gilthorpe MS, Alabas OA, Batin PD, et al. Excess mortality and guideline-indicated care following non-ST-elevation myocardial infarction. Eur Heart J Acute Cardiovasc Care. 2017;6(5):412–20. pmid:27142174
- 3. Belfroid E, Timen A, van Steenbergen JE, Huis A, Hulscher ME. Which recommendations are considered essential for outbreak preparedness by first responders? BMC Infect Dis. 2017;17(1):195. pmid:28270113
- 4. Belfroid E, Hautvast JL, Hilbink M, Timen A, Hulscher ME. Selection of key recommendations for quality indicators describing good quality outbreak response. BMC Infect Dis. 2015;15:166. pmid:25888491
- 5. Kharbanda EO, Asche SE, Sinaiko AR, Ekstrom HL, Nordin JD, Sherwood NE, et al. Clinical Decision Support for Recognition and Management of Hypertension: A Randomized Trial. Pediatrics. 2018;141(2). pmid:29371241
- 6. Rittmann B, Stevens MP. Clinical Decision Support Systems and Their Role in Antibiotic Stewardship: a Systematic Review. Curr Infect Dis Rep. 2019;21(8):29. pmid:31342180
- 7. Bruun M, Frederiksen KS, Rhodius-Meester HFM, Baroni M, Gjerum L, Koikkalainen J, et al. Impact of a clinical decision support tool on prediction of progression in early-stage dementia: a prospective validation study. Alzheimers Res Ther. 2019;11(1):25. pmid:30894218
- 8. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. pmid:32047862
- 9. Reeves JJ, Hollandsworth HM, Torriani FJ, Taplitz R, Abeles S, Tai-Seale M, et al. Rapid response to COVID-19: health informatics support for outbreak management in an academic health system. J Am Med Inform Assoc. 2020;27(6):853–9. pmid:32208481
- 10. McRae MP, Simmons GW, Christodoulides NJ, Lu Z, Kang SK, Fenyo D, et al. Clinical decision support tool and rapid point-of-care platform for determining disease severity in patients with COVID-19. Lab Chip. 2020;20(12):2075–85. pmid:32490853
- 11. Liu Y, Wang Z, Ren J, Tian Y, Zhou M, Zhou T, et al. A COVID-19 Risk Assessment Decision Support System for General Practitioners: Design and Development Study. J Med Internet Res. 2020;22(6):e19786. pmid:32540845
- 12. Currie DJ, Peng CQ, Lyle DM, Jameson BA, Frommer MS. Stemming the flow: how much can the Australian smartphone app help to control COVID-19? Public Health Res Pract. 2020;30(2). pmid:32601652
- 13. Wang S, Ding S, Xiong L. A New System for Surveillance and Digital Contact Tracing for COVID-19: Spatiotemporal Reporting Over Network and GPS. JMIR Mhealth Uhealth. 2020;8(6):e19457. pmid:32499212
- 14. Faezipour M, Abuzneid A. Smartphone-Based Self-Testing of COVID-19 Using Breathing Sounds. Telemed J E Health. 2020. pmid:32487005
- 15. Menni C, Valdes AM, Freidin MB, Sudre CH, Nguyen LH, Drew DA, et al. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nat Med. 2020;26(7):1037–40. pmid:32393804
- 16. Timmers T, Janssen L, Stohr J, Murk JL, Berrevoets MAH. Using eHealth to Support COVID-19 Education, Self-Assessment, and Symptom Monitoring in the Netherlands: Observational Study. JMIR Mhealth Uhealth. 2020;8(6):e19822. pmid:32516750
- 17. Ros M, Neuwirth LS. Increasing global awareness of timely COVID-19 healthcare guidelines through FPV training tutorials: Portable public health crises teaching method. Nurse Educ Today. 2020;91:104479. pmid:32473497
- 18. Zamberg I, Manzano S, Posfay-Barbe K, Windisch O, Agoritsas T, Schiffer E. A Mobile Health Platform to Disseminate Validated Institutional Measurements During the COVID-19 Outbreak: Utilization-Focused Evaluation Study. JMIR Public Health Surveill. 2020;6(2):e18668. pmid:32250958
- 19. Windisch O, Zamberg I, Zanella MC, Gayet-Ageron A, Blondon K, Schiffer E, et al. Using mHealth to Increase the Reach of Local Guidance to Health Professionals as Part of an Institutional Response Plan to the COVID-19 Outbreak: Usage Analysis Study. JMIR Mhealth Uhealth. 2020;8(8):e20025. pmid:32749996
- 20.
Firstline. 2020 [Available from: https://firstline.org/.
- 21. Helou RI, Catho G, Peyravi Latif A, Mouton J, Hulscher M, Teerenstra S, et al. Study protocol for an international, multicentre stepped-wedge cluster randomised trial to evaluate the impact of a digital antimicrobial stewardship smartphone application. BMJ Open. 2020;10(6):e033640. pmid:32503867
- 22. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35. pmid:23522377
- 23. Charani E, Kyratsis Y, Lawson W, Wickens H, Brannigan ET, Moore LSP, et al. An analysis of the development and implementation of a smartphone application for the delivery of antimicrobial prescribing policy: Lessons learnt. J Antimicrob Chemother. 2013;68(4):960–7. pmid:23258314
- 24. Payne KF, Weeks L, Dunning P. A mixed methods pilot study to investigate the impact of a hospital-specific iPhone application (iTreat) within a British junior doctor cohort. Health Informatics J. 2014;20(1):59–73. pmid:24550565
- 25. Panesar P, Jones A, Aldous A, Kranzer K, Halpin E, Fifer H, et al. Attitudes and behaviours to antimicrobial prescribing following introduction of a smartphone App. PLoS ONE. 2016;11(4). pmid:27111775
- 26.
National Institute for Public Health and the Environment. COVID-19 development in graphs 2021 [Available from: https://www.rivm.nl/coronavirus-covid-19/grafieken.
- 27. Yoon CH, Ritchie SR, Duffy EJ, Thomas MG, McBride S, Read K, et al. Impact of a smartphone app on prescriber adherence to antibiotic guidelines in adult patients with community acquired pneumonia or urinary tract infections. PLoS ONE. 2019;14(1).
- 28. Hoff BM, Ford DC, Ince D, Ernst EJ, Livorsi DJ, Heintz BH, et al. Implementation of a Mobile Clinical Decision Support Application to Augment Local Antimicrobial Stewardship. J Pathol Inform. 2018;9:10. pmid:29692947
- 29. Young HL, Shihadeh KC, Skinner AA, Knepper BC, Sankoff J, Voros J, et al. Implementation of an institution-specific antimicrobial stewardship smartphone application. Infect Control Hosp Epidemiol. 2018;39(8):986–8. pmid:29925458
- 30. Li HO, Bailey A, Huynh D, Chan J. YouTube as a source of information on COVID-19: a pandemic of misinformation? BMJ Glob Health. 2020;5(5). pmid:32409327
- 31. Bhagavathula AS, Aldhaleei WA, Rahmani J, Mahabadi MA, Bandari DK. Knowledge and Perceptions of COVID-19 Among Health Care Workers: Cross-Sectional Study. JMIR Public Health Surveill. 2020;6(2):e19160. pmid:32320381
- 32. Kang HS, Son YD, Chae SM, Corte C. Working experiences of nurses during the Middle East respiratory syndrome outbreak. Int J Nurs Pract. 2018;24(5):e12664. pmid:29851209
- 33. Shenouda JEA, Davies BS, Haq I. The role of the smartphone in the transition from medical student to foundation trainee: a qualitative interview and focus group study. BMC Med Educ. 2018;18(1):175. pmid:30064424
- 34. Sadat-Ali MA-O A.K.; Azam Q; Bukari H; Al-Zahrani A.J.; Al-Turki R.A.; Al-Omran A.S. Bacterial flora on cell phones of health care providers in a teaching institution. Am J Infect Control. 2010;38(5):404–5. pmid:20363049