https://orcid.org/0009-0004-1920-1117
Figures
Abstract
The gold standard treatment for apnea of prematurity is caffeine citrate, which is known for its effect on diaphragm muscle activity. The purpose of this study was to investigate the electrical activity of the diaphragm in preterm newborns, which was measured 30 minutes before and 60 minutes after the administration of a loading dose of caffeine citrate. In this prospective, observational, longitudinal study at a tertiary-level neonatal ICU, data were collected from 36 patients (13 females, 23 males) with a mean gestational age of 31 2/7 ± 2 1/7 weeks and a mean birth weight of 1532 ± 439 grams. The average caffeine level was 15.7 ± 5.9 mg/dL (reference range: 5–30 mg/dL). The results revealed significant increases in all the parameters used to determine the inspiratory phases of the respiratory cycle postcaffeine administration (p < 0.001). The study concluded that caffeine administration significantly enhances diaphragmatic electrical activity in preterm newborns.
Citation: Zidan TB, Troster EJ, Beck J, Sanches LR, Ferreira CES, Zacharias RS, et al. (2025) Effect of caffeine citrate on diaphragmatic electrical activity in pre-term newborns. PLoS ONE 20(4): e0320992. https://doi.org/10.1371/journal.pone.0320992
Editor: Hakan Aylanc, Canakkale Onsekiz Mart University School of Medicine, TÜRKIYE
Received: April 14, 2024; Accepted: February 28, 2025; Published: April 11, 2025
Copyright: © 2025 Zidan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are available from Figshare repository (DOI: https://doi.org/10.6084/m9.figshare.26752516).
Funding: The author(s) received no specific funding for this work.
Competing interests: The study was designed and conducted at the Hospital Israelita Albert Einstein in São Paulo, and none of the authors have any conflicts of interest related to patents. The analysis of the collected data was conducted in Toronto by Dr. Beck's team. We confirm that Dr. Beck's participation in the study did not involve any conflicts of interest.
Introduction
Prematurity and the survival of preterm newborns have increased in recent years. The World Health Organization estimates that 15 million premature births occur each year. The worldwide rate of prematurity ranges from 5 to 18%, and its complications are the main cause of mortality in children under 5 years of age [1,2].
Apnea of prematurity is one of the most frequent diagnoses in neonatal intensive care units; its incidence is inversely proportional to the newborn’s gestational age, and it affects almost all newborns under 28 weeks of gestational age. Recurrent episodes of apnea, bradycardia and hypoxemia worsen the prognosis and increase morbidity among this population [3].
The gold standard treatment for apnea of prematurity is caffeine citrate, which is the most frequently used drug in neonatal intensive care; its routine use is widely recommended for preterm newborns [4]. Among its benefits are reductions in the occurrence of apnea during prematurity, extubation failure, the duration of mechanical ventilation and oxygen need, and the incidence of bronchopulmonary dysplasia [5–7]. Long-term studies have shown reduced neurodevelopmental disability at 18–21 months with caffeine citrate use, with follow-ups at 5 and 11 years showing improved gross motor function and reduced motor impairment [8–10].
Caffeine citrate acts by blocking A1 and A2 adenosine receptors, stimulating the respiratory center [11]; it also increases diaphragmatic electrical activity and contractility, although the exact mechanism remains unknown [12,13].
Diaphragmatic electrical activity (Edi) can be monitored, and this parameter is used to adjust mechanical ventilation in neurally adjusted ventilatory assist mode (NAVA – neurally adjusted ventilatory assist) [14]. In this ventilatory strategy, the inspiratory pressure is proportional to the intensity of Edi, and ventilator pressurization ceases when the diaphragm’s neural activity begins to decline. The maximum Edi value (Edi max) indicates inspiratory nerve stimulus intensity and breath duration, whereas the minimum Edi (Edi min) represents diaphragm basal tone, preventing alveolar collapse during expiration.
Few studies have investigated the effect of caffeine on the electrical activity of the diaphragm in preterm newborns [15–17]. The aim of this study was to analyze Edi behavior in preterm newborns before and after the administration of a loading dose of caffeine citrate. The secondary objectives were to observe the respiratory rate and occurrence of breathing pauses after drug administration.
We hypothesized that Edi would increase after drug administration.
Materials and methods
Study design
This was a prospective, observational, longitudinal study (pre- and postintervention) conducted at a tertiary-level neonatal ICU.
Inclusion criteria
Newborns of both genders, of any chronological age, with a gestational age of less than 37 weeks and clinical indications for caffeine citrate treatment, were included. The criteria included preterm newborns with a gestational age of ≤28 weeks, those between 28 and 32 weeks receiving mechanical ventilation, and those with a clinical diagnosis of apnea. Gestational age was determined by the date of the mother’s last menstrual period or early first-trimester ultrasound in its absence. Central apnea was clinically defined as breathing pauses ≥20 seconds, or shorter with bradycardia (heart rate <100 bpm), cyanosis, or pallor. Newborns were included if they had ≥2 clinical apnea episodes within an hour. Caffeine was started prophylactically for all patients ≤28 weeks and those 28–32 weeks under mechanical ventilation or therapeutically for all patients with a clinical apnea diagnosis.
Exclusion criteria
Patients with major congenital malformations, upper digestive tract anomalies preventing esophageal catheter insertion, primary neurological or drug-related abnormalities impairing respiratory center function, or those under high-frequency mechanical ventilation were excluded.
Patient population
We included patients on invasive mechanical ventilation, namely, a neuturally adjusted ventilatory assist (NAVA), synchronized intermittent mandatory ventilation (SIMV) and assisted control (AC), as well as those on noninvasive ventilation (noninvasive NAVA and CPAP), and those breathing spontaneously. Respiratory support was defined based on clinical parameters, with no changes between baseline and postcaffeine measurements. None of the patients were sedated.
Out of the 78 eligible patients, 42 were excluded (4 major malformations, 18 refusal of consent, 7 data recording issues, 1 severe asphyxia and 12 concurrent use of the system), resulting in 36 included patients.
Patients were recruited between July 19, 2018, and October 3, 2019.
Caffeine treatment
The caffeine citrate loading dose was 20 mg/kg, as recommended by the largest randomized controlled trial and the US Food and Drug Administration [7,18,19].
Evaluation of diaphragmatic electrical activity
A naso/oro gastric feeding tube designed to measure the electrical activity of the diaphragm [Maquet, Getinge Group, Solna, Sweden] was inserted and positioned in the lower esophagus, close to the crural diaphragm. The catheter contained nine electrodes that captured and processed the Edi into a waveform and removed contamination from the electrical signals from the heart, the esophagus and the environment [14,20,21]. The Edi and all the mechanical ventilator parameters were continuously recorded and transferred simultaneously to a computer via Servo-Tracker software [Maquet, Getinge Group, Solna, Sweden] and using a data transmission cable adaptable to a Servo-n mechanical ventilator [Maquet, Getinge Group, Solna, Sweden]. The data were collected 30 minutes before and 60 minutes after the caffeine loading dose (Fig. 1, see blue bars).
Precaffeine data: first blue column. Loading dose of caffeine: yellow column. Time after loading: white column. Postcaffeine data: second blue column.
Edi signal analysis
The Edi waveform was automatically analyzed automatically (by Servo-n or Servo-I software, Maquet Getinge Group, Sweden) and was characterized by its timing and amplitude. The parameters of interest included the peak Edi amplitude, end-expiratory amplitude, and neural respiratory rate, which were quantified before and after caffeine administration. To define an “Edi breath”, three specific time points of the neural respiratory cycle were determined: beginning of neural inspiration (beginning of Edi increase defined as greater than 0.5μV above the basal line (IOn), peak of Edi (highest Edi value during inspiration “PeakIon”), and end of inspiration was determined by a decrease equal to or greater than 70% of the Edi (PIOff) (Fig. 2). Additionally, the end expiratory Edi (lowest Edi value during expiration) was determined.
Svi: pneumatic state; Edi: diaphragmatic electrical activity; PIOn: start of pneumatic inspiration; PIOff: start of pneumatic expiration; PEOff: start of the next inspiration cycle; PreIOn: previous inspiratory stop (70% peak Edi); IOn: inspiratory start (first sample increase of a detected Edi breath); PostIOn: inspiratory stop (70% peak Edi); PeakIOn: inspiratory peak Edi; PreIOff: inspiratory start; IOff: inspiratory stop; PostIOff: next inspiratory start.
Neural respiratory activity was recorded for the inspiratory and expiratory phases (Fig. 3). Respiratory pauses were quantified based on flat Edi periods of 5–10 sec, 10–19 sec or longer.
1. Edi min 2. Edi peak 3. Mean inspiratory Edi 4. Minimum expiratory Edi (MinEEdi).
Serum caffeine levels
Serum caffeine levels were measured 24 hours after the loading dose or maintenance dose, according to scheduled routine collections. No venous punctures were performed exclusively for this purpose.
A 400-μL blood sample was collected, and caffeine levels were analyzed via a method developed at the Laboratory of Clinical Chemistry of the Hospital Israelita Albert Einstein. This analysis was performed via ultrahigh-performance liquid chromatography with a diode array detector (UHPLC-DAD). Chromatographic separation was performed via a C18 column (2.1x 50 mm, 1.8 m); the mobile phase consisted of water and methanol, and the flow rate was 0.4 mL/minute. The sample preparation involved protein precipitation with methanol and zinc sulfate, followed by filtration and injection of the supernatant via UHPLC-DAD equipment [22,23].
Statistics
The sample calculation was based on a convenience sample collected between July 2018 and October 2019, including all newborns who met the criteria for inclusion, within the data collection period.
Parameter distributions were studied via histograms and boxplots, and generalized estimating equation models were applied for the inferential analysis [24], considering the correlation between measurements from the same newborn before and after caffeine administration. Normal or gamma distributions with a logarithmic link function were used, with the best-fitting distribution chosen based on QIC goodness-of-fit results [25].
Comparisons between pre- and postcaffeine measurements in the total sample were performed, and the model fit considered the effect of evaluation time. To compare the measurements within the pre- and postcaffeine groups, the mode and time of the evaluation and the interaction effect between mode and time were used as the explanatory variables of the model. The p values obtained in the multiple comparisons between the evaluation times within each mode were corrected via the sequential Bonferroni method.
The analyses were performed via SPSS statistical software [26], and a significance level of 0.05 was adopted.
Ethical considerations
The study was approved by the Hospital Israelita Albert Einstein Ethics Committee (approval number 2.730.352). Data were collected after informed consent (verbal and written) was obtained from parents.
The study was registered internally at our hospital on two platforms: the SGPP (General System of Research Protocols) and the hospital’s Ethics and Research Committee. Research projects are subsequently audited by the hospital’s scientific research integrity office. It was also registered and approved in the Brazilian Registry of Clinical Trials, a body that is part of the World Health Organization.
Results
Data were collected from 43 patients, with seven losses due to data recording problems, including problems with the data transmission cable and incompletely captured data, resulting in 36 analyzed patients. The demographic characteristics and types of respiratory support used are shown in Table 1. The caffeine level was 15.7 ± 5.9 mg/dl (reference range: 5–30 mg/dl). The minimum value obtained was 4.8 mg/dl, and the maximum value was 31.8 mg/dl.
Caffeine was administered therapeutically in 20 patients (55.5%) and prophylactically in 16 patients (44.5%). Data were collected in the first 12 hours of life for 25 patients (69.4%), 3 patients (8.3%) had their data collected between 12 and 24 hours of life, and 8 patients (22.3%) were monitored for more than 24 hours of life. The average age at data collection was 2 hours of life.
The results of the comparisons of measurements before and after caffeine citrate administration are shown in Table 2. We evaluated the minimum inspiratory Edi (MinIEdi), maximum inspiratory Edi (MaxIEdi), mean inspiratory Edi and MinEEdi before and after the caffeine citrate loading dose. MinEdi increased from 2,32–2,93 µV (p =0.007), and MaxEdi increased from 10,35–12,17 µV (p<0.05). The mean inspiratory Edi increased from 6,76–8,11 µV (p<0.05), and MinEEdi increased from 2,31–2,98 µV (p=0.004). We also evaluated the neural respiratory rate before and after caffeine administration. There was a statistically significant increase in the neural respiratory rate from 66.5 to 75.9 bpm (breaths per minute) (p=0.01).
We evaluated the number of respiratory pauses and apnea episodes before and after the caffeine citrate loading dose. There was a reduction in the number of respiratory pauses between 5–10 s and 11–19 s after the loading dose; however, this reduction was not statistically significant (Fig. 4).
± standart deviation. Precaffeine data: black columns Postcaffeine data: gray columns.
Discussion
This study demonstrated that caffeine administration in preterm newborns promotes an increase in the electrical activity of the diaphragm.
Our study is the first to document the serum caffeine level and to demonstrate adequate therapeutic levels in newborns whose diaphragmatic electrical activity data were collected. These data are useful for ensuring that the serum concentration of the drug is adequate.
The effect of caffeine on the central nervous system in preterm newborns is well known and documented. Caffeine acts by inhibiting adenosine receptors, which stimulate the respiratory center, improve responsiveness to increased CO2 and reduce neuronal apoptosis in hypoxic-ischemic lesions [11,27]. There are, however, few studies on its effects on diaphragm function.
Our study revealed an increase in diaphragm electrical activity in all phases of the respiratory cycle (inspiration and expiration) after caffeine loading. All recorded respiratory cycles were studied separately. The Edi values at the beginning, end, peak, minimum and mean of the inspiratory and expiratory cycles significantly increased. Additionally, there was an increase in the respiratory rate after the loading dose was administered.
These data are consistent with those of previous studies showing an improvement in the contractility of respiratory muscles [12,28] and the action of caffeine citrate on peripheral chemoreceptors [29]. It has also been shown that caffeine causes increases in minute volume and tidal volume, a decrease in resistance and an increase in lung compliance in newborns diagnosed with bronchopulmonary dysplasia [30]. Dekker et al. also demonstrated an increase in respiratory effort with increased minute volume and tidal volume [31]. These findings may be explained by an improvement in respiratory mechanics resulting from increased diaphragm contractility and an increase in the respiratory rate, as demonstrated in this study.
Other studies have shown increased diaphragmatic electrical activity after caffeine loading [15–17]. However, these studies had small sample sizes, and Kraaijenga et al. [15] studied only spontaneously breathing newborns, monitoring diaphragmatic electrical activity by transcutaneous electromyography, a technique that does not allow for separate evaluation of diaphragmatic electrical activity during the inspiratory and expiratory phases. Furthermore, an increase in Edi measured by transcutaneous electromyography may partly result from other adjacent muscles, such as intercostal or abdominal muscles. Williams et al. [17] also evaluated the diaphragmatic response via electromyography and reported an increase in electrical activity and respiratory mechanics.
Kraaijenga et al. [15] expressed diaphragmatic activity as the percentage change in electromyography of the diaphragm (dEMG) and reported a 43% increase in dEMG after caffeine administration. In our study, we showed a 68% increase in the inspiratory peak Edi after the loading dose of caffeine. The comparison between the two studies revealed a greater increase in the inspiratory peak Edi when a more refined methodology was used.
Parikka et al. [16] reported that caffeine citrate reduced 5- to 10-second respiratory pauses. We found a reduction in the absolute number of respiratory pauses, which, however, was not statistically significant.
This study has several limitations. The use of a convenience sample diminishes its power; however, we did not find any literature reference using the same variables for sample size calculation. Since we found a significant difference in pre- and postcaffeine use in diaphragm electrical activity, we did not commit a type 2 error.
We could also not correlate the serum level of caffeine with diaphragmatic electrical activity because the caffeine serum level was collected together with other routine tests of newborns. Although we could not make this correlation, the literature shows that the standard doses commonly used provide the expected clinical effects, such as a reduction in apnea episodes and a reduction in extubation failures [32–34].
The absence of a control group in our study could be considered a limitation. However, from an ethical point of view, it would be inappropriate to postpone or stop administering a medication that is formally indicated for this group of newborns. In this case, each patient served as their own control through measurements taken before and after caffeine administration.
The duration of data recording and the possibility that the increase in Edi parameters may have been transient could be considered another limitation of our study. Kraaijenga et al. [15] and Williams et al. [17] reported that differences in diaphragmatic parameters were no longer significant beyond 120 min and 60 min from the caffeine load, respectively. However, Williams et al. studied only invasively ventilated newborns, and this respiratory support could lead to a decreased response of the diaphragm to caffeine stimulation. In our study, we could not correlate the ventilatory mode or lack of support with diaphragmatic electrical activity, as few patients were on mechanical ventilation (only 13.9%).
In preterm newborns, caffeine was effective at reducing apnea, reducing the need for invasive mechanical ventilation and lowering the rates of bronchopulmonary dysplasia, intracranial hemorrhage, and patent ductus arteriosus. Additionally, it improves the prognosis for pulmonary function and neuropsychomotor development [35]; its use is routine in neonatal intensive care units, and understanding its mechanism of action is extremely important for ensuring its safe use.
In conclusion, we demonstrated an increase in the electrical activity of the diaphragm in all phases of the inspiratory and expiratory cycles after the caffeine loading dose was administered to newborns, and adequate serum levels were observed. Additionally, we showed that the neural respiratory rate increased after caffeine loading, possibly reflecting a reduction in episodes of apnea.
References
- 1. World Health Organization. Preterm birth [Internet]. [cited 2018, Few]. Available from: https://www.who.int./home/newsroom/factsheets/detail/pretermbirth.
- 2. Liu L, Oza S, Hogan D, Chu Y, Perin J, Zhu J, et al. Global, regional, and national causes of under-5 mortality in 2000-15: an updated systematic analysis with implications for the Sustainable Development Goals. Lancet. 2016;388(10063):3027–35. pmid:27839855
- 3. Eichenwald E, Committee on Fetus and Newborn AA of P. Apnea of prematurity. Pediatrics. 2016;137:e20153757.
- 4. Kreutzer K, Bassler D. Caffeine for apnea of prematurity: a neonatal success story. Neonatology. 2014;105(4):332–6. pmid:24931325
- 5. Johnson PJ. Caffeine citrate therapy for apnea of prematurity. Neonatal Netw. 2011;30(6):408–12. pmid:22052121
- 6. Abu-Shaweesh JM, Martin RJ. Caffeine use in the neonatal intensive care unit. Semin Fetal Neonatal Med. 2017;22(5):342–7. pmid:28801176
- 7. Schmidt B, Roberts RS, Davis P, Doyle LW, Barrington KJ, Ohlsson A, et al. Caffeine therapy for apnea of prematurity. N Engl J Med. 2006;354(20):2112–21. pmid:16707748
- 8. Schmidt B, Roberts RS, Davis P, Doyle LW, Barrington KJ, Ohlsson A, et al. Long-term effects of caffeine therapy for apnea of prematurity. N Engl J Med. 2007;357(19):1893–902. pmid:17989382
- 9. Schmidt B, Anderson PJ, Doyle LW, Dewey D, Grunau RE, Asztalos EV, et al. Survival without disability to age 5 years after neonatal caffeine therapy for apnea of prematurity. JAMA. 2012;307(3):275–82. pmid:22253394
- 10. Schmidt B, Roberts RS, Anderson PJ, Asztalos EV, Costantini L, Davis PG, et al. Academic Performance, Motor Function, and Behavior 11 Years After Neonatal Caffeine Citrate Therapy for Apnea of Prematurity: An 11-Year Follow-up of the CAP Randomized Clinical Trial. JAMA Pediatr. 2017;171(6):564–72. pmid:28437520
- 11. Dunwiddie TV, Masino SA. The role and regulation of adenosine in the central nervous system. Annu Rev Neurosci. 2001;24:31–55. pmid:11283304
- 12. Supinski G, Deal E, Kelsen S. The effects of caffeine and theophylline on diaphragm contractility. American Review of Respiratory Disease. 1984;130:429–33.
- 13. Aubier M. Pharmacotherapy of respiratory muscles. Clin Chest Med. 1988;9(2):311–24. pmid:3292130
- 14. Stein H, Firestone K. Application of neurally adjusted ventilatory assist in neonates. Semin Fetal Neonatal Med. 2014;19(1):60–9. pmid:24238745
- 15. Kraaijenga JV, Hutten GJ, de Jongh FH, van Kaam AH. The Effect of Caffeine on Diaphragmatic Activity and Tidal Volume in Preterm Infants. J Pediatr. 2015;167(1):70–5. pmid:25982138
- 16. Parikka V, Beck J, Zhai Q, Leppäsalo J, Lehtonen L, Soukka H. The effect of caffeine citrate on neural breathing pattern in preterm infants. Early Hum Dev. 2015;91(10):565–8. pmid:26217936
- 17. Williams EE, Hunt KA, Jeyakara J, Subba-Rao R, Dassios T, Greenough A. Electrical activity of the diaphragm following a loading dose of caffeine citrate in ventilated preterm infants. Pediatr Res. 2020;87(4):740–4. pmid:31622975
- 18. Caffeine Citrate. [Internet] Drugs.com. 2019 [cited 2019 Nov 11]. Available from: https://www.drugs.com/pro/caffeine-citrate.html.
- 19.
Hey E, editor. Neonatal Formulary. 6th ed. Chichester: John Wiley & Sons; 2011.
- 20. Sinderby C, Beck J. Neurally adjusted ventilatory assist (NAVA): un update and summary of experiences. Neth J Crit Care. 2007;11:243–52.
- 21. Beck J, Sinderby C, Lindström L, Grassino A. Influence of bipolar esophageal electrode positioning on measurements of human crural diaphragm electromyogram. J Appl Physiol (1985). 1996;81(3):1434–49. pmid:8889785
- 22. Lee HS, Khoo YM, Chirino-Barcelo Y, Tan KL, Ong D. Caffeine in apnoeic Asian neonates: a sparse data analysis. Br J Clin Pharmacol. 2002;54(1):31–7. pmid:12100222
- 23. Ptolemy AS, Tzioumis E, Thomke A, Rifai S, Kellogg M. Quantification of theobromine and caffeine in saliva, plasma and urine via liquid chromatography-tandem mass spectrometry: a single analytical protocol applicable to cocoa intervention studies. J Chromatogr B Analyt Technol Biomed Life Sci. 2010;878(3–4):409–16. pmid:20045386
- 24.
Faraway JJ. Extending the linear Model with R: generalized linear, mixed effects and nonparametric regression models. Boca Raton: Chapman & Hall/CRC; 2006.
- 25.
Altman DG. Practical statistics for medical research. London: CRC Press; 1991.
- 26.
IBM Corp. IBM SPSS Statistics for Windows, Version 24.0. Armonk (NY): IBM Corp; 2016.
- 27. Kilicdag H, Daglioglu YK, Erdogan S, Zorludemir S. Effects of caffeine on neuronal apoptosis in neonatal hypoxic-ischemic brain injury. J Matern Fetal Neonatal Med. 2014;27(14):1470–5. pmid:24392823
- 28. Kassim Z, Greenough A, Rafferty GF. Effect of caffeine on respiratory muscle strength and lung function in prematurely born, ventilated infants. Eur J Pediatr. 2009;168(12):1491–5. pmid:19271237
- 29. Chardon K, Bach V, Telliez F, Cardot V, Tourneux P, Leke A, et al. Effect of caffeine on peripheral chemoreceptor activity in premature neonates: interaction with sleep stages. J Appl Physiol. 2004;96(6):2161–6.
- 30. Davis JM, Bhutani VK, Stefano JL, Fox WW, Spitzer AR. Changes in pulmonary mechanics following caffeine administration in infants with bronchopulmonary dysplasia. Pediatr Pulmonol. 1989;6(1):49–52. pmid:2704583
- 31. Dekker J, Hooper SB, van Vonderen JJ, Witlox RSGM, Lopriore E, Te Pas AB. Caffeine to improve breathing effort of preterm infants at birth: a randomized controlled trial. Pediatr Res. 2017;82(2):290–6. pmid:28288150
- 32. Saroha V, Patel RM. Caffeine for preterm infants: Fixed standard dose, adjustments for age or high dose? Semin Fetal Neonatal Med. 2020;25(6):101178. pmid:33168464
- 33. Rosen C, Taran C, Hanna M, Gueta I, Loebstein R, Strauss T, et al. Caffeine citrate for apnea of prematurity-One dose does not fit all a prospective study. J Perinatol. 2021;41(9):2292–7. pmid:34290376
- 34. Bruschettini M, Brattström P, Russo C, Onland W, Davis PG, Soll R. Caffeine dosing regimens in preterm infants with or at risk for apnea of prematurity. Cochrane Database Syst Rev. 2023;4(4):CD013873. pmid:37040532
- 35. Moschino L, Zivanovic S, Hartley C, Trevisanuto D, Baraldi E, Roehr CC. Caffeine in preterm infants: where are we in 2020? ERJ Open Res. 2020;6(1):00330–2019. pmid:32154294