https://orcid.org/0009-0005-2672-487X
Figures
Abstract
Objective
To describe the characteristics of patients who received outpatient therapy services through an infant bridge program using telehealth mode of service delivery and to identify if attendance rates vary by mode of service delivery. We hypothesized that telehealth visits will increase attendance rates.
Participants
Eighty infants with a history of NICU admission and scheduled for a therapy appointment between June 1, 2019 and December 31, 2020 were included in the study. Participants had an average(SD) gestational age of 34.63(4.41) weeks and length of stay was 43.55(56.03) weeks. The majority were English-speaking (96.3%), White (37.5%), and had commercial insurance (72.5%).
Main outcome measure
Descriptive analyses were conducted across the entire group along with service delivery model subgroup analysis. Logistic regression was performed to assess patient characteristics associated with attendance and if service delivery model influences attendance.
Results
In the analysis of 596 scheduled visits, there were more completed telehealth sessions than for in-person sessions (90.0% versus 84.1%, p = .011). For in-person sessions, infants (N = 40) with lower birth gestational ages (p = .009), longer length of stay (p = .041), and Medi-Cal insurance (p = .006) were more likely to have ≥2 missed appointments. For the telehealth sessions, infants (N = 40) who had longer length of stay (p = .040) were more likely to have ≥2 missed appointments. There is a higher likelihood of ≥2 missed appointments for patients with a longer length of stay (OR = 1.02, 95% CI [1.01, 1.03]) and for in-person service delivery when compared to telehealth (OR = 6.25, 95% CI [1.37, 28.57]).
Conclusions
Telehealth was associated with higher likelihood of attendance, revealing that telehealth has the potential to increase access to early therapy services for certain populations. Future studies with larger sample sizes to determine which populations benefit from telehealth is recommended.
Citation: Nguyen TT, Pak SS, Miller MJ (2024) Patient characteristics and telehealth are associated with attendance rates in an outpatient rehabilitation infant bridge program. PLoS ONE 19(3): e0301219. https://doi.org/10.1371/journal.pone.0301219
Editor: Meryem Merve Ören Çelik, Istanbul University Istanbul Faculty of Medicine: Istanbul Universitesi Istanbul Tip Fakultesi, TURKEY
Received: July 25, 2023; Accepted: March 12, 2024; Published: March 27, 2024
Copyright: © 2024 Nguyen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript.
Funding: Dr. Miller was supported by funding from the NIH (KL2TR001870). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: UCSF BCH, UCSF Benioff’s Children Hospital; NICU, neonatal intensive care unit
Introduction
One in 10 infants are born prematurely in the United States and preterm infants have higher rates of developmental delays and disabilities [1]. Developmental disabilities may include breathing problems, feeding difficulties, cerebral palsy, developmental delay, vision problems, and hearing problems [1]. Early rehabilitative services (i.e., occupational and physical therapies) that start in the neonatal intensive care unit (NICU) are used to mitigate the potential risks of developmental disabilities among preterm infants.
Publicly funded early intervention programs that provide rehabilitative services for preterm infants who are discharged from the NICU effectively improve developmental and health outcomes for infants and families [2]. For example, early intervention rehabilitative services that include developmental interventions for NICU graduates were found to positively influence the development of motor and cognitive skills when compared to infants who did not receive early intervention rehabilitative services [3–5]. Parents who participated in early intervention rehabilitative services also felt more confident in caring for their infants, helping their infants progress with their developmental milestones, and developing coping strategies when compared to parents who did not receive rehabilitative services [6]. Higher volume of early intervention rehabilitative services also leads to greater functional gains, where an additional hour of rehabilitative services a month was associated with a 3-point increase in the social-emotional, cognitive, and adaptive domains on the Child Outcomes Summary [7].
Attending appointments is critical to receiving a higher volume of rehabilitative services. While a lower volume of rehabilitation services may be less effective in making functional gains, there are additional health service delivery consequences to missing rehabilitation appointments. The downstream effect of missed appointments includes wasted clinical time and resources resulting in an opportunity cost for other patients that could have used those services [8, 9]. Further, this places a burden on the administrative processes and resources, as the patient needs to be contacted and rescheduled, increasing the marginal cost for the clinic.[10] Despite these clinical challenges, difficulty with transportation, inconvenient locations, financial resources, and employer benefits (i.e., maternity leave, flexible time off, insurance coverage) all influence a families’ ability to attend NICU follow-up clinic and therapy appointments [11, 12].
Telehealth is a potential service delivery method that could be used to address barriers to receiving rehabilitation services during the transition from the NICU to home for preterm infants [13]. Telehealth eases the time and financial burdens of traveling to the clinic, potentially increasing access to therapy services for those living in underserved and rural area [14–17]. Telehealth has also improved attendance rates in NICU follow-up clinics and pediatric occupational therapy settings [13, 18]. However, the influence of telehealth for early rehabilitative services for NICU graduates has not been explored. Therefore, the purpose of this study was to determine if rehabilitation service attendance among NICU graduates differed among those with in-person or telehealth service delivery model. We also wanted to identify patient characteristics that were related to rehabilitation attendance. We hypothesize that telehealth visits were associated with higher attendance rates. This data is used to help guide which patient characteristics may benefit from in-person and/or telehealth services.
Method
Study design, site, participants, and data sources
We conducted a retrospective analysis of electronic health records data from infants who had a history of NICU admission and were scheduled for an appointment in our University of California, San Francisco Benioff’s Children Hospital (UCSF BCH) infant bridge program between June 1, 2019 and December 31, 2020. The data from the electronic health records was accessed between November 24, 2021 and December 1, 2021. The outpatient rehabilitation therapy team at UCSF BCH provides short-term outpatient physical and/or occupational therapy to bridge the gap in services between discharge home from the NICU to initiation of early intervention services offered through the state, which can take up to 6 months. In March 2020, UCSF BCH infant bridge program offered telehealth therapy services during the COVID-19 pandemic and has continued to offer these services to address ongoing challenges with the pandemic and patient and family barriers to accessing care.
We included all infants with a history of NICU admission and who received UCSF BCH bridge therapy services between June 1, 2019 and December 31, 2020. Infants with appointments between June 1, 2019 to March 30, 2020 were seen for in-person sessions, and infants with appointments between April 1, 2020 and December 31, 2020 were seen for telehealth sessions. This study, including a waiver of consent, was reviewed and approved by the University of California, San Francisco Institutional Review Board.
Measures
Rehabilitation attendance was our primary measure of interest and was categorized as participants having <2 or ≥ 2 missed appointments. We selected two or more missed appointments as a threshold for our analysis because our clinic policy indicates this is the number of missed appointments (i.e., “no-show” or cancel <24 hours’ notice) when patients will need a new referral for continued services.
Sociodemographic data included preferred language (English/Spanish), patient-identified race/ethnicity (Asian; White; Latinx; Black, Multi-Race/Ethnicity, and Other), sex assigned at birth (male/female), and gestational age (weeks). The number of participants in race/ethnicity categories of Black, Multi-Race/Ethnicity, and Other were combined into one variable due to small sample size.
Clinical data included visit type (in-person/telehealth), hospitalization length of stay (days), insurance (commercial/Medi-Cal; where Medi-Cal is California’s Medicaid healthcare program), medical complexity, rehabilitation provider type per session (occupational and/or physical therapy), and date of services. We extracted the visit type to ensure that those who were seen after April 1, 2020, received services using telehealth. Medical complexity was categorized as complex chronic disease, non-complex chronic disease, or without chronic disease using the Pediatric Medical Complexity Algorithm (PMCA) [19]. The PCMA is a validated tool that uses ICD-10 codes, number of body systems involved, and whether the disease is progressive to categorize medical complexity.
Data analysis
First, we tested for differences in the distribution of the number of completed and missed appointments by in-person and telehealth using independent samples t-tests. Next, we used Pearson’s chi-squared and independent samples t-tests to assess differences in service delivery models (in-person versus telehealth) by rehabilitation attendance, sociodemographic characteristics (preferred language, patient-identified race/ethnicity, sex assigned at birth, gestational age), and clinical characteristics (hospitalization length of stay, insurance, medical complexity, rehabilitation provider type). We then stratified the participants into service delivery model (in-person and telehealth) to assess if there were differences in the number of missed appointments groups (<2 or ≥ 2 missed appointments) by sociodemographic characteristics (preferred language, patient-identified race/ethnicity, sex assigned at birth, gestational age), and clinical characteristics (hospitalization length of stay, insurance, medical complexity, rehabilitation provider type) by service delivery model. Finally, we used backwards stepwise logistic regression to identify patient characteristics that were associated with ≥2 missed appointments (p < .050). In our first step, we generated a full model with all predictor variables that had bivariate associations with attendance at a statistical significance of p < .250. Multi-collinearity among the predictor variables in the full model was tested using Spearman’s Rho to minimize risk for bias. We proceeded with backward elimination because none of the variables were highly correlated (R>0.70). In each cycle of backward elimination, the variable in the model with the largest p-value was removed. This iterative cycle continued until the final model only had predictor variables with a significance value of p < .100. IBM Statistical Product and Service Solution (SPSS) 28 was used for data management and analysis.
Results
Participants and rehabilitation utilization
Between June 1, 2019 and December 31, 2020, there were 80 infants who were provided rehabilitation services in the UCSF BCH infant bridge program (N = 40 in-person, N = 40 telehealth) and 596 scheduled visits. Infants seen in the program had an average (SD) gestational age of 34.63 (4.41) weeks and length of stay was 43.55 weeks (56.03), and the majority were English-speaking (96.3%), White (37.5%), and had commercial insurance (72.5%). The proportion of completed versus scheduled sessions was greater for telehealth compared to in-person (90.0% versus 84.0%, p = .011) (Fig 1). There were no differences in rehabilitation attendance, sociodemographic characteristics, or clinical characteristics between infants seen in-person or telehealth (Table 1).
Overall, infants with ≥2 missed appointments, when compared to those with <2 missed appointments, had lower gestational ages (32.24 weeks versus 35.27 weeks, p = .011), longer lengths of stay (82.41 days versus 33.06 days, p = .021), and Medi-Cal insurance (52.9% versus 20.6%, p = .080). Similarly, infants with ≥2 in-person missed appointments had a lower gestational birth age (32.75 weeks versus 36.43 weeks, p = .009), longer length of stay (74.67 days versus 20.11 days, p = .041), and Medi-Cal insurance (66.7% versus 21.4%, p = .006). For the telehealth service delivery model, infants with ≥2 missed appointments only had a longer length of stay (101.00 days versus 43.43 days, p = .040), where no differences were noted for gestational age, length of stay, or insurance type (Table 2).
Six variables had bivariate associations ≥2 missed appointments (p < .250) and were included in our full model for backward logistic regression: race/ethnicity, gestational age, length of stay, insurance, medical complexity, type of therapy, and service delivery model (Table 2). Upon completion of iterative backward elimination of variables, the odds (OR, 95% CI) of ≥2 missed appointments was greater for patients with a longer length of stay (OR = 1.02, 95% CI [1.01, 1.03]) and for in-person service delivery when compared to telehealth (OR = 6.25, 95% CI [1.37, 28.57]) (Table 3).
Discussion
The purpose of this study was to determine if rehabilitation service attendance among NICU graduates differed among those with in-person or telehealth service delivery model. We identified that 91.0% of scheduled bridge appointments were completed, compared to 84.0% of in-person appointments. We also aimed to identify sociodemographic and clinical characteristics that were associated with rehabilitation session attendance. First, we identified that lower gestational age, longer length of stay and Medi-Cal insurance were associated with missing ≥2 in-person sessions, whereas only longer length of stay was associated with missing ≥2 telehealth sessions. Our final analysis indicated that longer length of stay and in-person sessions were associated with missing ≥2 sessions in our UCSF BCH infant bridge program. These findings can be used to inform future decisions in which service delivery models for outpatient therapy services best address our patients’ needs.
The first objective of this study was to describe the characteristics of patients who received outpatient therapy services through the UCSF BCH infant bridge program using telehealth mode of service delivery. There were no significant differences in the characteristics of participants who had in-person visits and those who had telehealth visits. On the contrary, a previous study in a pediatric neurology setting found telehealth use differed among patients living in rural areas by insurance types and other sociodemographic characteristics.(20) Dayal et al.(20) also reported that patients who use telehealth had different neurological diagnoses and fewer complex chronic conditions, whereas we found that our participants who had in-person visits when compared to telehealth visits who had complex chronic diseases were similar (27.5% versus 30.0%, p = .615). While our sample characteristics may differ from prior work because of the populations served, further research is needed to determine if study findings vary because of access patterns that emerged during the COVID-19 pandemic.
The second objective of this study was to identify which patient characteristics influence rehabilitation session attendance and if attendance differs by mode of service delivery. We confirmed our hypothesis and found that telehealth is associated with fewer missed rehabilitation session appointments. This is consistent with previous studies’ findings for other outpatient practice settings [9, 20, 21]. Childs et al. [21] found that appointment attendance was greater for both adolescents (75.6%) and adults (72.2%) who received telehealth versus in-person services in a psychiatric intensive outpatient program. Dayal et al. [20] found that patients who received telehealth versus in-person pediatric outpatient neurology care were more likely to attend their visits (OR = 1.46, 95% CI [1.27–1.68]). Finally, Snoswell & Comans [9] found a 68.0% reduction in missed appointments with an odds ratio of 0.32 (p<0.002) for patients who received telehealth versus in-person in an immunology outpatient clinic.
Participants who had longer length of stays have greater odds of ≥2 missed appointments (OR 1.02, 95% CI [1.01, 1.03]). Importantly, there were differences in the mean length of stays by missed appointment status for both in-person (20.11 versus 74.67, p = .041) and telehealth (43.43 versus 101.00, p = .040) groups. These findings are different than prior studies which identified that longer lengths of stay were associated with attendance in NICU follow-up appointments [22, 23]. Specifically, longer lengths of stay were associated with greater odds of attending their high-risk follow-up clinic appointments at the corrected age of 18–24 months [23] and neonatal neurodevelopmental follow-up clinic [22]. Our study findings may be different because of differences in practice settings and purpose of the visit. Swearingen et al. [22] and Kim et al. [23] looked specifically at high-risk infant follow up clinics, in which all infants who meet their criteria would get follow up with a multi-disciplinary developmental team at specific time points (usually every 6 months). This difference may also be due to a difference in patient population as Swearingen et al. [22] studied infants who were born at less than <33 weeks gestation and Kim et al. [23] looked at very low birth weight infants (<1500 grams). These studies attributed their findings to that infants with shorter length of stay are usually older in birth gestational age and less medically complex, therefore, parents may have less concerns about their infant’s development. In our present study of the UCSF BCH infant bridge program, it is possible that families of infants who have longer length of stays may be burned out from prolonged therapy services in the hospital or that they may have more appointments to manage that can either cause them to forget or need to re-prioritize their appointments, but further research is needed determine the cause of missed appointments.
We also found that there was a significant difference between the means of gestational ages and insurance types for attendance rates. Participants with a lower gestational age were associated with ≥2 missed appointments, specifically in the in-person group (36.43 days versus 32.75 days, p = .009). Similarly, Kim et al. [23] found that the follow up rates were inversely related to gestational age. On the contrary, Swearingen et al. [22] found that older gestational age was associated with not attending NICU follow-up appointments. This may be due to differences in patient population and practice setting as stated previously. We also found that participants who have public (Medi-Cal) insurance had significantly more ≥2 missed appointments than participants who have private insurance for the in-person group (57.1% versus 15.4%, p = .006) but not for the telehealth group (12.5% versus 12.5%, p = 1.000), therefore showing a potential disparity. These findings suggest that telehealth may be an option to address barriers to attendance for infants with lower gestational ages and for people who have Medi-Cal insurance.
We found that race/ethnicity was also not associated with attendance. However, small sample size with limited representation of Black and Multi-Race/Ethnicity was a limitation in this study and had to be combined into one group. Childs et al. [21] found that telehealth improved attendance across all racial and ethnic groups for psychiatric services. However, Walters et al. [24] found that there are racial disparities in telemedicine utilization, as Black patients used telemedicine less frequently than non-Black patients. Swearingen et al. [22] and Pai et al. [25] found that African American race was associated with not attending NICU follow-up appointments. Clinicians should carefully assess if telehealth may be beneficial for preterm infants who may need multisystem evaluation and rehabilitation. Future studies should include a larger sample size including a more racially and ethnically diverse sample and different preferred languages.
Study limitations
We included data from a relatively small sample of patients at one site, potentially limiting our analytic power and generalizability of our findings to other contexts and populations. We also used data from the initial stages of the COVID-19 pandemic when public policies and telehealth service delivery were changing rapidly. Future studies with larger sample sizes and across multiple sites should be considered to determine if there is an impact on attendance when patients and clinicians collaborate to choose which service delivery model best meets their needs.
Conclusion
Telehealth was associated with higher odds of rehabilitation session attendance to an infant bridge program. Thus, telehealth could be a feasible method to target attendance barriers, especially for those with longer hospital length of stay. These findings should be used to guide future research, policy, and clinical efforts to improve the quality of rehabilitative care for NICU graduates.
Supporting information
S1 Checklist. STROBE statement—Checklist of items that should be included in reports of observational studies.
https://doi.org/10.1371/journal.pone.0301219.s001
(DOCX)
References
- 1.
Centers for Disease Control and Prevention. Preterm Birth [Internet]. 2023 [cited 2024 Feb 25]. https://www.cdc.gov/reproductivehealth/maternalinfanthealth/pretermbirth.htm
- 2. Little AA, Kamholz K, Corwin BK, Barrero-Castillero A, Wang JC. Understanding barriers to early intervention services for preterm infants: Lessons from two states. Acad Pediatr. 2014;15(4):430–8.
- 3. Spittle A, Orton J, Anderson PJ, Boyd R, Doyle LW. Early developmental intervention programmes provided post hospital discharge to prevent motor and cognitive impairment in preterm infants. Cochrane Database of Systematic Reviews [Internet]. 2015 Nov 24 [cited 2020 Aug 24];(11):CD005495. Available from: http://doi.wiley.com/10.1002/14651858.CD005495.pub4 pmid:26597166
- 4. McManus BM, Carle AC, Poehlmann J. Effectiveness of part C early intervention physical, occupational, and speech therapy services for preterm or low birth weight Infants in Wisconsin, United States. Acad Pediatr. 2012;12(2):96–103. pmid:22230187
- 5. van Hus J, Jeukens-Visser M, Koldewijn K, Holman R, Kok J, Nollet F, et al. Early intervention leads to long-term developmental improvements in very preterm infants, especially infants with bronchopulmonary dysplasia. Acta Paediatr. 2016;105(7):773–81. pmid:26936312
- 6. Ochandorena-Acha M, Noell-Boix R, Yildirim M, Cazorla-Sánchez M, Iriondo-Sanz M, Troyano-Martos MJ, et al. Experiences and coping strategies of preterm infants’ parents and parental competences after early physiotherapy intervention: Qualitative study. Physiother Theory Pract [Internet]. 2020 [cited 2020 Nov 21];1–14. Available from: https://pubmed.ncbi.nlm.nih.gov/32892686/
- 7. McManus BM, Richardson Z, Schenkman M, Murphy N, Morrato EH. Timing and intensity of early intervention service use and outcomes among a safety-net population of children. JAMA Netw Open. 2019 Jan 4;2(1):e187529. pmid:30681716
- 8. Kolb CM, Born K, Banker K, Barth PC, Aaronson NL. Improving attendance and patient experiences during the expansion of a telehealth-based pediatric otolaryngology practice. Otolaryngology—Head and Neck Surgery (United States). 2021 May 1;164(5):952–8. pmid:33079014
- 9. Snoswell CL, Comans TA. Does the choice between a telehealth and an in-person appointment change patient attendance? Telemedicine and e-Health. 2021 Jul 1;27(7):733–8. pmid:32831007
- 10. Beecham L. Missed GP appointments cost NHS money. BMJ. 1999 Aug 28;319(7209):536.
- 11. Ballantyne M, Benzies K, Rosenbaum P, Lodha A. Mothers’ and health care providers’ perspectives of the barriers and facilitators to attendance at Canadian neonatal follow-up programs. Child Care Health Dev. 2015;41(5):722–33. pmid:25272331
- 12. Cada EA, O’Shea RK. Identifying barriers to occupational and physical therapy services for children with cerebral palsy. J Pediatr Rehabil Med. 2008;1(2):127–35. pmid:21791756
- 13. Das A, Cina L, Mathew A, Aziz H, Aly H. Telemedicine, a tool for follow-up of infants discharged from the NICU? Experience from a pilot project. Journal of Perinatology [Internet]. 2020 [cited 2020 Sep 28];40(6):875–80. Available from: pmid:31959907
- 14. Clark RR, Fischer AJ, Lehman EL, Bloomfield BS. Developing and implementing a telehealth enhanced interdisciplinary pediatric feeding disorders clinic: A program description and evaluation. J Dev Phys Disabil. 2019;31(2):171–88.
- 15.
McGrath PJ, Watters C, Moon E. Technology in pediatric pain management. In: Finley GA, McGrath PJ, Chambers CT, editors. Bringing pain relief to children: Treatment Approaches. Totowa: Humana Press Inc; 2006. p. 159–74.
- 16. Tenforde AS, Borgstrom H, Polich G, Steere H, Davis IS, Cotton K, et al. Outpatient physical, occupational, and speech therapy synchronous telemedicine: A survey study of patient satisfaction with virtual visits during the COVID-19 pandemic. Am J Phys Med Rehabil. 2020;99(11):977–81. pmid:32804713
- 17. Peterson KM, Ibañez VF, Volkert VM, Zeleny JR, Engler CW, Piazza CC. Using telehealth to provide outpatient follow-up to children with avoidant/restrictive food intake disorder. J Appl Behav Anal. 2021;54(1):6–24. pmid:33145801
- 18. Smith MA, Tomita MR. Combined effects of telehealth and modified constraint-induced movement therapy for individuals with chronic hemiparesis. Int J Telerehabil. 2020 Mar 1;12(1):51–62. pmid:32983368
- 19. Simon TD, Cawthon ML, Stanford S, Popalisky J, Lyons D, Woodcox P, et al. Pediatric medical complexity algorithm: A new method to stratify children by medical complexity. Pediatrics. 2014;133(6):e1647–54. pmid:24819580
- 20. Dayal P, Chang CH, Benko WS, Ulmer AM, Crossen SS, Pollock BH, et al. Appointment completion in pediatric neurology telemedicine clinics serving underserved patients. Neurol Clin Pract. 2019 Aug 1;9(4):314–21. pmid:31583186
- 21. Childs AW, Bacon SM, Klingensmith K, Li L, Unger A, Wing AM, et al. Showing up is half the battle: The impact of telehealth on psychiatric appointment attendance for hospital-based intensive outpatient services during COVID-19. Telemedicine and e-Health. 2021 Aug 1;27(8):835–42. pmid:33999738
- 22. Swearingen C, Simpson P, Cabacungan E, Cohen S. Social disparities negatively impact neonatal follow-up clinic attendance of premature infants discharged from the neonatal intensive care unit. Journal of Perinatology [Internet]. 2020;40(5):790–7. Available from: http://dx.doi.org/10.1038/s41372-020-0659-4 pmid:32203182
- 23. Kim NH, Youn YA, Cho SJ, Hwang JH, Kim EK, Kim EAR, et al. The predictors for the non-compliance to follow-up among very low birth weight infants in the Korean neonatal network. PLoS One. 2018 Oct 1;13(10). pmid:30273357
- 24. Walters J, Johnson T, DeBlasio D, Klein M, Sikora K, Reilly K, et al. Integration and impact of telemedicine in underserved pediatric primary care. Clin Pediatr (Phila). 2021 Oct 1;60(11–12):452–8. pmid:34382880
- 25. Pai V V., Kan P, Lu T, Gray EE, Bennett M, Jocson MAL, et al. Factors associated with follow-up of infants with hypoxic–ischemic encephalopathy in a high-risk infant clinic in California. Journal of Perinatology. 2021 Jun 1;41(6):1347–54. pmid:33311530