Older individuals receiving home assistance are at high risk for emergency visits and unplanned hospitalization. Anticipating their health difficulties could prevent these events. This study investigated the effectiveness of an at-home monitoring method using social workers’ observations to predict risk for 7- and 14-day emergency department (ED) visits.
This was a prospective cohort study of persons ≥75 years, living at home and receiving assistance from home care aides (HCA) at 6 French facilities. After each home visit, HCAs reported on participants’ functional status using a smartphone application that recorded 27 functional items about each participant (e.g., ability to stand, move, eat, mood, loneliness). We recorded ED visits. Finally, we used machine learning techniques (i.e., leveraging random forest predictors) to develop a 7- and 14-day predictive algorithm for the risk of ED visit.
The study included 301 participants, and the HCA made 9,987 observations. Over the mean 10-month follow-up, 97 participants (32%) had at least one ED visit. Modeling techniques identified 9 contributory factors from the longitudinal records of the HCA and developed a predictive algorithm for the risk of ED visit. The predictive performance (i.e., the area under the ROC curve) was 0.70 at 7 days and 0.67 at 14 days.
Citation: Veyron J-H, Friocourt P, Jeanjean O, Luquel L, Bonifas N, Denis F, et al. (2019) Home care aides’ observations and machine learning algorithms for the prediction of visits to emergency departments by older community-dwelling individuals receiving home care assistance: A proof of concept study. PLoS ONE 14(8): e0220002. https://doi.org/10.1371/journal.pone.0220002
Editor: Lars-Peter Kamolz, Medical University Graz, AUSTRIA
Received: March 12, 2019; Accepted: July 5, 2019; Published: August 13, 2019
Copyright: © 2019 Veyron et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Visits to the emergency department (ED) and the consequent hospital admissions among elderly adults are an important public health issue [1,2]. The number of ED visits has sharply increased in recent years . In the US, almost one of every three US Emergency Medical Services (EMS) emergency responses involves an older adult , and this number is expected to continue to grow as the baby-boomer generation ages.
Prevention of unplanned hospital admission is a major focus for the treatment of the elderly. Since a large proportion of ED visits are avoidable [5–7], strategies to identify high-risk patients and enable them to be treated in outpatient care settings might help to improve the appropriate use of EDs and to control health expenditures related to EDs . Even when hospitalization is required, a planned admission in an appropriate ward is a better option than is being hospitalized following an ED visit , as long as the patient is not in a life-threatening situation. Accordingly, developing methods of predicting ED visits and identifying at-risk persons is a very promising research approach [10,11]. Particularly, patient symptom monitoring by e-health tools might improve survival and reduce the need for ED visits [12,13].
Emergencies among the elderly are mainly the result of complex situations that often comprise an acute event—the crisis—and a chronic vulnerable state that forms through the accumulation of chronic conditions frequently encountered in this population, such as medical diseases and social, psychological, or socio-economic frailty . Several studies have identified major risk factors for ED visits [7,14–16]. Based on the results of these studies, prediction tools for identifying risks for ED visits or hospitalizations have been developed. These tools include the Community Assessment Risk Screen (CARS), a simple tool to identify elderly persons at risk for ED visits or hospitalization ; the Elder Risk Assessment Index ; and the Adjusted Clinical Groups (ACG) . All three tools are based on individual characteristics recorded at a single point in time, and have been found to be effective in predicting this risk in the subsequent 12–24 months (with a predictive performance of around 0.70, as assessed by the area under the curve of the receiver operating characteristic [ROC] model). However, to our knowledge, researchers have not developed a tool capable of predicting the risk of ED visits over a shorter period.
Nurse aides can accurately use well-calibrated instruments to assess nursing home residents' functional health . In the present study, we were interested in whether observations recorded by home care aides (HCAs) using smartphone technology were capable of predicting ED visits for frail adults aged 75 or older who were receiving in-home help. We hypothesized that the longitudinal monitoring of simple items related to everyday life and recorded by HCAs during their visits could detect changes in the functional status of older adults and could contribute to identifying items related to acute or subacute health problems, and thus contribute to predicting ED visits over a short period.
Materials and methods
Design and participants
In this prospective observational cohort study, participants were recruited from among adults aged 75 and over who were living at home and receiving the assistance from an HCA at one of the six home care services located in three French departments (Essonne, Val de Marne, Loir-et-Cher). These HCAs were not healthcare professionals and typically provided assistance with non-medical tasks (e.g., help with meals, assisting with personal care, housekeeping, running errands). Dependency levels of the persons were established according to the French national instrument, which stratifies dependency level from groupe iso-ressources (GIR) 1 (very severe dependency) to GIR 6 (no dependency) . Persons with severe dependency, or those with a GIR 1 or GIR 2 dependency level, were excluded from the study since most of them were receiving daily nursing care involving surveillance and alerts to medical staff. For people with low dependency (GIR 3 to 5), daily nursing care is much less frequent, and systematic alerts to medical staff are difficult to follow. All eligible persons were invited to participate and were included if they provided consent. We identified 350 eligible persons at the participating facilities, and 301 of them (86%) consented to participate. The recruitment process occurred between June 6, 2016 and July 27, 2017, and the home visit recordings by the HCAs started on September 1, 2016 and ended on January 19, 2018.
The research protocol was approved by the official French independent ethics committee for biomedical research, the Comité de Protection des Personne Ile-de-France VI. Prior to the start of the study, participants and HCAs were informed about the nature and purpose of the study, and all provided their written consent.
HCA recruitment and demographics
A total of 142 HCAs were employed by six different home care services, of which 136 (95.7%) were enrolled in the study by home care services managers. Table 1 shows the demographics of HCAs (the specific home care services were anonymized using the notations N°1 to 6).
Immediately after participants consented to participate, we obtained information on their demographics, home characteristics, medical conditions, and medical and paramedical support from the home help facility. All this information was recorded by a coordinator on a questionnaire comprising 23 items. Table 2 shows 15 items with a completeness rate of 60% or more; the supplemental material lists the eight remaining variables (S1 Table).
We asked the HCAs to work as they usually do except for the data collection. At the end of each home visit, the HCAs recorded information about the participants via a smartphone application developed by one of the authors. First, the HCA took a photo of the personal QR code assigned to each participant, which we had placed in his/her home. This recorded the participant’s name, date, and time of the visit. Second, the HCA answered 27 simple questions related to the patient’s functional status, behavior, or support for help (Table 3). For each question, the possible answers were yes/no/do not know. We explained the purpose of the study to the HCAs, as well as providing training and a user manual on how to use the application and the exact meaning of each term used in the application. The application ensured real-time data transmission to the investigators without further dissemination, which we explained to the HCAs. The average observation time for participants was 10.7 months (SD = 3.5 months).
With the help of EDs around the facilities, we identified participants’ ED visits using systematic reviews of their registers. For each ED visit, an investigator recorded the date, the reasons for the visit, and the visit outcome (i.e., hospital admission or not).
Predictive modeling for ED visits
We randomly selected participants to form the training sample used to build predictive models and then assessed the relevance of the models from the remaining participants (the test sample) with the events that had not been used to build the model. We excluded poorly filled variables (i.e., completeness below 60%) from the modeling.
Using machine learning techniques (random forests), we developed several predictive models using baseline data and the home visits variables to predict ED visit with a 7 or 14-day horizon. In order to respect the proportions between ED visits and no ED visits, the learning sample was composed of 1000 negatives / 20 positives and the test sample of 500 negatives / 10 positives.
Note that several machine learning techniques were actually attempted, including linear and logistic regressions, as well as random forests. However, we obtained the best results, particularly in terms of generalization, with the random forests. It is also noteworthy that we engineered the features passed on to the machine learning model. That is, we built a so-called temporal variance matrix that aggregates all the questions into one value for each week and learned from the evolution of that value from week to week. The best results were obtained when making predictions using those features.
We evaluated the performance of the models according to the Gini index (the area under the curve [AUC]) as well as the local area under the curve (for the 1% of individuals predicted to have the highest ED visit risk).
Table 2 details the characteristics of the participants and the home care organization. The participants’ mean age was 88 years (SD = 5.8 years), and the sample included 226 women (75.1%). During the study period, 32 participants (11%) dropped out from the study before the end of the follow-up, and among them, 19 had an ED visit before dropout. The reasons for dropouts were death (n = 14), nursing home admission (n = 5), and other reasons such as stopping home help service (n = 13).
Feasibility and completeness of records
There was heterogeneity in the completeness of the baseline data recorded by the home care service coordinators, ranging from 20% to 100%. Table 2 lists the variables related to dependency and the social factors with high completeness rates. By contrast, the variables related to medical status, like the number of prescribed drugs, history of previous hospital admission or emergencies visits, as well as the occurrence of actual paramedic interventions had a low completeness rate. There were eight variables with a completeness rate below 60% (listed in supplements), which we excluded from the modeling.
Using smartphones, 136 HCAs recorded data from 9987 home visits, which corresponds to 31% of the expected home visits during the follow-up period. The mean completeness rate of variables was 85%, and no variable had a completeness rate of < 60% (Table 3). Thus, all the variables were included in the models.
ED Visits and hospital admissions
During the follow-up, 97 participants (32.5%) visited EDs, and among them, 35 went two or more times (up to 5 visits); thus, overall, 152 ED visits were recorded (Table 4). Hospital admission following an ED visit occurred for 60 participants (20% of the overall cohort and 61% of those who visited an ED). During the same period, 19 participants were hospitalized by a direct admission without any ED visits.
Construction and evaluation of the predictive models
First, we compared participants’ characteristics according to the presence/absence of ED visits during follow-up (Table 5). Then, using all the variables selected based on completeness, we developed several predictive models and assessed their performance in predicting 7-day and 14-day visits to ED. We obtained the best 7-day predictive model by using the data from nine variables recorded by the HCA observations (Table 3), whereas the data from baseline variables did not improve the accuracy of the models. The global performance assessed by the area under the curve of the ROC curve was 0.70 for the 7-day predictive model and 0.63 for the 14-day predictive model. The slope of the curve between the origin and 3% (positive likelihood ratio) for the 7-day model was 16.4 (Fig 1A and 1B). For the 7-day model, the sensitivity and specificity were 36% and 98%, respectively. The positive predictive and negative predictive values were 21% and 99%, respectively.
(a) ROC curve for 7-day predictive model and (b) ROC curve for 14-day predictive model.
Among the nine variables having a significant contribution to the realized model (Table 3), two were related to activities of daily living (i.e., the person is able to move in his/her home, has prepared his/her meal), four to possible medical symptoms (i.e., the person seems feverish, is painful, has trouble breathing, has fallen), and the remaining three to communication and social support (i.e., the person communicates little, seems sad, has no visit of, or contact with family members).
In this proof of concept study with frail elderly persons who were receiving home help (social assistance), we found that it is feasible to organize the longitudinal recording of patients’ functional status by their HCAs over a period of several months, by using a simple smartphone application at each home visit. We also found that these records can predict the risk of ED visit within the following 7 and 14 days. Our findings open the way to develop an innovative approach for the implementation of new systems for real-time monitoring of the risk for ED visits and to expand anticipatory interventions that decrease the risk for unplanned hospital admission in this high-risk population.
We conducted our study with elderly participants (i.e., ≥75) who were experiencing mild to moderate dependency and receiving home care. Their risk for ED visits or hospital admission was high, as indicated by the fact that 33% visited an ED, and 20% were hospitalized during this 10-month survey. A large study by Jones et al.  with a cohort of 32,253 elderly persons receiving home care in Ontario also observed a high incidence rate of ED visits and hospitalizations, even if those rates were lower than seen in our sample. We took advantage of the help delivered by home care services by asking HCA to record simple items related to the functional status of the participants at each home visit. The high rate of participation of the aides is a strength of our study since HCA are not health care professionals and have typically not been involved in monitoring symptoms and participating in research . The successful participation in this study is likely due to the ease of completion, with simple items and the use of a friendly app installed on a smartphone. For those elderly persons using in-home care, the involvement and empowerment of HCAs in health monitoring is a promising way to improve the management of chronic disease and medical events without increasing health expenditures cost. To our knowledge, researchers have not reported such an approach for home care, and our study shows that HCA involvement is feasible even over a long period of time.
In our study, we obtained longitudinal data on the functional status of the participants, and in line with our hypothesis, found that changes in several variables were predictive of a 7- or 14-day ED visit. Interestingly, the predictive variables comprise items related to possible medical symptoms (e.g., having trouble breathing), to functional status (e.g., changes in the ability to move in the home) as well as items related to mood problems or loneliness. All these items might reflect the initiation of a health event and/or a worsening of the health status, which might expose the participant to a crisis and a visit to ED. Jones et al.  observed that, among people receiving home care, the likelihood of visiting the ED was greater on the day of the home visit compared to the days without a visit. This was probably related to the identification of medical problems by HCAs when they visit participants in their care, which can lead to the alerting of patients’ family caregivers or physicians, and subsequently to ED visits. However, this association was observed only with nursing visits but not with HCA visits, probably because nurses are more capable of recognizing medical problems and alerting others when required.
The AUC (0.70) in our study was comparable to those found in other state-of-the-art studies using machine learning techniques to predict future behavior. Recent examples include the prediction of sepsis with an AUC of 0.67  and the prediction of delirium with an AUC of 0.68 
More importantly, the slope of the ROC at origin for the 7-day model was 16.4, and had a positive likelihood ratio of 3%. This shows that we can very accurately predict a certain fraction of ED visits. Certain causes of ED visits (e.g., acute decompensation, accidents) could not be foreseen by any means one week in advance. Thus, the sensitivity reflects the fact that we predict a complex response (all causes of ED visits within seven days) from very little data (27 simple observations, once per week). We further believe that a sensitivity of 36% is very significant already. In addition, further studies could establish a threshold value, which results from the status evaluation by social workers. Medical staff (e.g., nurses) should visit the respective person on the very same day and after further investigation should have to decide whether a family doctor must be summoned, whether an immediate clinic visit is necessary, or whether an appointment (e.g., in specialty clinics) would have to be made on the following day or in the next few days.
The study has a number of limitations. First, we had little medical information from participants, although several medical factors were found to predict emergency room visits over a 12-month period.[16,24–26]. In our study, we attempted to record information about participants’ medical status on our baseline questionnaire by the home care services coordinators, but we obtained a low completeness rate. This rate might be because these professionals had a low degree of knowledge regarding the medical status of the participants. Despite this limitation, it is impressive that the longitudinal records on functional status were a significant predictor of ED visits in the short term, even without precise information about their chronic diseases, drug utilization, and previous hospitalizations or ED visits. To the extent that the nature of the information recorded by home support workers is very different from that contained in conventional medical information, we believe that combining the two types of information could improve the predictive model. Second, the sample size of our study is small to conduct powerful machine learning studies. However, the promising results of this proof-of-concept study encourage us to develop this approach in larger populations, which will lead not only to validate the approach, but also to improve the predictive model.
These findings highlight a novel approach to improving the predictive performance of the models and recognizing more efficient tools for the short-term prediction of ED visits—namely, by including both baseline records about chronic diseases and longitudinal records about symptoms and functional status.
In summary, we conclude that HCA observations can successfully contribute to the short-term prediction of ED visits by elderly persons receiving home care and that this approach together with long-term prediction models might constitute a novel method of predicting ED visits. This method may promote real-time interventions capable of reducing avoidable ED utilization in this high-risk population.
S1 Table. List of the eight baseline items with a completeness rate below 60%.
These variables were not considered in the models.
S2 Table. Data for 7-days ROC and 14-days ROC.
The authors thank the coordinators and the staff of the home care services for their contribution to the study, and the directors of the emergencies wards for their help in identifying visits of the participants.
- 1. Kahn JH, Magauran BG, Olshaker JS, Shankar KN. Current trends in geriatric emergency medicine. Emerg Med Clin North Am. 2016;34: 435–452. pmid:27475008
- 2. Wolinsky FD, Liu L, Miller TR, An H, Geweke JF, Kaskie B, et al. Emergency department utilization patterns among older adults. J Gerontol A Biol Sci Med Sci. 2008;63: 204–209. pmid:18314459
- 3. Tang N, Stein J, Hsia RY, Maselli JH, Gonzales R. Trends and characteristics of US emergency department visits, 1997–2007. JAMA. 2010;304: 664–670. pmid:20699458
- 4. Duong HV, Herrera LN, Moore JX, Donnelly J, Jacobson KE, Carlson JN, et al. National characteristics of emergency medical services responses for older adults in the United States. Prehosp Emerg Care. 2018;22: 7–14. pmid:28862480
- 5. Adams JG. Emergency department overuse: perceptions and solutions. JAMA. 2013;309: 1173–1174. pmid:23512065
- 6. Uscher-Pines L, Pines J, Kellermann A, Gillen E, Mehrotra A. Emergency department visits for nonurgent conditions: systematic literature review. Am J Manag Care. 2013;19: 47–59. pmid:23379744
- 7. Gasperini B, Cherubini A, Pierri F, Barbadoro P, Fedecostante M, Prospero E. Potentially preventable visits to the emergency department in older adults: Results from a national survey in Italy. PloS One. 2017;12: e0189925. pmid:29267333
- 8. Fingar KR, Barrett ML, Elixhauser A, Stocks C, Steiner CA. Trends in Potentially Preventable Inpatient Hospital Admissions and Emergency Department Visits: Statistical Brief #195. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006. Available: http://www.ncbi.nlm.nih.gov/books/NBK338139/
- 9. Hwang U, Morrison RS. The geriatric emergency department. J Am Geriatr Soc. 2007;55: 1873–1876. pmid:17916122
- 10. Jin B, Zhao Y, Hao S, Shin AY, Wang Y, Zhu C, et al. Prospective stratification of patients at risk for emergency department revisit: resource utilization and population management strategy implications. BMC Emerg Med. 2016;16: 10. pmid:26842066
- 11. Hu Z, Jin B, Shin AY, Zhu C, Zhao Y, Hao S, et al. Real-time web-based assessment of total population risk of future emergency department utilization: statewide prospective active case finding study. Interact J Med Res. 2015;4: e2. pmid:25586600
- 12. Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. 2017;318: 197–198. pmid:28586821
- 13. Denis F, Basch E, Septans A-L, Bennouna J, Urban T, Dueck AC, et al. Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer. JAMA. 2019;321: 306–307. pmid:30667494
- 14. Crane SJ, Tung EE, Hanson GJ, Cha S, Chaudhry R, Takahashi PY. Use of an electronic administrative database to identify older community dwelling adults at high-risk for hospitalization or emergency department visits: the elders risk assessment index. BMC Health Serv Res. 2010;10: 338. pmid:21144042
- 15. Takahashi PY, Heien HC, Sangaralingham LR, Shah ND, Naessens JM. Enhanced risk prediction model for emergency department use and hospitalizations in patients in a primary care medical home. Am J Manag Care. 2016;22: 475–483. pmid:27442203
- 16. Parameswaran Nair N, Chalmers L, Connolly M, Bereznicki BJ, Peterson GM, Curtain C, et al. Prediction of hospitalization due to adverse drug reactions in elderly community-dwelling patients (the PADR-EC score). PLoS ONE. 2016;11. pmid:27798708
- 17. Shelton P, Sager MA, Schraeder C. The community assessment risk screen (CARS): identifying elderly persons at risk for hospitalization or emergency department visit. Am J Manag Care. 2000;6: 925–933. pmid:11186504
- 18. Hartig MT, Engle VF, Graney MJ. Accuracy of nurse aides’ functional health assessments of nursing home residents. J Gerontol A Biol Sci Med Sci. 1997;52: M142–148. pmid:9158555
- 19. Belmin J, Auffray J-C, Berbezier C, Boirin P, Mercier S, de Reviers B, et al. Level of dependency: a simple marker associated with mortality during the 2003 heatwave among French dependent elderly people living in the community or in institutions. Age Ageing. 2007;36: 298–303. pmid:17384419
- 20. Jones A, Schumacher C, Bronskill SE, Campitelli MA, Poss JW, Seow H, et al. The association between home care visits and same-day emergency department use: a case-crossover study. CMAJ. 2018;190: E525–E531. pmid:29712671
- 21. Thome B, Dykes A-K, Hallberg IR. Home care with regard to definition, care recipients, content and outcome: systematic literature review. J Clin Nurs. 2003;12: 860–872. pmid:14632979
- 22. Horng S, Sontag DA, Halpern Y, Jernite Y, Shapiro NI, Nathanson LA. Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning. PloS One. 2017;12: e0174708. pmid:28384212
- 23. Wong A, Young AT, Liang AS, Gonzales R, Douglas VC, Hadley D. Development and validation of an electronic health record–based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment. JAMA Netw Open. 2018;1. pmid:30646095
- 24. de Gelder J, Lucke JA, de Groot B, Fogteloo AJ, Anten S, Heringhaus C, et al. Predictors and outcomes of revisits in older adults discharged from the emergency department. J Am Geriatr Soc. 2018;66: 735–741. pmid:29489015
- 25. Hao S, Jin B, Shin AY, Zhao Y, Zhu C, Li Z, et al. Risk prediction of emergency department revisit 30 days post discharge: a prospective study. PloS One. 2014;9: e112944. pmid:25393305
- 26. Pishgoo B. A novel prediction model for all cause emergency department visits in ischemic heart disease. J Res Med Sci Off J Isfahan Univ Med Sci. 2011;16: 262–268.