Skip to main content
Advertisement
  • Loading metrics

The law of non-usage attrition in a technology-based behavioral intervention for black adults with poor cardiovascular health

  • Muhammed Y. Idris ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft

    myidris@msm.edu

    Affiliations Department of Medicine, Morehouse School of Medicine, Atlanta, Georgia, United States of America, Clinical Research Center, Morehouse School of Medicine, Atlanta, Georgia, United States of America

  • Mohamed Mubasher,

    Roles Methodology, Writing – review & editing

    Affiliations Clinical Research Center, Morehouse School of Medicine, Atlanta, Georgia, United States of America, Community Health and Preventive Medicine, Morehouse School of Medicine, Atlanta, Georgia, United States of America

  • Ernest Alema-Mensah,

    Roles Data curation, Investigation

    Affiliations Clinical Research Center, Morehouse School of Medicine, Atlanta, Georgia, United States of America, Community Health and Preventive Medicine, Morehouse School of Medicine, Atlanta, Georgia, United States of America

  • Christopher Awad,

    Roles Writing – original draft

    Affiliation Department of Medicine, Emory University School of Medicine, Atlanta, Georgia, United States of America

  • Kofi Vordzorgbe,

    Roles Writing – original draft

    Affiliation Department of Medicine, Emory University School of Medicine, Atlanta, Georgia, United States of America

  • Elizabeth Ofili,

    Roles Funding acquisition, Supervision

    Affiliation Department of Medicine, Morehouse School of Medicine, Atlanta, Georgia, United States of America

  • Arshed Ali Quyyumi,

    Roles Funding acquisition, Writing – review & editing

    Affiliations Department of Medicine, Emory University School of Medicine, Atlanta, Georgia, United States of America, Emory Clinical Cardiovascular Research Institute, Emory University, Atlanta, Georgia, United States of America

  • Priscilla Pemu

    Roles Conceptualization, Funding acquisition, Writing – review & editing

    Affiliations Department of Medicine, Morehouse School of Medicine, Atlanta, Georgia, United States of America, Clinical Research Center, Morehouse School of Medicine, Atlanta, Georgia, United States of America

Abstract

Digital health innovations, such as telehealth and remote monitoring, have shown promise in addressing patient barriers to accessing evidence-based programs and providing a scalable path for tailored behavioral interventions that support self-management skills, knowledge acquisition and promotion of relevant behavioral change. However, significant attrition continues to plague internet-based studies, a result we believe can be attributed to characteristics of the intervention, or individual user characteristics. In this paper, we provide the first analysis of determinants of non usage attrition in a randomized control trial of a technology-based intervention for improving self-management behaviors among Black adults who face increased cardiovascular risk factors. We introduce a different way to measure nonusage attrition that considers usage over a specific period of time and estimate a cox proportional hazards model of the impact of intervention factors and participant demographics on the risk of a nonusage event. Our results indicated that not having a coach (compared to having a coach) decreases the risk of becoming an inactive user by 36% (HR = .63, P = 0.04). We also found that several demographic factors can influence Non-usage attrition: The risk of nonusage attrition amongst those who completed some college or technical school (HR = 2.91, P = 0.04) or graduated college (HR = 2.98, P = 0.047) is significantly higher when compared to participants who did not graduate high school. Finally, we found that the risk of nonsage attrition among participants with poor cardiovascular from “at-risk” neighborhoods with higher morbidity and mortality rates related to CVD is significantly higher when compared to participants from “resilient” neighborhoods (HR = 1.99, P = 0.03). Our results underscore the importance of understanding challenges to the use of mhealth technologies for cardiovascular health in underserved communities. Addressing these unique barriers is essential, because a lack of diffusion of digital health innovations exacerbates health disparities.

Author summary

Mobile-health (mHealth) technologies continue to serve immensely valuable roles in current healthcare systems by bridging several accessibility gaps and facilitating patient-provider interactions. However, and ironically, they also present several accessibility gaps due to design features and/or individual characteristics. In this study, we explore the effects of having an adjunct health coach with a behavior-modification mHealth app, Health360x (H360x) in promoting user-interface interaction in a population of African American patients with significant cardiovascular (CV) risk factors. While we saw an improvement in CV risk factors, we also identified an interesting attrition in app usage in patients assigned a health coach; and observed similar effects in those with higher socioeconomic status. These findings may be due to a higher (baseline or acquired) self-efficacy in these patients, and suggests that longitudinal assessments of app engagement in this population should be approached with careful consideration of attrition.

Introduction

Cardiovascular disease (CVD) is the leading cause of death in the United States and disproportionately impacts racial and ethnic minorities, particularly Black/African American patients, who have a CVD risk factor-adjusted death rate 30% higher than their non-Hispanic white counterparts [1]. African American patients face disparities in incidence of CVD-related events, including stroke, myocardial infarction, heart failure, and acute coronary syndrome, leading to significant functional impairment and mortality [2]. Risk factors associated with CVD-related disparities in Black communities include higher prevalence of hypertension, diabetes, and obesity [3]. Disparities in health behaviors, including physical activity, smoking, greater than moderate drinking, unhealthy diets, and psychological distress, are associated with these risk factors [3]. Lifestyle changes (i.e., physical activity, diet, adherence to medication, and smoking) and adherence to cardiac rehabilitation have a significant impact on the risk of future CVD events, but even these are hindered by marked racial and ethnic disparities in participation in the use of cardiac rehabilitation among eligible patients in the U.S. [4].

Previous research has shown that tailored behavioral interventions can contribute to the improvement of key lifestyle factors that contribute to ideal cardiovascular health among Black/African American patients [5, 6]. However, targeting and tailoring lifestyle interventions for preventing and managing CVD can be labor intensive, cost prohibitive, and inaccessible to underserved patient populations. Recent technological innovations, including telehealth and remote monitoring, have shown promise in addressing patient barriers to accessing evidence-based programs and providing a scalable path for tailored behavioral interventions that support self-management skills, knowledge acquisition and promotion of relevant behavioral change [7]. However, significant attrition continues to plague internet-based studies, a result that can be attributed to characteristics of the intervention, or individual user characteristics.

Intervention factors include familiarity and adaptability to remotely connected smart devices, usability problems or complexity, ease of discontinuance, not observing early advantage to its use as well as the existence of a trusted coach [5, 8]. Individual characteristics that contribute to attrition include demographic factors like socioeconomic status, education, less contact with change agents and poor access to technology [25]. This is even more noticeable among minority and underserved populations, who experience challenges accessing reliable health sources on the web, navigating technology, and accessing sources in their respective language and appropriate reading levels in order to make informed health decisions [9]. While others have systematically examined attrition in controlled trials [10] and real-world applications [11] of behavioral intervention technologies (BITs), there are currently no studies that consider the role of race/ethnicity on attrition rates, engagement measures, and associated variables. Moreover, it is important to distinguish between dropout attrition (i.e., participants are lost to follow-up) and nonusage attrition (i.e., participation in the study, but are not engaging with BITs). As Eysenbach (2005) notes, “the loss-to-follow-up attrition curve usually follows the nonusage attrition curve because a high proportion of loss to follow-up is a result of nonusage (“losing interest” is the underlying variable which explains both curves)” [12].

In this paper, we provide the first analysis of the determinants of nonusage attrition in a randomized control trial of a technology-based intervention for improving self-management behaviors among Black adults with multiple cardiovascular risk factors. In what follows we introduce the Morehouse-Emory Cardiovascular (MECA) Study, which investigated multi-level exposures that explain intraracial and intraethnic differences in cardiovascular health outcomes. We then describe our methodology for measuring nonusage attrition from user logs exported from Health360x, a coach-assisted consumer health information technology (CHIT) designed to support chronic illness care through behavior change. We present attrition curves of nonusage over the 6-month intervention overall and as a function of the intervention arm. Our hypothesis is that attrition is influenced by a combination of intervention and individual characteristics.

Methods

Morehouse-Emory Cardiovascular (MECA) clinical intervention

The data used to assess the determinants of attrition with a behavioral intervention technology among Black adults with poor cardiovascular health was taken from a clinical intervention as part of the Morehouse-Emory Cardiovascular (MECA) Center for Health Equity Study [13]. The parent MECA study was primarily concerned with exploring intraethnic and intraracial differences in cardiovascular health outcomes. The Institutional Review Boards of both Morehouse School of Medicine and Emory University School of Medicine approved this research under an IRB Authorization Agreements #784479-2 and #00083584, respectively. Consent was obtained orally and documented by research coordinators in participant surveys. Written consent could not be obtained because participants were enrolled via telephone.

Poor cardiovascular health was defined by calculating a participant’s Life’s Simple 7 score, seven CVD risk factors as defined by the American Heart Association (AHA): smoking status, physical activity, weight, diet, blood glucose, cholesterol, and blood pressure [14]. Research has shown that patients with ideal scores in at least five risk factors over 4-7 years experienced 78% and 88% reductions in all-cause mortality and mortality associated with CVD, respectively [15]. Participant risk factors were measured during baseline visits in the parent MECA study. LS7 scores were calculated using guidelines set forth by the AHA, with score ranges between 0–14, giving 2 points for Ideal, 1 point for Intermediate, and 0 points for Poor components [16]. All participants in the parent MECA study with low LS7 scores (i.e., an LS7 composite score of 8 or lower) who agreed to be contacted for the clinical intervention study were called and consented by the research staff. Additional inclusion and exclusion criteria can be found in [5].

The goal of the 6-month MECA clinical intervention (the focus of this manuscript) was to influence lifestyle changes that are associated with ideal cardiovascular health, while addressing barriers to technology-enabled self-management of CVD through the use of trained health coaches. All participants received a web- and mobile-based behavioral intervention technology, Health360x, and were randomized into two groups; one group receiving an adjunct health coach (high tech-high touch) and the other using only Health360x (high tech). All participants were trained to use Health360x to monitor and manage cardiovascular risk factors including, blood sugar, hypertension, BMI, and total cholesterol, while improving their lifestyles in terms of diet, tobacco use, and physical activity. Before being randomized, all participants were trained on Health360x and assessed on their knowledge and ability to use Health360x. In a previous study, we’ve shown the benefit of engagement with Health360x and the role that health coaches can play in empowering participants to develop self-management skills that were related to improvements in clinical outcomes including reductions in blood pressure and blood glucose levels [8]. We tested the hypothesis that an intervention that combines technology with a trusted partner (high tech-high touch) will be more effective in engaging patients, increasing attainment of self-management goals, and ultimately improve cardiovascular health. Our results from this multi-faceted technology-enabled and coach-assisted clinical intervention indicated that our intervention can lead to improvements in cardiovascular health (as measured by LS7) corresponding to a 7% lifetime reduction of CVD incidence CV, and that health coaches can help improve overall LS7 in participants living in at-risk neighborhoods [5].

Health360x: Coach-assisted platform for chronic illness care

Health360x (Accuhealth, Atlanta, USA) is a proprietary application developed by researchers at Morehouse School of Medicine to support self-efficacy for self-management of chronic illnesses. Fig 1 provides screenshots of Health360x. This patented technology provides a variety of functionality ranging from built-in coaching to help overcome digital and health literacy barriers and promote self-efficacy; social network to promote motivation and community; illness-specific content to empower users with knowledge to improve their health condition; and a tracking system to record blood pressure, BMI, physical activity, and self-management through manual input or remote monitoring. All consented participants were trained on Health360x and then completed a brief usability test which included task scenarios around logging on, uploading and viewing data, setting and tracking behavioral goals, finding and completing study-related questionnaires, sending emails to the study team, posting to an online community forum, and accessing educational content. Participants also completed a brief test of their ability to use the core functionality of Health360x after training. All participants were had access to 24/7 technical support, but participants in the high tech-high touch group also had access to coaches to provide additional support with technical assistance.

Underlying study design considerations: Treatment assignment.

The intervention is designed to compare two approaches to engagement and in turn behavior change: technology alone (high tech only) versus technology coupled with a health coach (high tech-high touch). Only participants who successfully completed a usability test were randomly assigned to an intervention arm using a time-varying-permuted block size randomization algorithm with stratification based on gender and neighborhood/community prevalence of cardiovascular morbidity and mortality. Participants in the high tech-high touch group were randomly assigned to work with a coach in person or by phone every week for the 1st month, then every two 2 weeks for 8 weeks, and finally monthly for 3 months to develop and adapt a personalized action plan to monitor and manage CVD risk and improve lifestyles.

Covariates (Mediators, effects Modifiers analyses variables).

Participant demographics. Participant demographics including age, gender, marital status, income, education, and employment were collected during patient visits at Emory University Hospital and Morehouse School of Medicine during the baseline MECA clinical study. To determine whether a patient lived in a neighborhood/community that enhances or reduces CVD risk, participant addresses were mapped to census tracts. Census-tract level data on CVD-related related deaths and emergency department admissions for black adults between 35 and 64 from 2010 to 2014 was used to classify communities as ‘at risk’ for or ‘resilient’ against cardiovascular morbidity and mortality [17]. Finally, we included a participant’s baseline Life’s Simple 7 score as an independent variable.

Primary analysis outcome: Nonusage attrition. This was defined as a single or multiple events, thus measuring the intensity of nonusage. To measure nonusage attrition, we exported usage logs of time-stamped events for all participants in the study from Health360x. We then filtered usage logs to only include events that are relevant to the primary function of the app, including logging weight, blood pressure, blood glucose, medication, cholesterol, and appointments as well as creating/modifying action plans, sending messages, and filling out study specific surveys. We measured nonusage attrition by considering both the duration and intensity of usage. We could not find any literature that proposed a standard approach for defining a threshold for meaningful usage, so we elected to use the first quartile of the number of events by users as a cutoff. A participant was coded as inactive and a nonusage event was logged if they did not log a minimum of 9 relevant events in the last 24 hours. It is possible that a participant does not experience any nonusage events if they never reach the threshold of 9 events (i.e., a user logs in and somehow interacts with the app, but doesn’t perform any of the previously mentioned list of events that we would record as a ‘usage event’). It is also possible that participants can oscillate between being active and inactive users which is a natural feature of engagement with software applications. As such, it is possible to also possible observe multiple nonusage events for a given participant even if they may indeed be engaging with the app.

Research ethics

All procedures and protocols were conducted in accordance with the Emory University and Morehouse School of Medicine Institutional Review Boards (IRBs). Briefly, participants enrolled in the MECA study were consented in by phone and expressed adequate understanding of the study. Researchers were also adequately trained in educating participants and ensuring proper handling and security of patient data throughout the study. The digital data used for analysis in this article was highly secured, with security measures monitored annually by a an advisory/oversight committee to address any potential adverse events. Throughout this study, data access was restricted only to the appropriate investigators as required by IRB guidelines.

The study was also designed to ensure an adequate representation of different minority groups within the population in question. Patient demographics were estimated from statistics reported by the Morehouse Choice Accountability Care Organization; and we anticipated a 60:40 female to male ratio, with close to 65% of participants belonging to a minority group. While several demographic factors could potentially impact our outcomes, we did not discover any notable differences in our population in some of these variables (Table 1). However, we recognize that other factors like digital literacy and self-efficacy could strongly influence attitudes towards mHealth apps. We sought to address these by training our participants in the use of the app to a minimum competency level and randomly assigned them to having a coach or not. By design, our outcome measures analyzed the frequency of interaction with the app, and less on the proficiency of the user. Additionally, randomly assigning proficient and less proficient users to coaches would aim at reducing the effect of self-efficacy while highlighting the role of having a coach on usage attrition.

While the study focuses on a specific patient population (i.e. elderly African-Americans with CVD risk factors), we hope that the insights gained would be useful in understanding the efficacy of interventional and therapeutic designs in achieving their goals based on demographic features of the target population. mHealth apps provide a unique challenge in the domain of patient-provider interactions because they serve to automate one or more aspects of this relationship. Understanding the role of demographic factors on the efficacy of these apps-based interventions, as endeavored by this study, is essential in ensuring an improved and worthwhile experience.

Statistical considerations

Statistical power.

Using nonusage attrition as a single event we estimated statistical power using variable scenarios of hazard ratios of events between the two high-tech and high tech-high touch groups.

Statistical analysis.

Participant demographics. The participants’ characteristics, including demographics, baseline, and 6-month changes in CVD risk variables were summarized and compared by randomization groups (high tech vs. high tech-high touch). Binary/categorical variables like sex at birth, education, and the presence of diabetes were summarized using frequencies and percentages. Continuous variables like age, BMI, and blood glucose were summarized by means and standard deviations. We univariately tested balance of patients characteristics and predictive covariates between the two randomization groups using univariate logistics function with the likelihood of being on one of the randomization groups as an outcome vis-à-vis each of the potential predictors of the outcome measures, to determine any association with patient’s baseline characteristics. Statistically insignificant p-values indicate that the patient characteristic (e.g., age) is not correlated with likelihood of assignment to randomization group suggesting comparability between groups.

Nonusage attrition. As suggested by [12], formal analysis of attrition can be done using survival analysis to assess the association between predictor variables and the time until a nonusage attrition event occurs. We plot attrition curves and test the null hypothesis of no difference in survival between stratified groups using the log rank test. The semi-parametric Cox proportional-hazards regression model is the workhorse of survival modeling and is widely used in biomedical research to investigate the influence of multiple independent variables on time-to-event of interest [18]. In this simplest form, the Cox model is often used to model single events. We also tested for intensity of nousage attrition, since it is likely that participants will oscillate between being active and inactive users and thus the event of interest (nonusage attrition), is likely to occur more than once in a participant, a phenomenon known as intensity of or recurrent events. Several methods have been proposed to analyze adapt the Cox model for multiple events and differ in their assumptions about the dependency of events. We elected to use a frailty model because it assumes events are independent except for subject-specific random effects which account for unmeasured heterogeneity that cannot be described by predictors in the model alone [19]. Like linear mixed-effects models, regression estimates in the Cox regression model with random effects are subject-specific and should be interpreted as such. Sample size permitting we explored interactions between treatment assignment (i.e., Coach, No Coach) and a participant’s baseline Life’s Simple 7.

Results

Table 1 presents demographics in our overall sample of 146 participants in the MECA clinical intervention study. Fig 2 represents the frequency of user events or actions relevant to the primary function of Health360x (i.e., changing passwords vs. logging blood pressure). The number of relevant user actions over the course of the study ranged from 2 to 522, with an average of 38.27 (SD = 62.31) events per MECA participant.

thumbnail
Fig 2. Histogram of user events relevant to the primary function of Health360x.

https://doi.org/10.1371/journal.pdig.0000119.g002

Fig 3 presents nonusage attrition curves of participants in the MECA Clinical Intervention, where the y axis is the log of the proportion of users remaining who have not experienced a nonusage event. Fig 3A indicates that the median time to nonusage attrition when looking at the overall sample is 13 days. Fig 3B stratifies participants by intervention group. The median survival time for participants in the high tech-high touch group (i.e., Coach) is 9.5 days, while the median survival time in the high tech group (i.e., No Coach) is 20.5 days. Non-intersecting attrition curves when stratifying by intervention group indicate statistically significant differences in survival between strata. We can therefore conclude that there is a significant difference in nonusage attrition between the intervention groups. Fig 3C and 3D compare attrition curves by gender and baseline Life’s Simple 7 scores, respectively. The median survival time for male participants was 23 days and female participants 11 days. The median survival time for participants with baseline LS7 scores greater than or equal to 6 was 15 days, and those with LS7 scores smaller than 6 was 12 days. Results from log rank tests indicate that there is no statistically significant differences in attrition based on gender (P = 0.99) or baseline LS7 (P = 0.60), but there is a significant difference in attrition as a function of (P = 0.04).

thumbnail
Fig 3. Logarithmic nonusage attrition curves in the MECA clinical intervention.

https://doi.org/10.1371/journal.pdig.0000119.g003

Table 2 presents estimates from a Cox proportional hazard model of participant nonusage attrition conditional on unmeasured heterogeneity and intervention and participant characteristics. Model estimates suggest that compared to the Coaching intervention group, there is a 36% decrease in the risk of a participants in No Coach intervention group moving from being active to inactive users (HR = 0.64, P = 0.04). Results also suggest that the risk of nonusage attrition among participants who completed some college or technical school (HR = 2.91, P = 0.04) or graduated college (HR = 2.98, P = 0.047) is 3 times higher than those who did not graduate high school. Finally, we find that the risk of nonusage attrition among participants with poor CVD health from at risk neighborhoods is approximately two times the risk of participants from resilient neighborhoods (HR = 1.99, P = .03). We did not find evidence that marital status, income, or employment status are significant predictors of nonusage attrition in this population. We tested the proportional hazards assumption for the Cox regression model fit by checking for time dependency using the Schoenfeld residuals against the transformed time for reach predictor in our model. No predictors were statistically significant indicating that our model meets the proportional hazards assumption.

thumbnail
Table 2. Estimates of the Cox proportional hazard model of nonusage attrition.

https://doi.org/10.1371/journal.pdig.0000119.t002

Discussion

Principal findings

In this study we have presented a first-of-its-kind analysis of nonusage attrition in a technology-based behavioral intervention for Black/African American patients with poor cardiovascular health. Despite a high retention rate of over 90% in the study, we found that having a coach was likely to increase the risk of nonusage. We also found that several demographic factors can influence Non-usage attrition: Higher educational attainment was associated with greater rate of nonusage. This finding suggests, counterintuitively, that education may not play a protective role in promoting engagement with behavioral intervention technologies within Black communities. Finally, we found that the risk of nonusage attrition among participants with poor cardiovascular from “at-risk” neighborhoods with higher morbidity and mortality rates related to CVD is significantly higher when compared to participants from “resilient” neighborhoods. We also explored interaction effects, specifically the interaction treatment group by baseline Life’s Simple 7 dichotomized by the median of participant scores and the result did not show any significant difference in nonusage attrition. There was some imbalance between treatment groups based on income (specifically participants whose income was between $50,000 and $75,000, but income was not a significant predictor in our Cox regression model.

Some key considerations in mHealth app usage are prior digital literacy, technological self-efficacy, and perceived utility especially on the part of the participants. While we ensured a minimum competency level in navigating the app, we recognize that this did not guarantee a lack of an effect on app usage. We sought to minimize this effect by analyzing the frequency of interaction with the app regardless of the proficiency of the user so long as they appropriately completed a required basic function. The app was also designed with careful consideration of accessibility requirements, which include the ease of use and the need for fewer steps to complete an action. While we did not measure the effects of accessibility provisions on usage, a value of this study is to provide a useful analytical tool for future research of this relationship. Indeed, an intriguing question would be the role (or lack of a role) of coaches on surmounting usability challenges and encouraging use even in apps that fall short of adequate accessibility provisions.

Another potential contributor was patient fatigue, both in using the app and in participating in the general study. This was a unique challenge in the sense that the effect of this fatigue could more directly affect our outcome regardless of proficiency or self-efficacy. We sought to minimize this issue using randomization of participants into coaching groups, and also gradually reduce and taper the interactions with coaches still ensuring availability of assistance if needed. Future studies may further illuminate this effect using questionnaires at throughout and at the end of the study to assess willingness to perform key basic tenets of the study and correlating them with attrition and proficiency rats. On the part of this study, we noted a non-usage effect only after a minimum number of events (9) was met in 24 hours. This minimum represented the first quartile of events by users in our cohort, since we did not find any reliable usage thresholds in the literature. While this measure may still provide a technical demand for some patients who may be less comfortable or adept at using mobile devices, we believed it represented an adequate minimum usage for the designed purpose of the H360x app. However, it may be useful in the future to further explore these minimum thresholds and correlate them with app usage and perceptions, especially in populations susceptible to these effects.

Finally, the perceived utility of an app is a key feature in a patient’s usage of the app. While part the role of the coach was to foster this perceived utility, we recognize that this effect is strongly individual and may be difficult to assess given the varying degrees of digital literacy and efficacy. Interestingly, we noted an increase in attrition in individuals with a relatively higher education. It is conceivable that a higher education level would be accompanied by an increased willingness and ability to engage the app [20]. Conversely, the effect observed in this study may be due to a perceived lack of utility by patients who may feel less of a need to rely on this app as an adjunct tool in their healthcare. As with patient fatigue/willingness to use the app, periodic targeted questionnaires throughout the study may help to shed light on these effects and correlate them with usage attrition.

Limitations

Several limitations of this study are worth considering. First, the technology infrastructure underlying Health360x went through some significant changes during the intervention. While the features and functionality of the platform remained the same, the backend was revamped which led to participants facing some technical issues during development (e.g., unable to sign-in to profiles) and differences in how usage data was defined and collected. The latter issue around usage data limited our measurement of user engagement with Health360x to variables that captured when and how long participants used Health360x. Second, it is important to note the quality of coaches varies and as such may have a differential effect on nonusage attrition. While we tried to emphasize uniform training for coaches, it is likely that their personal and cultural experiences influence their interactions with participants. Our results are also likely sensitive to the threshold of 9 events that are relevant to the primary function of Health360x and a 24 hour time period. More work is needed to create a systematic approach for understanding and modeling model dose-response in app based digital health interventions. Another limitation is that data on digital health literacy was not collected from participants, which may indicate omitted variable bias.

Conclusion

Non-usage attrition is an ongoing concern because it can significantly impair the efficacy of behavioral intervention technologies (BITs) irrespective of its design benefits. Key contributors to nonusage attrition can be broadly described in terms of individual (user) characteristics and intervention factors [21, 25]. The primary goal of this study was to characterize determinants of usage attrition in a behavioral intervention technology, Health 360x, within a population of Black patients with poor cardiovascular health. Using individual level characteristics like education and other socioeconomic factors, we found that several demographic factors influence a participant’s risk of lapsing into nonusage attrition across several weeks, including education-level. A possible explanation that we will assess in future studies is that who receive more education may face a greater psychological burden of being the ones who made it and with that an added responsibility to provide. These observations illustrate a need for a nuanced understanding of how technology-enabled interventions disparately impact underserved communities and can lead to greater disparities.

We did not find evidence that gender, age, baseline Life’s Simple 7, marital status, income and unemployment influenced nonusage attrition. Other studies have found mixed evidence of how these factors influence nonusage attrition. Analysis of nonusage attrition in a randomized control trial of a web-based smoking cessation intervention found that relatively young, highly educated, and unmarried participants are at greater risk of nonusage attrition compared to older participants from high socio-economic backgrounds as well as older female participants from low SES backgrounds [22]. In an observational study of a physical activity program in Australia, researchers found a lower mean time to attrition among participants who were men and inconsistent results with respect to age [23, 24]. Comparing outcomes in a 12 week vs 52 week web-based weight loss intervention, Neve et al. found that age is associated with nonusage attrition in the short tem, but it’s influence dissipates over time [25]. We could find no studies that explore the impact of race and ethnicty on nonusage attrition.

Additionally, we tested the potential benefits of having an adjunct health coach to determine if this multi-faceted intervention was associated with nonusage attrition. Previous research has shown the positive impact of having a coach on short-term behavior modification [26]. The findings of this study suggest that the observed long-term benefits of a behavior-change technology with a coach may not be as closely related to constant BIT use, but rather on the coach-patient relationship and the participants self-efficacy for self-management [8, 27]. Indeed, it is possible that the perceived benefits of having a coach may reduce a patient’s diligent use of the BIT, as the coach assumes a more direct role in influencing behavioral change. Furthermore, it is important to consider individual differences in motivation style, as some patients who prefer a less ‘hands-on’ approach may find a coach-based intervention cumbersome. Further work should be done to create new measures to capture the multidimensionality of mHealth BIT usage, explore efficacy of BITs as a function of usage, and investigate the impact of social determinants on nonusage attrition, especially in the presence of adjunctive interventions like health coaches and peer navigators that we have shown affect usage. Future studies will explore interactions between treatment group, demographic variables, and social determinants.

Acknowledgments

We would like to acknowledge Elizabeth Olorundare, Jan Morgan Billingslea, Kayellen Umeakunne, Pamela Cooper, Brenda Lank-ford, and all participants and volunteer research assistants for the technical assistance through the study.

References

  1. 1. National Vital Statistics Report; 2021. Deaths: Final Data for 2018; [modified 2021 Jan 12, cited 2022 May 2]. Available from: https://www.cdc.gov/nchs/data/nvsr/nvsr69/nvsr69-13-508.pdf
  2. 2. Kamel H, Zhang C, Kleindorfer DO and others. Association of Black Race With Early Recurrence After Minor Ischemic Stroke or Transient Ischemic Attack: Secondary Analysis of the POINT Randomized Clinical Trial. JAMA Neurology. 2020 May;77(5):601–605. pmid:32091536
  3. 3. Virani SS, Alonso A, Aparicio HJ, Benjamin EJ, Bittencourt MS, Callaway CW, et al. Heart Disease and Stroke Statistics-2021 Update: A Report From the American Heart Association. Circulation. 2021;143(8):254–743. pmid:33501848
  4. 4. Suaya JA, Shepard DS, Normand SL, Ades PA, Prottas J, Stason WB. Use of Cardiac Rehabilitation by Medicare Beneficiaries After Myocardial Infarction or Coronary Bypass Surgery. Circulation. 2007;116(15):1653–1662. pmid:17893274
  5. 5. Washington-Plaskett T, Idris MY, Mubasher M, Ko YA, Islam SJ, Dunbar S, et al. Impact of Technology-Based Intervention for Improving Self-Management Behaviors in Black Adults with Poor Cardiovascular Health: A Randomized Control Trial. Int. J. Environ. Res. Public Health [Internet]. 2021 April [cited 2022 Jan 8];18(7):3660. Available from: https://www.mdpi.com/1660-4601/18/7/3660 pmid:33915812
  6. 6. Keyserling TC, Samuel-Hodge CD, Pitts SJ, Garcia BA, Johnston LF, Gizlice Z et al. A community-based lifestyle and weight loss intervention promoting a Mediterranean-style diet pattern evaluated in the stroke belt of North Carolina: the Heart Healthy Lenoir Project. BMC public health. BMC public health. 2005;16(1):1–22.
  7. 7. Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C et al. Evaluating Digital Health Interventions: Key Questions and Approaches. Am J Prev Med. 2016 Nov;51(5):843–851 pmid:27745684
  8. 8. Pemu PE, Quarshie AQ, Josiah-Willock R, Ojutalayo FO, Alema-Mensah E, Ofili EO. Socio-demographic psychosocial and clinical characteristics of participants in e-HealthyStrides©: an interactive ehealth program to improve diabetes self-management skills. J Health Care Poor Underserved. 2011;22(4 Suppl):146–64 pmid:22102311
  9. 9. Huh J, Koola J, Contreras A, Castillo AK, Ruiz M, Tedone KG, et al. Consumer Health Informatics Adoption among Underserved Populations: Thinking beyond the Digital Divide. Yearb Med Inform. 2018 Aug;27(1):146–155 pmid:30157518
  10. 10. Murray E, White IR, Varagunam M, Godfrey C, Khadjesari Z, McCambridge J. Attrition revisited: adherence and retention in a web-based alcohol trial. J Med Internet Res. 2013 Aug 30;15(8):e162 pmid:23996958
  11. 11. Guertler D, Vandelanotte C, Kirwan M, Duncan MJ. Engagement and Nonusage Attrition With a Free Physical Activity Promotion Program: The Case of 10,000 Steps Australia. J Med Internet Res. 2015;17(7):e176 pmid:26180040
  12. 12. Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e402. pmid:15829473
  13. 13. Islam SJ, Kim JH, Topel M, Liu C, Ko YA, Mujahid MS, et al. Cardiovascular risk and resilience among Black adults: Rationale and design of the MECA Study. Journal of the American Heart Association. 2020;9(9):e015247. pmid:32340530
  14. 14. American Heart Association; 2021. Life’s Essential 8; [modified 2021 Jul 10, cited 2022 June 2]. Available from: https://www.heart.org/en/healthy-living/healthy-lifestyle/my-life-check-lifes-simple-7
  15. 15. Ford ES, Greenlund KJ, Hong Y. Ideal cardiovascular health and mortality from all causes and diseases of the circulatory system among adults in the United States. Circulation. 2012;125(8):987–995. pmid:22291126
  16. 16. Folsom AR, Shah AM, Lutsey PL and others. American Heart Association’s Life’s Simple 7: avoiding heart failure and preserving cardiac structure and function. The American Journal of Medicine. 2015;128(9):970–976 pmid:25908393
  17. 17. Kim JH, Lewis TT, Topel ML, Mubasher M, Li C, Vaccarino V, et al. Identification of resilient and at-risk neighborhoods for cardiovascular disease among black residents: the Morehouse-Emory Cardiovascular (MECA) Center for Health Equity Study. Preventing Chronic Disease. 2019;16:E57 pmid:31074715
  18. 18. Cox DR. Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological). 1972;34(2):187–202
  19. 19. Amorim LD, Cai J. Modelling recurrent events: a tutorial for analysis in epidemiology. International Journal of Epidemiology. 2015;44(1):324–333. pmid:25501468
  20. 20. Mahajan S, Lu Y, Spatz ES, Nasir K, Krumholz HM. Trends and predictors of use of digital health technology in the United States. The American Journal of Medicine. 2021;134(1):129–134. pmid:32717188
  21. 21. Murray JM, French DP, Patterson CC, Kee F, Gough A, Tang J, et al. Predicting outcomes from engagement with specific components of an internet-based physical activity intervention with financial incentives: process analysis of a cluster randomized controlled trial. J Med Internet Res. 2019;21(4):e11394. pmid:31002304
  22. 22. Wen S, Wiers RW, Boffo M, Grasman RPPP, Pronk T, Larsen H. Subtypes of smokers in a randomized controlled trial of a web-based smoking cessation program and their role in predicting intervention non-usage attrition: Implications for the development of tailored interventions. Internet Interventions. 2021;26:100473. pmid:34765460
  23. 23. Guertler D, Vandelanotte C, Kirwan M, Duncan MJ. Engagement and nonusage attrition with a free physical activity promotion program: the case of 10,000 steps Australia. Journal of medical Internet research. 2015;17(7):e4339 pmid:26180040
  24. 24. Rayward AT, Vandelanotte C, Van Itallie A, Duncan MJ. The Association Between Logging Steps Using a Website, App, or Fitbit and Engaging With the 10,000 Steps Physical Activity Program: Observational Study. J Med Internet Res. 2021;23(6):e22151. pmid:34142966
  25. 25. Neve MJ, Collins CE, Morgan PJ. Dropout, nonusage attrition, and pretreatment predictors of nonusage attrition in a commercial Web-based weight loss program. J Med Internet Res. 2010;12(4):e1640.
  26. 26. An S, Song R. Effects of health coaching on behavioral modification amongadults with cardiovascular risk factors: Systematic review and meta-analysis. Patient Education and Counseling. 2020;103(10):2029–2038. pmid:32448627
  27. 27. Thom DH, Wolf J, Gardner H, DeVore D, Lin M, Ma A, et al. A Qualitative Study of How Health Coaches Support Patients in Making Health-Related Decisions and Behavioral Changes. The Annals of Family Medicine. 2016;14(6):509–516. pmid:28376437