https://orcid.org/0000-0003-4603-2289
Figures
Abstract
Background
Health applications have grown exponentially in the last decade and are well accepted by users and health services. It is essential that digital tools meet the needs of the target users and are reliable, hence the importance of evaluating them. The aim of this study was to adapt The App Behavior Change Scale (ABACUS) into the Spanish language and to assess their metric properties.
Methods
This was an study undertaken in two phases: (1) translation and cultural adaptation process; (2) psychometric evaluation. 65 physical activity apps were included in the study (9 running Apple, 50 running Android and 6 running both). Cross-cultural adaptation, translation, and metric evaluation was made. The internal consistency and test-retest reliability of the Spanish version of the ABACUS was evaluated using physical activity apps. Two reviewers independently rated the apps using the translated scale. The apps were then rated again 2 weeks later by the same reviewers to measure test-retest reliability.
Results
No major differences were observed between both versions of ABACUS (original and Spanish). Discrimination indices (item-scale correlation) obtained appropriate results for both raters. ABACUS-Es presented an excellent interrater reliability (0.82, 95% CI 0.80–0.84) and excellent internal consistency (Cronbach alpha = 0.82).
Conclusions
The Spanish version of ABACUS, ABACUS-Es, is an instrument with adequate metric properties to measure the potential behavior change of apps from health professionals and app developers. ABACUS will help guide future research efforts in the development and evaluation of apps to promote health behaviors, and provides guidance for app developers to include adequate behavior change techniques in apps in Spanish to be used to promote health behaviors.
Citation: Martin-Payo R, González-Nuevo-Vázquez C, Álvarez-Gómez E, Surendran S, Cachero-Rodríguez J, Fernandez-Alvarez MdM (2024) Spanish versión of The App Behavior Change Scale (ABACUS-Es): Adaptation and validation study. PLoS ONE 19(12): e0314753. https://doi.org/10.1371/journal.pone.0314753
Editor: Tomasz Cudejko, Northumbria University, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
Received: January 4, 2024; Accepted: November 17, 2024; Published: December 5, 2024
Copyright: © 2024 Martin-Payo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data is available at: Martin-Payo, R., Gonzalez-Nuevo Vazquez, C., Alvarez-Gomez, E., Surendran, S., Cachero-Rodriguez, J., & Fernandez-Alvarez, M. del M. (2024). Data for Spanish versión of The App Behavior Change Scale (ABACUS-Es) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.13132998.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
EHealth is conceptualized as “the use of internet technology by the public, health workers, and others to access health and lifestyle information, services and support” [1]. As part of eHealth, it is important to consider the use of mobile health (mHealth), including mobile apps and other wearable devices for health monitoring [2]. Health apps have exponentially grown in the last decade and are well accepted by users and healthcare services [3, 4]. Digital interventions based on app usage have generally showed to be effective in terms of costs and health outcomes [5, 6]. A high percentage of these apps are offered in digital markets with limited consideration of relevant aspects, such as its content or quality aspects [7, 8]. Consequently, it is important to highlight that an inadequate app design could cause harm to users [9].
What has been described above suggests the formulation of the following question: Can any app related to e-health be recommended to any person? The answer seems to be clear “no”. Not all apps can be recommended for any patients, as there must be consideration of their personal characteristics [10–12].
As suggested in the framework designed by Norgaard et al. [13], it is essential that digital tools cater the needs of the target users. For example, if a health professional wants to recommend apps to promote a healthy diet during pregnancy, it seems reasonable to recommend apps specifically designed to this aim [14], rather than those designed for adults in general [15]. To illustrate this, mention can be made of the results of the study carried out by Muñoz-Mancisidor et al. [12] regarding the total apps found in digital markets for recommendation during pregnancy. Only 5.6% of them could be recommended on the basis of their content, quality and strategies for behavior change. This undoubtedly poses a challenge for healthcare professionals, who must become familiarized with evaluations of this kind, in order to be able to recommend the app best suited for their patients.
Additionally, other app characteristics need to be considered, including its quality [8]. There are some tools specifically designed to assess the quality of the apps. A widely used tool for health app quality evaluation from the health professional and user perspective, are the Mobile Application Rating Scale and the User Mobile Application Rating Scale respectively [16, 17]. Both was initially designed in English and subsequently adapted into Spanish characteristics [18, 19].
Finally, the inclusion of behavior change techniques is crucial in apps designed to promote healthy behaviors. Some authors suggest that it is essential to use a combination of behavior change techniques and behavior change theory to increase intervention effectiveness [20]. In a similar way, other authors who conclude that the potential of apps to promote health-related behaviors could be enhanced by including behavior change techniques in their design [21]. As an example, and taking cardiovascular disease as a reference, representing the leading cause of mortality worldwide [22], it is well known that adopting healthy behaviors could contribute to reduce the risk of such disease in the general population [23], for example by using apps [24].
Two behavior change taxonomies can be highlighted. The Behavior change technique Taxonomy v1, developed by Michie et al. [25], is a consensus-based taxonomy of 93 behavior change techniques for reporting intervention content. ABACUS, developed by McKay et al. [26], is a 21-item scale that can be used to determine the behavior change potential specifically of apps, showing adequate internal coherence (Cronbach alpha = 0.93). This tool has been used to evaluate apps related to different aspects of health, including neurological disorders [27], vaping cessation [28], low back pain [29], well-being [21], physical activity [30], and physiotherapy care [31]. Likewise, scientific literature has been found that contemplates its use to evaluate apps in languages other than English, such as Dutch [32] or Russian [33]. However, no adaptations of ABACUS to these languages have been identified, and the authors have used the original version, which suggests adequate proficiency in both languages.
Adapting the tools to the native language of the evaluators widens the margins for use, in the sense that they do not need to speak the language of the tool (English in the case of ABACUS), and reduces the probability of information bias derived from imprecise measurements.
Since there was no found a taxonomy like those in Spanish, we developed this study with the aim to adapt ABACUS into the Spanish language and to assess their metric properties. The present study attempts to offer ABACUS in Spanish, to be used by developers, researchers and healthcare professionals in the development or recommendation of app for health purposes. This will contribute to reliably evaluate the behavior change strategies included in the apps as a criterion recommended in the scientific literature for indicating the effectiveness of behavior change. On the other hand, it eliminates the barrier related to the need to speak English for adequate use of ABACUS by developers, researchers and healthcare professionals.
Methods
Ethics
Ethical approval was obtained from the Ethics Committee of the Principado de Asturias (reference 2021.569).
Participants (Unit of analysis: Apps)
Apps available in the “Health and fitness” category on iTunes, Google Play, or both stores in the period between June to September 2022 were included. The following inclusion criteria were applied: i) apps targeting adults, ii) focusing on physical activity or exercise, iii) free to use, iv) available in Spanish. The exclusion criteria were the following: i) focusing on non-physical activity-based interventions; ii) requiring a wearable device to be used.
The assessment process consisted of four phases: i) app identification using the search term “physical activity”; ii) initial screening based on the apps marketing information; iii) downloading apps to an Apple device (iPad pro), an Android device (Samsung Galaxy A53); iv) removing duplicates from the Apple and/or Android device based on the criteria of the reviewer; and v) application of the inclusion and exclusion criteria commented above. Two reviewers independently assessed the behavior change techniques included in the apps using ABACUS.
According to Zou [34], a minimum sample size of 41 is required to determine whether the true inter-rater reliability lies within 0.15 of a sample observation of 0.80, with 87% assurance.
Instruments
The original version of ABACUS was cross-culturally adapted for the Spanish language. Consequently, a cross-cultural adaptation, translation, and measure of metric properties were developed. The corresponding STROBE checklist was used in this study.
The ABACUS scale [26] was designed to measure the potential behavior change of apps. It consists of 21 items grouped into 4 categories: knowledge and information (5 items), goals and planning (3 items), feedback and monitoring (7 items), and actions (6 items). Each item represents a behavior change technique, and its inclusion in the app is rated dichotomously (behavior change technique not included = 0 or included = 1).
Procedure
The adaptation and translation processes were developed using the method proposed by Ramada-Rodilla et al. [35] Initially, a translation into Spanish was independently made by two native Spanish speakers proficient in the English language. Subsequently, both created a consensus version after assessing both translations together. An expertise panel composed by both translators, a nurse expert in behavioral change interventions, an expert in methodology and expert in Spanish language. This version was back translated into English and the original author checked the semantic equivalence [21]. Finally, the metric properties were assessed: reliability, internal consistency and test-retest reliability of the scale.
Data analysis
The ABACUS-Es was used by two raters to review 65 physical activity apps.
Reliability of the scale was assessed using the Cohen kappa coefficient. Consistent with previous research [36], a kappa between 0.01 and 0.20 indicates slight agreement, between 0.21 and 0.40 fair, between 0.41 and 0.60 moderate, between 0.61 and 0.80 substantial and between 0.81 and 1.00 almost perfect. However, a negative kappa indicates less agreement than that would be expected by chance and suggests that there may have been inconsistencies in how measures were applied.
The internal consistency of the scale was calculated using Cronbach alpha. Interrater reliability was determined by interclass coefficient (ICC). Percentage agreement was also calculated.
The test-retest reliability of the scores assigned by the raters with the 15 apps was obtained by calculating the Pearson correlations between the two points in time.
All analyses were conducted using SPSS for Windows, version 27.0 (SPSS Inc).
Results
467 apps were identified. 345 were eliminated, because either their final objective was not physical activity, they included aspects related to meditation or mindfulness, they did not consider physical activity as a whole but instead focused on specific muscle groups, they involved economic transactions, or they required the use of an additional or wearable device.
Subsequently, apps were downloaded for screening and a second review was carried out. After downloading the apps, a further 57 apps were eliminated because: i) paid apps (40.4%); ii) not available in Spanish (19.3%); iii) non focusing on physical activity (22.8%); iv) requiring a wearable (5.3%); v) malfunctioning or not working (12.3%) (Fig 1). 65 apps were included in the study (9 running Apple, 50 running Android and 6 running both).
Cross-cultural adaptation and translation process
Following the methodology described in the “cross-cultural adaptation and translation process” section. No major differences were observed between the two independent translations, and no major modifications of the specific terminology were required. Therefore, the final Spanish version of ABACUS was created upon reaching consensus on any discrepancies; both the conceptual analysis and the translation were considered relevant and appropriate for the Spanish culture. Additionally, the original author approved the version after checking the semantic equivalence.
Metric properties assessment
The average score of ABACUS-Es was 8.8 (SD = 2.99), which ranged from 2 to 17. Reliability analysis testing was conducted with the 65 physical activity apps downloaded from the app store. All apps were rated independently by two reviewers. It was found that over 70% of the questions had high percentage agreement among reviewers (over 80%), with the scale overall reporting excellent interrater reliability (2-way mixed ICC = 0.82, 95% CI 0.80–0.84) and excellent internal consistency (Cronbach alpha = 0.82).
Although four questions returned a negative kappa, most of the items returned an agreement above moderate (Table 1).
Discussion
This study confirms the metric properties of the Spanish version of ABACUS (ABACUS-Es), a new instrument in Spanish language that measures the potential behavior change of apps. The conceptual and semantic equivalences suggests that the Spanish version is reliable in fulfilling the same purpose of the original scale and that it is functionally equivalent to the original version [26].
Similar metric properties were obtained by the present study as those presented in the English version of the scale [26]. Both instruments have excellent percentage agreement among reviewers (>80%), inter-rater reliability (0.92) and internal consistency. ABACUS-Es presented a slightly lower internal consistency (0.82) than the English version (0.92) [26]. Lower agreement among reviewers were found in category 3. Since there is no hypothesis to explain differences, further studies should be developed to explain the observed data.
The average score was 8.8 (SD = 2.99), which ranged from 2 to 17. This result is consistent with previous research where similar scores were found. Previous research that includes apps focused on smoking, alcohol use, physical activity, nutrition, and mental well-being scores for ABACUS ranged from 1 to 17, out of 21, with an average score of 7.8 (SD 2.8) [37]. In a research focusing on health apps for patients with a chronic condition or multimorbidity the average score was 8.07 (SD 2.30) [37].
ABACUS has been used to evaluate the behavior change potential of digital tools [14, 38, 39] or as a theory-driven approach to design mHealth interventions [40, 41]. So, it is supposed that ABACUS-Es, in combination with the content and quality assessment, will be use in a similar way. For example, to guide Spanish health professionals about which app would be best to recommend to their patients. As previous research concludes, identifying behavior change techniques that are most effective in promoting and maintaining positive health behaviors is essential to consider in app development because they significantly improve health behaviors and outcomes [42]. Thus, based on this premise, identifying behavior change techniques in the developed apps, also could contribute to improved health behaviors.
Previous authors found mixed effectiveness in app-based interventions, for example, same behavior change techniques have different effects depending on personal or clinical characteristics of the targe groups [43] or the health topic addressed [44]. It suggests the need to identify and use adequate behavior change techniques for obtain the best outcomes [3, 45, 46]. The none existence of a specific tool to perform this identification, could be considered as a barrier. Thus, it is important to highlight that the Spanish version of ABACUS contributes to providing a standardized method of evaluating smartphone apps for behavior change.
The issue addressed arose from the knowledge that a considerable percentage of health apps offered in digital markets have limited consideration of relevant aspects, such as the behavior change techniques included. Therefore, an inadequately designed health app could be inefficient. In addition, the original version of ABACUS demonstrated excellent metric properties and effectiveness in previous studies. As a strength of our study, we highlight the practical implications for developers, clinical practice and researchers. ABACUS-Es provides guidance to apps developers, could be used by healthcare professionals to assess behavior change techniques in apps available in Spanish, and will help guide future research efforts in the development and evaluation of apps to promote health behaviors.
One possible limitation of the study is that the validation was based solely on physical activity apps. Future research could address other behaviors and/or populations to reinforce the results obtained in the present study. Likewise, if effectiveness is observed in interventions where apps are used, subjected to prior analysis of behavior change strategies via the use of ABACUS-Es, this could be regarded as a correlation between the theoretical perspective and the actual results of the intervention, thereby in turn reinforcing the usefulness of ABACUS-Es. Another limitation is that ABACUS-Es cannot be checked alongside another Spanish scale for validity purposes, as no similar scales exist yet. Finally, the metric properties assessed were for Castilian Spanish (Spain), which must be taken into consideration if the version is to be used with other Spanish-speaking populations with a different variety of Spanish, such as Latin American. All those limitations suggest the need of further research assessing the feasibility of ABACUS.
Conclusions
The results of this study suggest that ABACUS-Es is an instrument with adequate metric properties that can be used to identify behavior change techniques and to measure the potential behavior change of apps. The results of this study show great promise for scale adaptation.
This tool will help health professionals identify behavior change techniques included in apps. It will also consider the best apps based on an empirical evaluation that combines content with quality. Additionally, the tool will also allow developers to design apps.
References
- 1. Wyatt JC, Liu JL. Basic concepts in medical informatics. J Epidemiol Community Health 2002; 56(11):808–12. pmid:12388565
- 2.
Perego P. Devices for mHealth. In: Andreoni G, Perego P, Frumento E, eds. mHealth Current and Future Applications. Switzerland: Springer Nature 2019: 87–99.
- 3. Aguiar M, Trujillo M, Chaves D, et al. mHealth Apps Using Behavior Change Techniques to Self-report Data: Systematic Review. JMIR Mhealth Uhealth 2022; 10(9):e33247. https://doi.org/10.2196/33247.
- 4.
Sriram R. Top 10 mHealth Apps That Are Revolutionizing Healthcare [Internet]. https://www.kolabtree.com/blog/top-10-mhealth-apps-that-are-revolutionizing-healthcare/. Accessed December 21, 2022.
- 5. Chandran VP, Balakrishnan A, Rashid M, et al. Mobile applications in medical education: A systematic review and meta-analysis. PloS One 2022; 17(3):e0265927. pmid:35324994
- 6. Gentili A, Failla G, Melnyk A, et al. The cost-effectiveness of digital health interventions: A systematic review of the literature. Front Public Health 2022; 10:787135. pmid:36033812
- 7. Boulos MN, Brewer AC, Karimkhani C, et al. Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J Public Health Inform 2014; 5(3):229. pmid:24683442
- 8. Grainger R, Devan H, Sangelaji B, et al. Issues in reporting of systematic review methods in health app-focused reviews: A scoping review. Health Informatics J 2020; 26(4):2930–45. pmid:32914696
- 9. Parker L, Karliychuk T, Gillies D, et al. A health app developer’s guide to law and policy: a multi-sector policy analysis. BMC Med Inform Decis Mak 2017; 17(1):141. pmid:28969704
- 10. Martín Payo R, Harris J, Armes J. Prescribing fitness apps for people with cancer: a preliminary assessment of content and quality of commercially available apps. J Cancer Surviv 2019a; 13(3):397–405. pmid:31030308
- 11. Martín-Payo R, Ferreras-Losilla L, González-Méndez X, et al. Apps for individuals diagnosed with breast cancer: a preliminary assessment of the content and quality of commercially available apps in Spanish. Mhealth 2021; 7:2. pmid:33634185
- 12. Muñoz-Mancisidor A, Martin-Payo R, Gonzalez-Mendez X, et al. Content, Behavior Change Techniques, and Quality of Pregnancy Apps in Spain: Systematic Search on App Stores. JMIR Mhealth Uhealth 2021; 9(11):e27995. pmid:34787587
- 13. Norgaard O, Furstrand D, Klokker L, et al. The e-health literacy framework: A conceptual framework for characterizing e-health users and their interaction with e-health systems. Knowl Manag E-Learn Int J KMEL 2015; 7(4): 522–40.
- 14. Faessen JPM, Lucassen DA, Buso MEC, et al. Eating for 2: A Systematic Review of Dutch App Stores for Apps Promoting a Healthy Diet during Pregnancy. Curr Dev Nutr 2022; 6(6):nzac087. pmid:35711572
- 15. Scarry A, Rice J, O’Connor EM, et al. Usage of Mobile Applications or Mobile Health Technology to Improve Diet Quality in Adults. Nutrients 2022; 14(12):2437. pmid:35745167
- 16. Stoyanov SR, Hides L, Kavanagh DJ, et al. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015; 3(1):e27. pmid:25760773
- 17. Stoyanov SR, Hides L, Kavanagh DJ, et al. Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016; 4(2):e72. pmid:27287964
- 18. Martin Payo R, Fernandez Álvarez MM, Blanco Díaz M, et al. Spanish adaptation and validation of the Mobile Application Rating Scale questionnaire. Int J Med Inform 2019b; 129:95–9. pmid:31445295
- 19. Martin-Payo R, Carrasco-Santos S, Cuesta M, Stoyan S, et al. Spanish adaptation and validation of the User Version of the Mobile Application Rating Scale (uMARS). J Am Med Inform Assoc 2021b; 28(12):2681–6. pmid:34613400
- 20. Bohlen LC, Michie S, de Bruin M, et al. Do Combinations of Behavior Change Techniques That Occur Frequently in Interventions Reflect Underlying Theory? Ann Behav Med 2020; 54(11):827–42. pmid:32959875
- 21. McKay FH, Wright A, Shill J, et al. Using Health and Well-Being Apps for Behavior Change: A Systematic Search and Rating of Apps. JMIR Mhealth Uhealth 2019a; 7(7):e11926. pmid:31274112
- 22. Roth GA, Mensah GA, Johnson CO, Addolorato G, Ammirati E, Baddour LM et al. GBD-NHLBI-JACC Global Burden of Cardiovascular Diseases Writing Group. Global Burden of Cardiovascular Diseases and Risk Factors, 1990–2019: Update From the GBD 2019 Study. J Am Coll Cardiol. 2020;76(25):2982–3021. pmid:33309175
- 23. Wu J, Feng Y, Zhao Y, Guo Z, Liu R, Zeng X, et al. Lifestyle behaviors and risk of cardiovascular disease and prognosis among individuals with cardiovascular disease: a systematic review and meta-analysis of 71 prospective cohort studies. Int J Behav Nutr Phys Act. 2024;21(1):42. pmid:38650004
- 24. Salas-Groves E, Galyean S, Alcorn M, Childress A. Behavior Change Effectiveness Using Nutrition Apps in People With Chronic Diseases: Scoping Review. JMIR Mhealth Uhealth. 2023;11:e41235. pmid:36637888
- 25. Michie S, Richardson M, Johnston M, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013; 46(1):81–95. pmid:23512568
- 26. McKay FH, Slykerman S, Dunn M. The App Behavior Change Scale: Creation of a Scale to Assess the Potential of Apps to Promote Behavior Change. JMIR Mhealth Uhealth 2019b; 7(1):e11130. pmid:30681967
- 27. Rendell R, Pinheiro M, Wang B, McKay F, Ewen A, Carnegie C, et al. Digital Apps to Improve Mobility in Adults with Neurological Conditions: A Health App-Focused Systematic Review. Healthcare (Basel). 2024;12(9):929. pmid:38727486
- 28. McKay F, Chan L, Cerio R, Rickards S, Hastings P, Reakes K, et al. Assessing the Quality and Behavior Change Potential of Vaping Cessation Apps: Systematic Search and Assessment. JMIR Mhealth Uhealth. 2024;12:e55177. pmid:38532616
- 29. Didyk C, Lewis LK, Lange B. Availability, content and quality of commercially available smartphone applications for the self-management of low back pain: a systematic assessment. Disabil Rehabil. 2022;44(24):7600–7609. pmid:34612133
- 30. Truelove S, Vanderloo LM, Tucker P, Di Sebastiano KM, Faulkner G. The use of the behaviour change wheel in the development of ParticipACTION’s physical activity app. Prev Med Rep. 2020;20:101224. pmid:33134041
- 31. Merolli M, Francis JJ, Vallance P, Bennell KL, Malliaras P, Hinman RS. Patient-Facing Mobile Apps to Support Physiotherapy Care: Protocol for a Systematic Review of Apps Within App Stores. JMIR Res Protoc. 2021;10(12):e29047. pmid:34889767
- 32. Faessen JPM, Lucassen DA, Buso MEC, Camps G, Feskens EJM, Brouwer-Brolsma EM. Eating for 2: A Systematic Review of Dutch App Stores for Apps Promoting a Healthy Diet during Pregnancy. Curr Dev Nutr. 2022;6(6):nzac087. pmid:35711572
- 33. Bunova A, Wiemker V, Gornyi B, Ferreira-Borges C, Neufeld M. Russian-Language Mobile Apps for Reducing Alcohol Use: Systematic Search and Evaluation. JMIR Mhealth Uhealth. 2022;10(1):e31058. pmid:35006083
- 34. Zou GY. Sample size formulas for estimating intraclass correlation coefficients with precision and assurance. Stat Med. 2012;31(29):3972–81. pmid:22764084
- 35. Ramada-Rodilla JM, Serra-Pujadas C, Delclós-Clanchet GL. Cross-cultural adaptation and health questionnaires validation: revision and methodological recommendations. Salud Publica Mex 2013; 55(1):57–66. https://doi.org/10.1590/s0036-36342013000100009.
- 36. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics 1977; 33(1): 159–74. https://doi.org/10.2307/2529310. pmid:843571
- 37. Bricca A, Pellegrini A, Zangger G, et al. The Quality of Health Apps and Their Potential to Promote Behavior Change in Patients With a Chronic Condition or Multimorbidity: Systematic Search in App Store and Google Play. JMIR Mhealth Uhealth 2022; 10(2):e33168. pmid:35119367
- 38. Brammall BR, Garad RM, Boyle JA, et al. Assessing the Content and Quality of Digital Tools for Managing Gestational Weight Gain: Systematic Search and Evaluation. J Med Internet Res 2022; 24(11):e37552. pmid:36427237
- 39. Hinman RS, Nelligan RK, Campbell PK, et al. Exercise adherence Mobile app for Knee Osteoarthritis: protocol for the MappKO andomized controlled trial. BMC Musculoskelet Disord 2022; 23(1):874. https://doi.org/10.1186/s12891-022-05816-6.
- 40. Chen Y, Ji M, Wu Y, et al. Individualized mobile health interventions for cardiovascular event prevention in patients with coronary heart disease: study protocol for the iCARE randomized controlled trial. BMC Cardiovasc Disord 2021; 21(1):340. pmid:34256698
- 41. Voorheis P, Zhao A, Kuluski K, et al. Integrating Behavioral Science and Design Thinking to Develop Mobile Health Interventions: Systematic Scoping Review. JMIR Mhealth Uhealth 2022; 10(3):e35799. https://doi.org/10.2196/35799.
- 42. Michie S, West R, Sheals K, et al. Evaluating the effectiveness of behavior change techniques in health-related behavior: a scoping review of methods used. Transl Behav Med 2018; 8(2):212–24. pmid:29381786
- 43. Wang Y, Min J, Khuri J, et al. Effectiveness of Mobile Health Interventions on Diabetes and Obesity Treatment and Management: Systematic Review of Systematic Reviews. JMIR Mhealth Uhealth 2020; 8(4):e15400. pmid:32343253
- 44. Arroyo AC, Zawadzki MJ. The Implementation of Behavior Change Techniques in mHealth Apps for Sleep: Systematic Review. JMIR Mhealth Uhealth 2022; 10(4):e33527. pmid:35377327
- 45. Bearne LM, Sekhon M, Grainger R, et al. Smartphone Apps Targeting Physical Activity in People With Rheumatoid Arthritis: Systematic Quality Appraisal and Content Analysis. JMIR Mhealth Uhealth 2020; 8(7):e18495. pmid:32706727
- 46. Patterson K, Davey R, Keegan R, et al. Behaviour change techniques in cardiovascular disease smartphone apps to improve physical activity and sedentary behaviour: Systematic review and meta-regression. Int J Behav Nutr Phys Act 2022; 19(1):81. pmid:35799263