https://orcid.org/0000-0003-3122-1749
Figures
Abstract
Mobile health applications, which employ wireless technology for healthcare, can aid behaviour change and subsequently improve health outcomes. Mobile health applications have been developed to increase physical activity, but are rarely grounded on behavioural theory and employ simple techniques for personalisation, which has been proven effective in promoting behaviour change. In this work, we propose a theoretically driven and personalised behavioural intervention delivered through an adaptive knowledge-based system. The behavioural system design is guided by the Behavioural Change Wheel and the Capability-Opportunity-Motivation behavioural model. The system exploits the ever-increasing availability of health data from wearable devices, point-of-care tests and consumer genetic tests to issue highly personalised physical activity and sedentary behaviour recommendations. To provide the personalised recommendations, the system firstly classifies the user into one of four diabetes clusters based on their cardiometabolic profile. Secondly, it recommends activity levels based on their genotype and past activity history, and finally, it presents the user with their current risk of developing cardiovascular disease. In addition, leptin, a hormone involved in metabolism, is included as a feedback biosignal to personalise the recommendations further. As a case study, we designed and demonstrated the system on people with type 2 diabetes, since it is a chronic condition often managed through lifestyle changes, such as physical activity increase and sedentary behaviour reduction. We trained and simulated the system using data from diabetic participants of the UK Biobank, a large-scale clinical database, and demonstrate that the system could help increase activity over time. These results warrant a real-life implementation of the system, which we aim to evaluate through human intervention.
Author summary
Mobile health applications employ wireless technology for healthcare to aid behaviour change and improve health. Mobile health applications have been developed to increase physical activity, but they present issues such as not using behavioural theory and not being personalised to the patient. In this work, we propose a mobile health intervention to increase physical activity and sedentary behaviour in people with type 2 diabetes, since it is a chronic condition often managed through lifestyle changes. The intervention is based on behavioural theory and consists of a recommendation system that is personalised to the patient’s characteristics and adapts to their skills. The system uses data from activity trackers, hormonal levels and genetic variants to issue highly personalised and effective recommendations. To prove the efficacy of the system, we simulated it using data from diabetic participants of the UK Biobank, a large-scale clinical database, and demonstrated that the system could help increase activity over time. These results warrant a real-life implementation of the system, which we aim to evaluate through human intervention.
Citation: Cavallo FR, Toumazou C (2023) Personalised lifestyle recommendations for type 2 diabetes: Design and simulation of a recommender system on UK Biobank Data. PLOS Digit Health 2(8): e0000333. https://doi.org/10.1371/journal.pdig.0000333
Editor: Nicole Yee-Key Li-Jessen, McGill University, CANADA
Received: December 15, 2022; Accepted: July 18, 2023; Published: August 30, 2023
Copyright: © 2023 Cavallo, Toumazou. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The datasets used for this study can be obtained from the UK Biobank with appropriate application (ukbiobank.ac.uk).
Funding: FRC was supported by the UK EPSRC Doctoral Training Partnership (Award Reference 2124497). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: CT is the co-founder of DnaNudge.
Introduction
mHealth (mobile health) technology is the use of wireless technology in medical care that can improve health outcomes due to its widespread appeal, accessibility and ability to reach large populations at a low-cost [1]. Many mHealth interventions have been developed to promote physical activity (PA), which mainly use wrist-worn or smartphone accelerometers and employ simple digital techniques. These mHealth interventions rarely utilise a holistic approach in attempting to positively alter one’s behaviour, such as grounding the intervention on behavioural theory and personalisation, which has been positively correlated with successful behaviour changes [2]. A systematic review and meta-regression [3] found that personalisation is effective in changing lifestyle behaviours and that the source of data used in the personalisation greatly affects the effectiveness of the intervention. Therefore, one of the main research gaps in behaviour change interventions is the effective use of personalisation, which can increase engagement and effectiveness of the interventions by improving one’s perceived skills and motivation [4]. According to the Self-Efficacy Theory [5], motivation is driven by self-efficacy, i.e. a person’s belief in their ability to succeed, which is developed through repeated successes to challenges that become more difficult as the skills improve. Consequently, personalisation of the intervention is critical to ensure that the subject builds high self-efficacy to succeed in the behaviour change process.
To achieve personalisation, interventions using artificial intelligence, specifically recommender or expert systems, have been developed. For example, Ni et al. [6] used a long short-term memory recurrent neural network (RNN) trained on user identity, sport type and historical workout sequences to recommend workout routes based on personal criteria (such as workout length and heart rate) and expectations. The model also predicts whether a user’s heart rate will exceed a user-set threshold if they continue at the current pace and suggest ways to hit the user’s target. Mahyari et al. [7] developed an intervention with RNN personalised such that the recommendations are highly likely to be performed based on past exercise history. Zhao et al. [8] developed and tested the efficacy of a tree-like personalised gamification intervention that recommends activity type, time and location based on the user’s general information, personality, and daily activity data from smartphone or wearable tracker. However, these works make limited use of personalisation compared to the levels that could be achieved given the ever-increasing availability of personal data. In recent years, and more specifically with the COVID-19 pandemic, the availability of point-of-care [9] and consumer genetic tests [10] offers the possibility of collecting even more health-related data to develop technology-driven interventions. For example, DnaNudge, a direct-to-consumer genetic service informed by behavioural science, developed an expert system that recommends food products with a suitable nutritional value depending on one’s genetic risk of developing metabolic conditions [11]. DnaNudge also encourages users to reduce sedentary time, with the penalty of having decreased food choices until a preset number of steps is achieved. However, while DnaNudge uses genetic information to recommend food choices, it does not use genotypes specific to physical activity, nor does it provide activity recommendations; rather, it lets the user set their preferred activity levels.
The knowledge-based system presented in the paper addresses the gaps in the current literature on recommender systems for PA. To demonstrate the concept, the system is developed and evaluated on a type 2 diabetes (T2D) population. T2D is a highly prevalent chronic condition linked to insulin resistance caused by obesity, a diet high in carbohydrates and inactivity, and is managed through lifestyle changes and drug therapy. Lifestyle management, including PA promotion, is the first-line therapy to improve glucose control in T2D [12]. In addition, with the recent understanding of the damages of sedentary behaviour (SB)—i.e., waking activities that require lying or sitting—people with T2D are now also recommended to reduce their sedentary time, as this is associated with increased glucose metabolism biomarkers [13]. To promote a healthier lifestyle effectively, behaviour-change strategies are recommended. In particular, technology-enabled interventions can be especially useful in increasing activity and reducing sedentary time, thus improving glucose control in T2D patients [14].
However, T2D is a highly heterogeneous disease, and a one-size-fits-all approach may not be optimal. As such, the first aim of this work is to identify phenotypically distinct clusters of T2D which we then use to personalise the system. Specifically, we identify four clusters based on common biomarkers of T2D. Several studies have explored the clustering of diabetic patients [15–21], but all have analysed the clusters in terms of disease risk and progression and not in relation to lifestyle factors. The clusters were determined with various methods (K-means [16, 18], hierarchical clustering [17, 19, 20], TwoStep clustering [15, 21]) and using features including blood chemistry, anthropometric data and measures of glucose metabolism. However, none of the studies in the literature considered the associations between clusters and lifestyle risk factors. Determining such associations is essential to develop personalised behavioural therapies for T2D patients to lower the risk of comorbidities related to a poor lifestyle. Therefore, the first aim of this work is to replicate the clusters in a new sample, the UK Biobank, quantify the differences in PA and SB between clusters, and determine their effects on the risk of developing CVD–a condition commonly associated with T2D.
The second aim of this work is to design and test a new knowledge-based system for personalised lifestyle change. Despite the increasing complexity and number of mHealth interventions available, these technologies fail to induce long-term changes [1]. Reviews have highlighted the need to use behavioural theories to design interventions in order to achieve higher efficacy [2]. Therefore, we designed the high-level structure of the system following the Capability-Opportunity-Motivation Behavioural (COM-B) model and the Behaviour Change Wheel (BCW) [22]. The BCW provides a framework to develop complex interventions that is comprehensive, coherent and linked to a behavioural model, the COM-B model. This framework has been used to develop digital interventions that are both accepted by users and effective for a variety of behaviours, including improving diet [23–25], increasing physical activity [23, 26, 27] and reducing sedentary behaviour [28], management of hypertension [29] and diabetes [30, 31], smoking cessation [32, 33], alcohol reduction [34] and medication adherence [35, 36] or discontinuation [37].
The present system (high-level diagram shown in Fig 1) enables the behaviour change process by using several layers of personalisation: firstly by classifying the user into one of four diabetes clusters, secondly by recommending activity levels based on their genotype and past activity history, and finally by presenting the user with their risk of developing cardiovascular disease (CVD) based on their current activity levels. Another layer of personalisation is achieved by including leptin as a feedback biosignal to inform on the efficacy of the intervention towards weight loss. In fact, leptin is a biomarker for obesity management therapies as leptin resistance is related to obesity and leptin levels vary with dieting regimens [38]. To our knowledge, no previous system or intervention has used blood chemistry as a feedback signal for behavioural change interventions. Biomarkers have previously been used to assess the efficacy of interventions, but never as an integral part of the recommendation systems. Additionally, the proposed system includes SB alongside PA, which works have overlooked, despite it being paramount to improving glucose control in T2D populations.
Figure created with BioRender.com.
Finally, we present the results of simulations of the proposed system on the data from the UK Biobank, a large-scale clinical database with biomedical data on more than 500,000 participants.
Materials and methods
Data source
The UK Biobank [39] is a database of over 500,000 adults aged 37–73 recruited in the UK between 2006 and 2010. At recruitment, participants completed a questionnaire and verbal interview around sociodemographics, family history, early life exposures, psychosocial factors, environmental factors, lifestyle, and health status. At the same baseline visit, they had physical measures taken and blood, urine and saliva samples collected. Physical activity and sedentary behaviour data were measured through accelerometer for 100,000 participants for 7 days between 2013 and 2015.
Ethics approval for the UK Biobank study was obtained from the North West Centre for Research Ethics Committee (REC reference: 21/NW/0157). Informed consent for the UK Biobank study was obtained from participants during the baseline assessment. The work described here was approved by the UK Biobank under application ID 56132. All analyses were performed in accordance with the relevant guidelines and regulations.
In this study, only participants with a self-reported T2D diagnosis, verified by a nurse, were selected and included in the analyses. Due to participants missing data, the sample sizes vary depending on the analysis (see S1 Table). The workflow of the analyses conducted on the UK Biobank is described in Fig 2.
The graphs do not depict real data, but are for visualisation only.
Cluster analysis
We selected 16 biomarkers of T2D: systolic blood pressure, diastolic blood pressure, pulse rate, age at diagnosis, waist circumference (WC), body mass index (BMI), apolipoprotein A, apolipoprotein B, C-reactive protein, cholesterol, HDL-cholesterol, LDL direct, glucose, HbA1c, lipoprotein A and triglycerides. Clustering was done using the markers rescaled to the [0, 1] range. The dataset was split based on gender and analysed separately to avoid sex-dependent differences [15]. After removing participants with missing values for one or more biomarkers, 14,149 participants (9,054 males and 5,095 females) were included in the cluster analysis.
Four clustering methods were tested with the scikit-learn [40] and scikit-learn-extra in Python: k-means, k-medoids, agglomerative hierarchical clustering and Gaussian mixture models (GMM). The hyperparameters were tuned based on the silhouette width [41] metric. Cluster stability was evaluated using the bootstrapped Jaccard coefficient in the R fpc package [42], which computes the similarity between clusters on the whole sample to clusters on the bootstrapped dataset (1000 resamples). A cluster with a Jaccard coefficient value ≥ 0.75 is considered stable [42]. The clustering methods were evaluated based on the silhouette, Calisnki-Harabasz and Davies-Boulding scores. Differences in cardiometabolic biomarkers between clusters were assessed with one-way ANOVA (effect sizes reported as Cohen’s d), and a Bonferroni correction was applied to the p-value to account for multiple tests.
Regression analysis.
To assess inter-cluster differences in associations between PA (or SB) and CVD, we used logistic regression with statsmodels [43]. A full list of illnesses classified as CVD, including hypertension and high cholesterol, is included in the supporting information (S2 Text).
After removing participants with missing accelerometer data, or without at least 3 days of data and with data in each one-hour period of the 24-hour cycle (scattered over multiple days), n = 1805 participants were included in the regression analysis.
Firstly, we classified the raw accelerometer data into SB, light PA, walking, moderate-to-vigorous PA (MVPA) and sleep with balanced random forests and Hidden Markov models (with the accelerometer package by [44]). Despite the participants being instructed to wear the accelerometer all the time, non-wear segments were present and were imputed using the average of similar time-of-day data points taken from different days of the week. Secondly, we partitioned the day into time spent in overall PA (including time spent in light PA, walking and MVPA), SB and sleep, and we included the accelerometer in the regression model with a compositional approach [45]. In a compositional framework, time spent in the three different activities is considered as a relative proportion of the overall time budget (24 hours), and thus conventional statistical methods cannot be employed with compositional data [46] due to the implied collinearity between the activities. Therefore, we used isometric log-ratio (ILR) transformation. The ILR regression coefficients were also used to estimate the minutes of activity required to reduce the log(odds) of CVD Details on the transformation can be found in the supporting information (S3 Text).
The effect of common confounders including age, WC, alcohol intake, income and smoking status as covariates was examined and the final model was selected with backwards selection—only covariates reaching significance (p-value < 0.05) were retained in the final models. Since only WC, smoking status and age were significant covariates, participants with complete data for these variables were included in the analysis (n = 1805). Sensitivity analysis was done on a subset of participants with no missing data for all covariates (including alcohol intake and income) with n = 1615.
Survival analysis.
We assessed the effects of PA and SB on CVD onset in the different clusters with the Kaplan-Meyer Estimate. In order to predict survival time from PA and SB levels simultaneously, activity data was not transformed with ILR transformation and used as minutes/day. The time between birth and onset of CVD was used as the time-to-event of interest, with the event being diagnosis of CVD. The observations were censored at 68 years, and subjects that had not reached this age at the baseline visit were removed, leaving a sample size of n = 1215.
Survival regression was further done to assess the effect of relevant risk factors on the survival time. Smoking, sex, age, triglycerides, diastolic blood pressure (DBP), total cholesterol, LDL-cholesterol and WC were included as covariates in the models, as they are the most common predictors for CVD [47]. After examination of the partial residual plots, a non-linear term () and two interaction terms (PA×age and PA × DBP) were included. All features were scaled to the [0, 1] range. Four models were compared with k-fold cross-validation (k=10, 100 iterations): Cox’s proportional hazards, Weibull AFT, Log-logistic and Log-normal AFT models. The analysis was done with the Python lifelines package [48].
The selected model is then used in the recommender system to predict the survival time for each user and to communicate the risk of developing CVD in response to one’s levels of PA and SB.
The recommendation system
Design of the system as a theoretically driven behavioural intervention.
To design the system, we followed the steps needed to develop an intervention according to the BCW. Firstly, we specified the target behaviour (i.e. increasing PA and reducing SB) and the associated factors through a literature review. Ovid and Embase databases were systematically searched to include studies identifying barriers and facilitators to the target behaviours, specifically in type 2 diabetes patients. We selected five studies [49–53] that identified factors associated with the target behaviours pertaining to all COM-B model domains (capability, opportunity and motivation). The search terms and a summary of the selected papers are provided in the supporting information (S4 Text and S2 Table). All factors associated with the target behaviours that were identified by the selected papers are presented in Table 1, including those that cannot be addressed by a mHealth intervention (such as environmental factors), in order to have a more complete picture of the elements influencing physical behaviours in people with type 2 diabetes. Secondly, we mapped the identified factors to the COM-B model. And lastly, we identified behaviour change techniques through the help of the Theoretical Domains Framework (TDF) [54], and we mapped them to system features. The design process is shown in Table 1.
According to the COM-B model, an intervention must match the patient’s abilities to be effective and sustained. Challenges recommended by the system have to align with the patients’ skills and constantly adjust as these change over time. Therefore, in the proposed system, the recommendations are initialised to the patient’s baseline activity and optimised over time depending on the probability of compliance and leptin levels. The probability of compliance with the recommendations is paramount to prevent feelings of failure in the patient, which can hinder engagement with the intervention. Moreover, PA and SB trends enable self-monitoring and self-regulation, which increase motivation. Finally, rewards are embedded in the system as a time-to-onset of comorbidities gained or lost in response to their new activity behaviour.
Classification into diabetes cluster.
The first component of the proposed system is the classification of the patient into one of the diabetes clusters. To select the best classifier, three models—K-nearest neighbours (KNN), support vector classifier (SVC) and random forest (RF)—were trained and validated on the clustering set (n = 26400). The test set was selected as the set of under-68 years old subjects (n = 348) later used in the prediction of survival times, as this set is used to simulate the system and thus should not be used to train the classifier.
For each of the three models, features were standardised and selected through forward sequential feature selection in mlxtend [55], using balanced accuracy as the selecting metric. The model hyperparameters were tuned with randomised search cross-validation, using the scikit-learn RandomizedSearchCV function which implements a fit and score method [40], with balanced accuracy being selected as the scoring method. The tuned models with the optimal number of features were then validated on the test set.
Optimised recommendation module.
The main component of the system is the recommendation module, which issues optimised recommendations to improve metabolic health while keeping the probability of compliance high. The core inputs for the recommendation module are leptin levels, probability of compliance, and upper and lower limit of PA and SB.
Leptin varies in relation to dieting therapies and thus, leptin levels are a real-time physiological signal providing feedback on the efficacy of the intervention. During diets, leptin levels change at a greater rate than weight and can predict weight loss two weeks in advance [56]. Therefore, monitoring leptin can be more meaningful than monitoring weight, and can be employed as feedback to adjust recommendations if future weight gain is predicted. Monitoring leptin levels is also relevant for chronic disease management, as hyperleptinemia is correlated to increased risk of obesity and cardiovascular outcomes in T2D patients [57].
The probability of compliance (Eq 1) is related to the historical levels of activity and therefore, encodes the skill level of the patients. In order to keep compliance high, the recommendations are adjusted based on the proportion of recommendations followed in the past and their past activity levels. To include the effect of genetics on motivation, the recommendation score is also weighted by a polygenic risk score. The polygenic risk score is calculated using effect sizes reported by Doherty et al. [58] that quantify the association between selected Single Nucleotide Polymorphisms (SNP) and PA or SB. Information on the selected SNP is given in Table 2.
(1)
(2) R is the number of recommendations issued and
is the polygenic risk score, where gi is the genotype and ei is the effect size corresponding to SNP i. The factor α = 1 − |Ar − Ap| represents the influence of past activity on the probability of compliance, where Ar is the new recommended activity and Ap is the predicted activity estimated through fitting a logistic function on the historical activity levels. To ensure 0 ≤ α ≤ 1, the recommended and predicted activity are calculated as daily proportions. The algorithm to determine the optimised recommendation is shown in Fig 3.
Simulations
To simulate the effects of the recommender system without real-life users, we simulated physical activity based on whether the proposed recommendation was followed or not, which was modelled through different probability distributions. If the recommendation was followed, the PA (or SB) level is set equal to the recommendation, otherwise, it is set equal to the level recorded (or simulated) on the previous day. PA and SB levels are reported as portion of the day. We used the data presented by Finkelstein et al. [59] and by Meyer et al. [60] to model the probability of following a recommendation over time.
The effectiveness of the system in inducing behaviour change was simulated on the young dataset with no missing genetic data (n = 290). The proposed system was simulated with the aforementioned models for a duration of 365 days.
As leptin levels were not available in the UK Biobank, we assigned baseline leptin levels using the regression coefficients estimated by Zuo et al. [61] to predict leptin from gender, age, BMI, WC, arm fat mass, energy intake, PA and smoking. Missing data were replaced by the sample mean. However, for energy intake there was a significant amount of missing data, and values were replaced through multivariate sample imputation [40]. Leptin was updated based on the results found in a systematic review by Fedewa et al. [62], which found that engaging in chronic exercise (≥ 2 weeks) is associated with a reduction of leptin with an effect size of 0.24. Consequently, leptin is reduced by the effect size if the recommendations are followed for 2 weeks; otherwise, it is kept unchanged.
Results
Cluster analysis
GMM with n = 4 and a spherical covariance matrix was chosen as the best clustering method based on the silhouette, Calisnki-Harabasz and Davies-Boulding scores. The four clusters generated by GMM achieved a Jaccard score ≥ 0.75 and therefore were deemed stable.
The cluster characteristics are summarised in Table 3. Cluster 1 had lower T2D onset age, lower BMI/WC and improved cardiometabolic profile. Cluster 2 had an average T2D onset age (around 51 years for both males and females), higher BMI and slightly worse blood pressure and CRP than cluster 1. Clusters 3 and 4 were characterised by higher onset age and worse cardiometabolic profile, with cluster 4 having the highest values of blood pressure, lipids (combining cholesterol, HDL-C, LDL-C, apolipoprotein A and B, and lipoprotein A), glucose, HbA1c and CRP across all clusters. The differences between clusters were similar in males and females, as shown in Table 3. However, the prevalence of comorbidities differed between clusters, with males cluster 3 having the highest prevalence of hypertension and CVD (31% and 65%, respectively) and males cluster 4 having the highest prevalence of high cholesterol (40%). For females, cluster 4 had the highest prevalence of high cholesterol (43%) and CVD (25%); and cluster 3 had the highest prevalence of hypertension (67%). Cluster 1, for both males and females, was characterised by significantly higher insulin use (28% in males, 33% in females vs < 19% in other clusters).
Association between PA, SB, and CVD.
Given the similar cluster characteristics (see Table 3), males and females were combined for regression analysis. After backward feature selection, the final models for the clusters are:
(3)
(4)
(5)
The odd of CVD and mean time spent in PA and SB for each cluster is presented in Table 4. Sensitivity analyses produced similar results, thus confirming the robustness of the coefficients.
PA and SB are reported in minutes per day.
The time steps required to reduce the risk of CVD by 1% for each cluster are presented in Table 5. Some of the steps in Table 5 were adjusted, given the wrong step direction due to p-value>0.05. Specifically, the PA time steps for clusters 2 ad 4 are replaced by the time step for the whole sample (20 minutes/day). The SB time step for cluster 2 is set to 18 minutes/day, given the similarity in SB levels with clusters 1 and 3.
The log(odds) correspond to changes in CVD log-odds for increases in time spent in the given behaviour and corresponding decrease in the others.
Effect of PA and SB on survival function.
Fig 4 shows the Kaplan-Meier survival functions estimated on censored subjects (n = 1215), where the biggest difference is observed between clusters 1 and 4. This result is expected, since cluster 4 presents a worse cardiometabolic profile than cluster 1 and thus, is more prone to CVD development. Fig 5 shows the effects of changing SB and PA on cluster 2 as a representative example; the effect of PA is visibly pronounced, while the effects of varying sedentary time are more subtle.
For survival regression, Cox’s model was selected as it obtained the highest concordance score (0.74 for Cox, Weibull and Log logit, 0.73 for Log normal), which measures the accuracy of the ranking of predicted times. The estimated Cox’s model was used to predict the cluster-dependent survival times for subjects younger than 68 years. Survival times are calculated the month at which the survival function crosses p = 0.5 and are shown in Table 6.
Data presented as mean (standard deviation).
Recommender system classifier selection
Based on the best accuracy scores, three features were selected for KNN and SVC (sex, lipoprotein A and triglycerides), while four were selected for RF (sex, pulse rate, LDL-cholesterol and triglycerides). While the three classifiers achieved comparable performance, the support vector classifier was chosen as the preferred classifier as it uses fewer features than the Random Forest and reached a better accuracy than the KNN. The performance for each classifier tested is shown in Table 7.
Simulated effect of system on behaviour change
We simulated the proposed system with three different probability models for a duration of 365 days. The probability of following the recommendations on any given day was sampled from a Bernoulli distribution, and we modelled the parameter p with 3 different functions that reflect the time-dependent nature of adherence to human interventions. The models were estimated from data reported by Finkelstein et al. [59] (model in Eq 6, Fig 6A), who measured the percentage of participants who wore the activity tracker for at least 1 day in a week and by Meyer et al. [60] (model in Eq 7, Fig 6B), who recorded the average number of days in a week that participants wore the tracker. The parameters for both models were estimated with least squares [63]. The third model was time-invariant, with constant p over the duration of the intervention (Eq 8).
(6)
(7)
(8)
The data was used to fit the models for adherence probability described in Eqs 6 and 7, respectively.
The resulting activity and adherence profiles for the young dataset are presented in Fig 7. Within the young dataset, four participants had follow-up accelerometer data available, collected on average approximately 1.5 years after the baseline measurements. Consequently, it was possible to compare the activity at follow-up with the simulated activity at a time matching the time difference between baseline and follow-up measurements. Fig 8 shows that, although only a constantly high adherence (p = 0.8) to the intervention would bring significant improvements in activity compared to baseline, all adherence models achieved improvements compared to activity measured at follow-up. For PA, such difference was statistically significant for all models evaluated (p-value < 0.05, tested with paired t-test).
The top graphs show the proportion of the day spent in physical activity and sedentary behaviour across the simulated intervention. The bottom graphs show the simulated adherence to the intervention as a function of days. Data presented as mean and standard deviation.
Discussion
In this paper, we presented a novel analysis of T2D clusters and their associations with physical behaviours and CVD. We found four clusters phenotypically distinct clusters of T2D based on 16 cardiometabolic markers routinely measured in clinical settings. The clustering confirms the presence of different types of diabetes characterised by different cardiometabolic profiles. The clusters found in this analysis resemble the clusters found by Ahlqvist et al. [15]. They found one cluster which was insulin-dependent, with low BMI and poor metabolic control (similar to cluster 1); and other two clusters with higher onset age, no insulin resistance and with obesity (similar to clusters 3 and 4). However, compared to the clusters found by Ahlqvist et al., the clusters described here did not present differences in glucose metabolism (especially hbA1c), but rather highlight the differences in overall cardiometabolic health, suggesting that the differences in diabetes clusters may be a product of lifestyle differences. This hypothesis is supported by different levels of physical behaviours between the clusters: cluster 1 spend less time sedentary compared to the sample average and more time engaging in physical activity compared to the sample average. Conversely, clusters 3 and 4 are the most sedentary and least active clusters. This analysis suggests that people with T2D may benefit from individualised behavioural change interventions depending on their cardiometabolic profile and their T2D subtype.
Consequently, in this paper we present a new knowledge-based system for personalised activity recommendations. The system aims to increase PA and reduce SB in people with T2D by providing recommendations with a high chance of compliance based on the patient’s T2D cluster, previous activity, genetic risk and leptin levels. We trained and simulated the system using data from the UK Biobank and demonstrated that the system may help increase levels of activity. In fact, simulating the system with different probabilities of adherence showed improved physical activity compared to real-life activity measured after 1.5 years from baseline. Despite testing the system with computer simulations, we modelled the behavioural response to the system using real data collected on human participants by [59] and [60], who recorded the weekly wear of an activity tracker for 52 weeks. While these data are suitable for initial assessment, implementation of the system as a real-life intervention and user testing are needed to reliably assess the effectiveness of the system.
The novelty of the presented work lies in the adoption of a holistic approach to behaviour change interventions: in fact, we used a wide array of data—including biomarkers, genotype, leptin levels and past activity—to achieve highly personalised recommendations. A novel concept in our work is the use of genetic information to optimise the personalised recommendations. Genetic risk communication has been previously used for lifestyle behaviour change interventions, but has been found to have a very small effect on reducing unhealthy behaviours such as smoking, physical activity and nutrition [64]. However, to our knowledge, no intervention has been developed that uses genetic markers associated with the target behaviour. The inclusion of such markers may contribute to improving lifestyle behaviours because, in our system, genetic information is related to the capability of the subject. On the other hand, previous interventions have mapped genetic information to motivation and specifically to the knowledge of the risks associated to a poor lifestyle. Future research should confirm whether genetic associations can be used to directly inform lifestyle recommendations and achieve a significant change in lifestyle behaviours.
Limitations of the current work
This work presents several limitations. Firstly, the behavioural analysis is limited as it only employed findings from a literature review, while conducting new qualitative research with the target population could have provided further insights into barriers and facilitators of the target behaviour, specifically in regard to mHealth interventions. Nonetheless, it is possible to improve the behavioural analysis by conducting focus groups or semi-structured interviews with the target population and incorporating the results in future versions of the proposed system.
The system is evaluated through computer simulations rather than through a randomised controlled trial, which is the gold standard to assess the efficacy of an intervention [65]. A direct consequence is the impossibility of implementing features, such as goal customisation, which require the user’s input. Additionally, at this stage, it is not possible to assess the system’s user-friendliness and how it would impact the acceptance and efficacy of the intervention. It is also not possible to assess whether communicating the risk of developing CVD is the best way to increase motivation, and other rewards may be more effective. Additionally, several issues can undermine the efficacy of the system in real-life settings, which could not be accounted for in simulations. Firstly, the engagement with the system is currently modelled with time-changing probability distributions, which may be different from the real-world adherence to the system’s recommendations. Secondly, simulations cannot capture the individual variation in leptin levels in different individuals and the response to different types of physical activity. Therefore, modelling leptin can introduce errors in the optimisation of recommendations and thus in the efficacy simulations. Thirdly, simulations cannot account for all the factors that may affect users’ activity and behaviour and therefore, the system efficacy may be invalidated in free-living settings.
A further limitation of the current system version is the use of overall activity, which combines any activity not classified as sedentary time or sleep. Classifying activities based on type and intensity may be more beneficial for the user as it can provide a more precise indication of which kind of activity one should replace sedentary time with.
Considerations for real-world implementation
The promising results presented in this paper warrant the implementation of the system into a real-life intervention, whose efficacy can be tested with a randomised controlled trial. To implement the intervention, the system would be integrated with a point-of-care test for leptin, which we presented in previous works [66, 67]. Additionally, the recommendations can be augmented with information on activity mode and duration, which are especially important when dealing with cardiometabolic conditions that require special considerations due to the associated risks. Additional gamification features, such as goal setting and scoreboards, can be integrated into the system to render the system customisable by the user and increase self-efficacy. As data availability increases and new features are added to the system, data can be represented using knowledge graphs to capture complex relationships and better integrate data from different sources. Specifically, a personal knowledge graph [68] would be used to define structural relationships between the patient and their data, which can then be used to improve recommendations, infer missing data, and predict disease risk.
However, before implementing the system into a real-world intervention, several considerations have to be made. While personalisation promotes acceptance of mHealth services [69], it also raises privacy and data security concerns. Tangari et al. [70] found that very few mHealth apps respect user privacy–88% of apps found in the major app stores can access and share personal data. As the present system relies on sensitive health data, its implementation must ensure privacy and user data security by using encrypted HTTP traffic and encrypting the data stored in the user’s device. Therefore, while mHealth applications often present a lack of data security, ensuring a high level of data protection is achievable in order to optimise the risk-benefit trade-off of mHealth interventions.
Once the initial implementation of the mHealth system is completed according to the best data protection practices, a first round of user testing should be conducted to assess the system’s usability, perceived benefits, and prospective acceptability. Subsequently, the system should be updated to reflect the feedback collected during the user trial. Finally, a randomised controlled trial would assess the efficacy of the proposed system, which is a first attempt at providing comprehensive and highly personalised activity recommendations by linking sensing data with genetics and daily tracking. If proved effective and acceptable in real-life settings, implementation of such theories could have a major impact in the prevention and management of chronic diseases that are triggered by behavioural practices.
Supporting information
S2 Text. List of diseases within the CVD classification.
https://doi.org/10.1371/journal.pdig.0000333.s002
(DOCX)
S3 Text. Compositional data and isometric log-ratio transformation.
https://doi.org/10.1371/journal.pdig.0000333.s003
(DOCX)
S4 Text. Ovid search strategy for review on barriers and facilitators to physical activity and sedentary behaviour.
https://doi.org/10.1371/journal.pdig.0000333.s004
(DOCX)
S2 Table. Summary of selected studies assessing barriers and facilitators to physical activity and sedentary behaviour in people with type 2 diabetes.
https://doi.org/10.1371/journal.pdig.0000333.s006
(DOCX)
References
- 1. Buckingham SA, Williams AJ, Morrissey K, Price L, Harrison J. Mobile health interventions to promote physical activity and reduce sedentary behaviour in the workplace: A systematic review. Digital Health. 2019;5:1–50. pmid:30944728
- 2. Monteiro-Guerra F, Rivera-Romero O, Fernandez-Luque L, Caulfield B. Personalization in Real-Time Physical Activity Coaching Using Mobile Applications: A Scoping Review. IEEE Journal of Biomedical and Health Informatics. 2020;24(6):1738–1751. pmid:31751254
- 3. Tong HL, Quiroz JC, Kocaballi AB, Fat SCM, Dao KP, Gehringer H, et al. Personalized mobile technologies for lifestyle behavior change: A systematic review, meta-analysis, and meta-regression. Preventive Medicine. 2021;148(November 2020):106532. pmid:33774008
- 4. Bol N, Høie NM, Nguyen MH, Smit ES. Customization in mobile health apps: explaining effects on physical activity intentions by the need for autonomy. Digital Health. 2019;5:1–12. pmid:31807312
- 5. Bandura A. Self-efficacy: Toward a unifying theory of behavioral change. Advances in Behaviour Research and Therapy. 1978;1(4):139–161.
- 6.
Ni J, Muhlstein L, McAuley J. Modeling heart rate and activity data for personalized fitness recommendation. The Web Conference 2019—Proceedings of the World Wide Web Conference, WWW 2019. 2019;2:1343–1353.
- 7.
Mahyari AG, Pirolli P. Physical exercise recommendation and success prediction using interconnected recurrent neural networks. arXiv. 2020; p. 1–9.
- 8. Zhao Z, Arya A, Orji R, Chan G. Effects of a personalized fitness recommender system using gamification and continuous player modeling: System design and long-term validation study. JMIR Serious Games. 2020;8(4):1–27. pmid:33200994
- 9. Shrivastava S, Trung TQ, Lee NE. Recent progress, challenges, and prospects of fully integrated mobile and wearable point-of-care testing systems for self-testing. Chemical Society Reviews. 2020;49(6):1812–1866. pmid:32100760
- 10. Horton R, Crawford G, Freeman L, Fenwick A, Wright CF, Lucassen A. Direct-to-consumer genetic testing. BMJ. 2019;367. pmid:31619392
- 11. Chen CH, Karvela M, Sohbati M, Shinawatra T, Toumazou C. PERSON—Personalized Expert Recommendation System for Optimized Nutrition. IEEE Transactions on Biomedical Circuits and Systems. 2018;12(1):151–160. pmid:29377803
- 12. Davies MJ, D’Alessio DA, Fradkin J, Kernan WN, Mathieu C, Mingrone G, et al. Management of Hyperglycemia in Type 2 Diabetes, 2018. A Consensus Report by the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes Care. 2018;41(12):2669–2701. pmid:30291106
- 13. Sardinha LB, Magalhães JP, Santos DA, Júdice PB. Sedentary Patterns, Physical Activity, and Cardiorespiratory Fitness in Association to Glycemic Control in Type 2 Diabetes Patients. Frontiers in Physiology. 2017;8:262. pmid:28503154
- 14. Howland C, Wakefield B. Assessing telehealth interventions for physical activity and sedentary behavior self-management in adults with type 2 diabetes mellitus: An integrative review. Research in Nursing & Health. 2021;44(1):92–110. pmid:33091168
- 15. Ahlqvist E, Storm P, Käräjämäki A, Martinell M, Dorkhan M, Carlsson A, et al. Novel subgroups of adult-onset diabetes and their association with outcomes: a data-driven cluster analysis of six variables. The Lancet Diabetes and Endocrinology. 2018;6(5):361–369. pmid:29503172
- 16. Anjana RM, Baskar V, Nair ATN, Jebarani S, Siddiqui MK, Pradeepa R, et al. Novel subgroups of type 2 diabetes and their association with microvascular outcomes in an Asian Indian population: A data-driven cluster analysis: The INSPIRED study. BMJ Open Diabetes Research and Care. 2020;8(1):1506.
- 17. Cho SB, Kim SC, Chung MG. Identification of novel population clusters with different susceptibilities to type 2 diabetes and their impact on the prediction of diabetes. Scientific Reports. 2019;9(1):1–9. pmid:30833619
- 18. Karpati T, Leventer-Roberts M, Feldman B, Cohen-Stavi C, Raz I, Balicer R. Patient clusters based on HbA1c trajectories: A step toward individualized medicine in type 2 diabetes. PLoS ONE. 2018;13(11). pmid:30427908
- 19. Arévalo Lorido JC, Carretero Gómez J, Gómez Huelgas R, Quirós López R, Dávila Ramos MF, Serrado Iglesias A, et al. Comorbidity in patients with type 2 diabetes mellitus and heart failure with preserved ejection fraction. Cluster analysis of the RICA registry. Opportunities for improvement. Revista Clínica Española (English Edition). 2020;220(7):409–416. pmid:31932045
- 20. Alexandre K, Vallet F, Peytremann-Bridevaux I, Desrichard O. Identification of diabetes self-management profiles in adults: A cluster analysis using selected self-reported outcomes. PLOS ONE. 2021;16(1):1–17. pmid:33481883
- 21. Amato MC, Pizzolanti G, Torregrossa V, Pantò F, Giordano C. Phenotyping of type 2 diabetes mellitus at onset on the basis of fasting incretin tone: Results of a two-step cluster analysis. Journal of Diabetes Investigation. 2016;7(2):219–225. pmid:27042274
- 22. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The Behavior Change Technique Taxonomy (v1) of 93 Hierarchically Clustered Techniques: Building an International Consensus for the Reporting of Behavior Change Interventions. Annals of Behavioral Medicine. 2013;46(1):81–95. pmid:23512568
- 23. Shoneye CL, Mullan B, Begley A, Pollard CM, Jancey J, Kerr DA. Design and development of a digital weight management intervention (today): Qualitative study. JMIR mHealth and uHealth. 2020;8(9):1–17. pmid:32641284
- 24. Chai LK, May C, Collins CE, Burrows TL. Development of text messages targeting healthy eating for children in the context of parenting partnerships. Nutrition and Dietetics. 2019;76(5):515–520. pmid:30426627
- 25. Brown A, Nathan N, Yoong S, Janssen L, Chooi A, Hudson N, et al. A multicomponent mHealth-based intervention (SWAP IT) to decrease the consumption of discretionary foods packed in school lunchboxes: Type I effectiveness-implementation hybrid cluster randomized controlled trial. Journal of Medical Internet Research. 2021;23(6):1–16. pmid:34185013
- 26. Truelove S, Vanderloo LM, Tucker P, Di Sebastiano KM, Faulkner G. The use of the behaviour change wheel in the development of ParticipACTION’s physical activity app. Preventive Medicine Reports. 2020;20:101224. pmid:33134041
- 27. Henshall C, Davey Z. Development of an app for lung cancer survivors (iEXHALE) to increase exercise activity and improve symptoms of fatigue, breathlessness and depression. Psycho-Oncology. 2020;29(1):139–147. pmid:31773808
- 28. Stephenson A, Garcia-Constantino M, McDonough SM, Murphy MH, Nugent CD, Mair JL. Iterative four-phase development of a theory-based digital behaviour change intervention to reduce occupational sedentary behaviour. Digital Health. 2020;6:1–15. pmid:32257366
- 29. Wheeler TS, Michael Vallis T, Giacomantonio NB, Abidi SR. Feasibility and usability of an ontology-based mobile intervention for patients with hypertension. International Journal of Medical Informatics. 2018;119(March):8–16. pmid:30342690
- 30. Jennings HM, Morrison J, Akter K, Kuddus A, Ahmed N, Kumer Shaha S, et al. Developing a theory-driven contextually relevant mHealth intervention. Global Health Action. 2019;12(1). pmid:31154988
- 31. Rodriguez DV, Lawrence K, Luu S, Yu JL, Feldthouse DM, Gonzalez J, et al. Development of a computer-aided text message platform for user engagement with a digital Diabetes Prevention Program: a case study. Journal of the American Medical Informatics Association. 2021;29(1):155–162. pmid:34664647
- 32. Fulton EA, Brown KE, Kwah KL, Wild S. Stopapp: Using the behaviour change wheel to develop an app to increase uptake and attendance at NHS stop smoking services. Healthcare (Switzerland). 2016;4(2). pmid:27417619
- 33. Tombor I, Shahab L, Brown J, Crane D, Michie S, West R. Development of SmokeFree Baby: a smoking cessation smartphone app for pregnant smokers. Translational Behavioral Medicine. 2016;6(4):533–545. pmid:27699682
- 34. Garnett C, Crane D, West R, Brown J, Michie S. The development of Drink Less: An alcohol reduction smartphone app for excessive drinkers. Translational Behavioral Medicine. 2019;9(2):296–307. pmid:29733406
- 35. Ribaut J, Leppla L, Teynor A, Valenta S, Dobbels F, Zullig LL, et al. Theory-driven development of a medication adherence intervention delivered by eHealth and transplant team in allogeneic stem cell transplantation: The SMILe implementation science project. BMC Health Services Research. 2020;20(1):1–22. pmid:32878623
- 36. Arden MA, Hutchings M, Whelan P, Drabble SJ, Beever D, Bradley JM, et al. Development of an intervention to increase adherence to nebuliser treatment in adults with cystic fibrosis: CFHealthHub. Pilot and Feasibility Studies. 2021;7(1):1–18. pmid:33390191
- 37. Bowers HM, Kendrick T, Glowacka M, Williams S, Leydon G, May C, et al. Supporting antidepressant discontinuation: The development and optimisation of a digital intervention for patients in UK primary care using a theory, evidence and person-based approach. BMJ Open. 2020;10(3). pmid:32152159
- 38. Wadden TA, Considine RV, Foster GD, Anderson DA, Sarwer DB, Caro JS. Short-and long-term changes in serum leptin in dieting obese women: effects of caloric restriction and weight loss. The Journal of Clinical Endocrinology & Metabolism. 1998;83(1):214–218. pmid:9435444
- 39. Sudlow C, Gallacher J, Allen N, Beral V, Burton P, Danesh J, et al. UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age. PLOS Medicine. 2015;12(3):1–10. pmid:25826379
- 40. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research. 2011;12:2825–2830.
- 41. Rousseeuw PJ. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics. 1987;20(C):53–65.
- 42. Hennig C. Cluster-wise assessment of cluster stability. Computational Statistics and Data Analysis. 2007;52(1):258–271.
- 43.
Seabold S, Perktold J. statsmodels: Econometric and statistical modeling with python. In: 9th Python in Science Conference; 2010.
- 44. Willetts M, Hollowell S, Aslett L, Holmes C, Doherty A. Statistical machine learning of sleep and physical activity phenotypes from sensor data in 96,220 UK Biobank participants. Scientific Reports. 2018;8(1):7961. pmid:29784928
- 45. Chastin SFM, Palarea-Albaladejo J, Dontje ML, Skelton DA. Combined effects of time spent in physical activity, sedentary behaviors and sleep on obesity and cardio-metabolic health markers: A novel compositional data analysis approach. PLoS ONE. 2015;10(10):e0139984. pmid:26461112
- 46. Dumuid D, Stanford TE, Martin-Fernández JA, Pedišić Ž, Maher CA, Lewis LK, et al. Compositional data analysis for physical activity, sedentary time and sleep research. Statistical Methods in Medical Research. 2018;27(12):3726–3738. pmid:28555522
- 47. Damen JAAG, Hooft L, Schuit E, Debray TPA, Collins GS, Tzoulaki I, et al. Prediction models for cardiovascular disease risk in the general population: Systematic review. BMJ (Online). 2016;353. pmid:27184143
- 48. Davidson-Pilon C. lifelines: survival analysis in Python. Journal of Open Source Software. 2019;4(40):1317.
- 49. Avery L, Denton S, Lavender M, Sniehotta F, Trenell M. Barriers and enabling factors to use of a physical activity behavioural intervention for adults with Type 2 diabetes delivered in primary care: qualitative findings from the Movement as Medicine open pilot study. Diabetic Medicine. 2015;32(Suppl. 1):30–206.
- 50. Erickson D. Barriers to Physical Activity in People With Type 2 Diabetes Enrolled in a Worksite Diabetes Disease Management Program. The Diabetes Educator. 2013;39(5):626–634. pmid:23782623
- 51. Swoboda CM, Miller CK, Wills CE. Frequency of diet and physical activity goal attainment and barriers encountered among adults with type 2 diabetes during a telephone coaching intervention. Clinical Diabetes. 2017;35(5):286–293. pmid:29263571
- 52. Casey D, De Civita M, Dasgupta K. Understanding physical activity facilitators and barriers during and following a supervised exercise programme in Type 2 diabetes: A qualitative study. Diabetic Medicine. 2010;27(1):79–84. pmid:20121893
- 53. Cradock KA, Quinlan LR, Finucane FM, Gainforth HL, Martin Ginis KA, de Barros AC, et al. Identifying barriers and facilitators to diet and physical activity behaviour change in type 2 diabetes using a design probe methodology. Journal of Personalized Medicine. 2021;11(2):1–26. pmid:33530618
- 54. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implementation Science. 2012;7:37. pmid:22530986
- 55. Raschka S. MLxtend: Providing machine learning and data science utilities and extensions to Python’s scientific computing stack. The Journal of Open Source Software. 2018;3(24).
- 56. Mars M, De Graaf C, De Groot CPGM, Van Rossum CTM, Kok FJ. Fasting leptin and appetite responses induced by a 4-day 65%-energy-restricted diet. International Journal of Obesity. 2006;30(1):122–128. pmid:16158086
- 57.
Katsiki N, Mikhailidis DP, Banach M. Leptin, cardiovascular diseases and type 2 diabetes mellitus review-article; 2018. Available from: http://www.nature.com/articles/aps201840.
- 58. Doherty A, Smith-Byrne K, Ferreira T, Holmes MV, Holmes C, Pulit SL, et al. GWAS identifies 14 loci for device-measured physical activity and sleep duration. Nature Communications. 2018;9(1):5257. pmid:30531941
- 59. Finkelstein EA, Haaland BA, Bilger M, Sahasranaman A, Sloan RA, Nang EEK, et al. Effectiveness of activity trackers with and without incentives to increase physical activity (TRIPPA): a randomised controlled trial. The Lancet Diabetes and Endocrinology. 2016;4(12):983–995. pmid:27717766
- 60.
Meyer J, Wasmann M, Heuten W, El Ali A, Boll SCJ. Identification and classification of usage patterns in long-term activity tracking. Conference on Human Factors in Computing Systems—Proceedings. 2017;2017-May:667–678.
- 61. Zuo H, Shi Z, Yuan B, Dai Y, Wu G, Hussain A. Association between Serum Leptin Concentrations and Insulin Resistance: A Population-Based Study from China. PLoS ONE. 2013;8(1). pmid:23349940
- 62. Fedewa MV, Hathaway ED, Ward-Ritacco CL, Williams TD, Dobbs WC. The Effect of Chronic Exercise Training on Leptin: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Sports Medicine. 2018;48(6):1437–1450. pmid:29582381
- 63.
Newville M, Stensitzki T, Allen DB, Rawlik M, Ingargiola A, Nelson A. Lmfit: Non-Linear Least-Square Minimization and Curve-Fitting for Python; 2016.
- 64. Hollands GJ, French DP, Griffin SJ, Prevost AT, Sutton S, King S, et al. The impact of communicating genetic risks of disease on riskreducing health behaviour: Systematic review with meta-analysis. BMJ (Online). 2016;352(October). pmid:26979548
- 65. Akobeng AK. Understanding randomised controlled trials. Archives of Disease in Childhood. 2005;90(8):840–844. pmid:16040885
- 66. Cavallo FR, Mirza KB, De Mateo S, Nikolic K, Rodriguez-Manzano J, Toumazou C. Aptasensor for Quantification of Leptin through PCR Amplification of Short DNA-Aptamers. ACS Sensors. 2021;6(3):709–715. pmid:33650854
- 67.
Cavallo FR, Mirza KB, de Mateo S, Manzano JR, Nikolic K, Toumazou C. A Point-of-Care Device for Sensitive Protein Quantification. 2021 IEEE International Symposium on Circuits and Systems (ISCAS). 2021;1:1–5.
- 68.
Balog K, Kenter T. Personal Knowledge Graphs: A Research Agenda. Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval. 2019; p. 217–220.
- 69. Guo X, Zhang X, Sun Y. The privacy–personalization paradox in mHealth services acceptance of different age groups. Electronic Commerce Research and Applications. 2016;16:55–65.
- 70. Tangari G, Ikram M, Ijaz K, Kaafar MA, Berkovsky S. Mobile health and privacy: cross sectional study. BMJ. 2021;373. pmid:34135009