Mixed methods qualitative design combining four data sources; (i) analytic data extracted directly from the software system, (ii) participant survey, (iii) focus groups to identify barriers and enablers to implementation and mechanisms of effect and (iv) recruitment screening logs and text message responses to examine engagement. This evaluation was conducted within the TEXT ME trial—a parallel design, single-blind randomized controlled trial (RCT) of 710 patients with coronary heart disease (CHD). Qualitative data were interpreted using inductive thematic analysis.
307/352 (87% response rate) of recruited patients with CHD completed the program evaluation survey at six months and 25 participated in a focus group. Factors increasing engagement included (i) ability to save and share messages, (ii) having the support of providers and family, (iii) a feeling of support through participation in the program, (iv) the program being initiated close to the time of a cardiovascular event, (v) personalization of the messages, (vi) opportunity for initial face-to-face contact with a provider and (vii) that program and content was perceived to be from a credible source. Clear themes relating to program delivery were that diet and physical activity messages were most valued, four messages per week was ideal and most participants felt program duration should be provided for at least for six months or longer.
This study provides context and insight into the factors influencing consumer engagement with a text message program aimed at improving health-related behavior. The study suggests program components that may enhance potential success but will require integration at the development stage to optimize up-scaling.
Australia and New Zealand Clinical Trials Registry, ACTRN12611000161921.
Citation: Redfern J, Santo K, Coorey G, Thakkar J, Hackett M, Thiagalingam A, et al. (2016) Factors Influencing Engagement, Perceived Usefulness and Behavioral Mechanisms Associated with a Text Message Support Program. PLoS ONE 11(10): e0163929. https://doi.org/10.1371/journal.pone.0163929
Editor: Gavin Daker-White, University of Manchester, UNITED KINGDOM
Received: May 12, 2016; Accepted: September 16, 2016; Published: October 14, 2016
Copyright: © 2016 Redfern et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The focus group transcripts contain a significant amount of identifying information and we do not have ethical approval to make these publicly available. We are very happy to make these available upon request to Professor Julie Redfern (firstname.lastname@example.org). All other relevant data are within the paper and its Supporting Information files.
Funding: This work is supported by a National Heart Foundation of Australia Grant-in-Aid (G10S5110) and a BUPA Health Foundation Grant. JR is funded by a National Health and Medical Research Council (NHMRC) Career Development Fellowship (1061793) co-funded with a National Heart Foundation Future Leader Fellowship (G160523). KS is funded by a University of Sydney International Postgraduate Research Scholarship. JT is funded by a University of Sydney Australian Postgraduate Award scholarship. MH is funded by a National Heart Foundation Future Leader Fellowship (1000034). CKC is funded by a Career Development Fellowship co-funded by the NHMRC and National Heart Foundation as well as a Sydney Medical Foundation Chapman Fellowship.
Competing interests: The authors have declared that no competing interests exist.
Coronary heart disease (CHD) remains the leading cause of death and disability worldwide. Although the incidence of CHD has declined in regions such as the United States and United Kingdom due to population health improvements in risk factors, such as smoking cessation, healthier diet and increased physical activity,[2,3] the number of people surviving major cardiovascular events is increasing. As such, a large percentage of hospitalisations for CHD are due to recurrent events and effective risk factor reduction, termed secondary prevention, can reduce these and therefore reduce the CHD burden.[5,6] Cardiac rehabilitation (CR) aims to deliver structured secondary prevention to people with CHD, however, these services are consistently underused and overall adherence to secondary prevention strategies remains suboptimal.[8,9,10] Consequently, new strategies in secondary prevention are required if we are to increase access and support for patients after hospital discharge.
In recent years, mobile phone text-message based interventions have arisen as a potential means of modifying health behaviors. Recent studies have demonstrated the effectiveness of mobile phone text messaging to change individual health behaviours such as smoking cessation,[12,13,14] weight loss, physical activity, blood pressure lowering and improved management of diabetes. However, translation of these programs to public health or health service delivery has been limited and evidence suggests that reducing multiple risk factors concurrently is more effective in reducing cardiovascular events than targeting single risk factors. In addition, very few studies included objective outcome measures and only the minority reported program evaluations with links to behavioural and psychological theory.
As recognised within the UK Medical Research Council framework for the development and evaluation of complex interventions, assessment of fidelity and causal mechanisms optimises the likelihood of implementation in clinical practice. A recent ‘systematic mapping review’ that detailed the value of qualitative research alongside randomized controlled trials (RCTs) found that it improved external validity, facilitated interpretation and provided an understanding of contextual factors that are important in terms of transferability and implementation. Overall, there are numerous published studies reporting outcomes of text message based interventions but very few have have addressed and assessed their fidelity and how certain components can lead to behavioural change with psychological underpinning. Therefore, the aim of this study was to conduct a program evaluation (using mixed methods) to examine features and factors that contribute to the effectiveness of a successful text messaging program in terms of increased user/participant engagement, perceived usefulness (in relation to message frequency, content and personalization), behavior change and program delivery preferences so as to inform future research and implementation.
Materials and Methods
We undertook a mixed methods approach within the Tobacco, EXercise and dieT messages (TEXT ME) single-blind RCT that aimed to reduce cardiovascular risk in 710 patients with CHD.[22,23] Results reporting clinical effectiveness of the TEXT ME intervention are reported elsewhere. The intervention group received four text messages per week for six months (further detail on the messaging program below) in additional to usual care and the control group received usual care without the text message program. In summary, at six months, the intervention group had significantly lower LDL cholesterol, systolic blood pressure (BP), body mass index (BMI), smoking rates and significantly higher physical activity levels, compared to the control group. Patients provided written informed consent and ethical approval was obtained from the Western Sydney Local Health District Human Research Ethics Committee.
The evaluation plan is summarised in Table 1. This program evaluation was designed alongside the TEXT ME trial and therefore data were collected in parallel to the RCT. The chosen methods follow a realist evaluation model which seeks to understand human choices and actions, within a systems context. Qualitative and quantitative data sources comprised; (i) analytic data extracted directly from the text message software system, (ii) survey of participants to identify usefulness and engagement, (iii) focus groups to identify features influencing user engagement as well as barriers and enablers to program implementation in order to understand context and mechanisms of change and; (iv) recruitment screening logs to examine generalizability and engagement. In the context of this research, the term ‘engagement’ refers to the concept of user interaction with the program where it is determined by a combination of user interaction with the program and their viewpoints regarding what program features made them more likely to read and follow the advice. The combination of survey (all intervention participants invited) and focus groups allowed for synthesis of specific participant viewpoints in addition to exploration of more detailed responses via focus groups. The evaluation examined fidelity of the intervention, usefulness and user engagement as well as delivery model and content of the intervention.
We also examined the data for evidence of behavior change techniques and links with psychological theory based on a published taxonomy. As reported previously, development of the text message program was based on a range of behavior change techniques including: the provision of information and encouragement; prompting about consequences, intention formation, monitoring self-behavior, and barrier identification; advice about setting graded tasks; and strategies aimed at relapse prevention and the use of prompting and cues. Within this present evaluation, we explored the focus group data, using thematic analysis, for links to these theoretical frameworks so as to explore mechanisms and context associated with behavior change.
Text messaging program
The text messaging program and its development process are detailed elsewhere.[22,23,27] The message program development was based on behavior change techniques linked to the theoretical framework reported by Abraham and Michie. In brief, it involved regular semi-personalised text messages providing advice, motivation, and information that aimed to improve general heart health, diet, physical activity and encourage smoking cessation (where relevant). Example messages are provided in Box 1. All participants received four messages per week for 24 weeks (smokers received fewer general heart health messages to accommodate smoking cessation messages). Messages were sent on four of five randomly selected week-days and arrived at random times during working hours. The messages were semi-personalized using baseline data about the participant’s preferred name and if the participant was vegetarian or a smoker. Participants were asked not to respond but they were informed the program would be managed by a computerized messaging engine and that they could reply with a message stating “STOP” if they wished to opt out of the program or they could contact the research coordinator listed on their patient information sheet.
BOX 1: Examples of text messages used in the TEXT ME trial*
[NAME], try identifying the triggers that make you want a cigarette & plan to avoid them
[NAME], for many it may take several attempts to quit, so keep trying
Did you know 90% of people don’t eat the recommended daily intake of vegetables (5 serves a day)?
Try avoiding adding salt to your foods by using other spices or herbs
Hi [NAME], don’t forget physical activity is good for you! It reduces your risk of diabetes, heart attack, stroke, and their complications
Walking is cheap. It can be done almost anywhere. All you need is comfortable shoes & clothing
General Cardiovascular Information
Have you got a chest pain Action Plan [NAME]? Find ideas at http://www.heartfoundation.org.au/Pages/default.aspx
Studies show that stress, worry & loneliness can increase the risk of heart disease. Please talk to a health professional if you need help
*Reproduced with permission from JAMA 2015.314(12):1. Copyright (2015) American Medical Association. All rights reserved.
1. Text-message software system analytics.
We pre-specified in developing the TEXT ME message management system to record data on (TEXT ME Engine V1.0, Cardiology Department, Westmead Hospital) the total number of messages sent to each intervention participant and the time point that messages stopped being sent. In addition, any text message responses received from a participant throughout the course of the study were automatically logged and subsequently coded to enable identification of positive and negative responses as well as analysis of their content.
2. User surveys.
To examine acceptability and feasibility of the TEXT ME program, participants allocated to the intervention group were administered a survey (in confidence) after the blinded six month follow-up assessment. The survey comprised a total of 18 items: 15 required either 5-point Likert or yes/no categorical responses that explored utility, influence on behavior, potential for sharing messages and items addressing message frequency, timing and program duration. There were three items that provided an opportunity for free text feedback about usefulness and suggestions for improvement.
3. Focus groups.
At final follow-up, participants allocated to the TEXT ME intervention program were invited to participate in a focus group discussion. Stratified purposeful sampling was used to identify potential participants to ensure a variety of views were explored. This included ensuring participants in each focus group had a balance of gender, ethnicity, revascularization and history of cardiac rehabilitation participation. Recruitment for focus groups was consecutive until thematic saturation was achieved (determined when no new descriptive concepts were emerging). Standard focus group methods were used including the use of a facilitator (JR), scribe, setting of ground-rules and audio-recording. A set of 12 open-ended prompts were used to guide and facilitate discussion around key topics (message delivery, content, usefulness and impact). Focus groups took place at the local hospital where participants were initially admitted. An additional independent person attended the focus groups to serve as a coordinator, manage recordings and take brief notes. The facilitator has more than 10 years of experience with CVD research, had no established relationship with participants and was not the Principal Investigator of the RCT.
4. Screening logs.
As part of the program, a screening log was kept to determine participation and reach. During the screening process to determine eligibility, basic details about potential participants and reasons for non-participation were recorded. This log was maintained by the study research assistant and were stored in a secure electronic database.
Demographic information and categorical results were summarised as means and proportions. Surveys were analysed for themes and the results were used to guide the development of focus group discussion points. To develop a coding framework based on the focus group data, a constant comparative method was used.[29,30] Two researchers (JR, GC) independently coded transcripts, line by line, and through two iterative stages a coding framework was developed. The coders were experienced clinician (nurse and allied health) researchers with expertise in CVD management, behaviour change and mixed methods research. Data checking was achieved by discussion between members of the multidisciplinary research team who are experts in the development and evaluation of secondary prevention programs. Where disagreements existed over the interpretation of data, these were resolved by consensus. Themes were then discussed amongst the full research group for review and further analysis. Throughout the coding process we constantly compared the content with previously coded data. This iterative process including collapsing and expanding various codes and themes until no new concepts were emerging. Direct quotations from the focus groups were used to illustrate some key emergent themes. In accordance with focus group methods, analysis examined the perspective of individuals and as well as emerging consensus between the participants around the issues being discussed.
Of the 352 participants who received the text messaging program, 307 (87%) completed the user survey at six months. Their mean age was 58 (±8.9) years and most were male 255/307 (83%). A total of 25 participants took part in four focus groups and their mean age was 62 (±6) years and 15/25 (60%) were male. Focus groups were held at Westmead Hospital in Sydney between April 2013 and April 2014 for 1.5 hours duration each. Demographic and summary information about the survey and focus group participants is presented in Table 2 (see also S1 and S2 Tables).
In terms of reach, a total of 1301 patients with CHD were approached to participate in the TEXT ME trial (of which 710 were enrolled into the trial and 352 were randomised to the intervention group). Of those approached, the majority owned a mobile phone (1036/1301, 80%) and very few had privacy concerns (8/1301, 0.6%) or were unable to communicate via text message (9/1301, 0.7%). Of the 591 who did not participate reasons included; not owning a mobile phone (44.8%, 265/591), not able to read messages in English language (34.7%, 205/591), decline to participate in research (11.8%, 70/591) and other non-specific reasons (8.6%, 51/591) . In terms of fidelity, based on analytic data extract from the software system, 96% (338/352) of participants allocated to the intervention group received all 98 scheduled messages during the six months. During the intervention period, participants were asked not to respond to messages they received, however, we still received a total of 308 text message replies from 116 unique mobile telephone numbers. Most of the replies received were related to saying thank you (39%) but others were to provide feedback about behavior (18%), to make a general comment (16%), to unsubscribe (12%), request further information (7%), extraneous comment unrelated to the program (6%) or were complaints (2%).
Factors influencing user engagement with the text message program
Analysis of qualitative data collected in the focus groups confirmed a high level of user engagement. Seven themes emerged as factors influencing program user engagement (Fig 1). These included; (i) the ability to save and share messages, (ii) having the support of providers and family, (iii) a feeling of support through receipt of the program, (iv) the program being initiated close to the time of a cardiovascular event, (v) personalization of the messages, (vi) opportunity for initial face-to-face contact with a provider and (vii) that the message content was consistent with previous advice and from a credible source. Table 3 provides direct quotations from focus group participants illustrating the identified themes and factors influencing engagement. User engagement was also demonstrated by the fact that despite participants being asked not to respond to messages we received a total of 403 text message replies from 134 distinct mobile telephone numbers during the six month intervention period. Most of the replies received were related to saying thank you (30%), to provide feedback about a healthy behavior (14%) or to make a general comment (12%). As further evidence of engagement, in the user survey, 96% of participants reported that during the course of the program they read at least three-quarters of all text messages. In addition, 54% of survey participants and 55% of focus group participants reported that they saved or shared the text messages and 21% left the messages in their inbox.
Program delivery and perceived usefulness
Overall, the text message-based program was considered to be convenient, instant and private. Focus group and survey participants were consistently positive about the program and the potential for continuation with illustrative quotations as follows.
“Think it is a good choice to offer to everyone in every hospital” (“C”, male, 65, FG1)
“This should continue, this program is excellent” (“C”, female, 51, FG1)
“I think that the program has a great positive impact….it should be available to everyone” (“D”, male, 61, FG3)
These findings are supported by previously reported survey data demonstrating that most participants reported the program was useful (91%), easy to understand (97%) and motivating (77%). Also, when asked about behavior related to specific risk factors as a result of the messages, 81% reported adopting a more healthy diet, 73% reported they increased their physical activity and 76% reported that the messages reminded them to take their prescribed medications. These self-reported improvements directly align with significant quantitative improvements in cardiovascular risk factors including biomedical measures of BP and cholesterol as well as validated questionnaires assessing physical activity and nutrition.
Using qualitative analysis, we specifically evaluated the program barriers and enablers in relation to characteristics of (i) message content, (ii) delivery frequency, (iii) message delivery timing, (iv) program duration and (v) one-way communication.
i. Message content.
In focus groups, the consensus was that the positive content, practical nature and variety of message content was good, easy to understand and included nothing offensive. In terms of content, there was a clear theme that the diet and physical activity messages were considered the most valuable (Table 4). However, participants found the medication and clinical information useful as well (Table 4). For improvement, participants suggested more content related to eating out and mental health and that the program could allow for variation to pre-defined data, for example, if they stopped smoking during the course of the program. In total, 96% of survey participants reported that the language of the text messages was appropriate with the minority reporting that the messages were too casual (2%) or too formal (2%).
ii. Message delivery frequency.
In the user survey, 87% of participants reported that the number of messages they received (4 per week) was appropriate with 8% reporting that there were too few and 5% reporting there were too many. When we explored this further in the focus groups, most participants felt the random timing of messages was important and that the frequency was generally considered ideal. Participants suggested that too many messages per day would be overwhelming and could lead to reduced engagement. Some of the comments related to message frequency include the following;
iii. Message timing.
Most survey and focus group participants reported that the timing of message receipt was appropriate with illustrative quotes as follows.
“It is good that the messages weren’t sent in the weekends” (“C”, female, 51, FG2)
‘I think receiving the text message late at night, like 10pm, is not what you want” (“J”, male, 62, FG3)
“I think Monday to Friday is perfect” (“K”, 60, female, FG1)
On further analysis in the focus groups, participants clearly supported the random delivery timing and felt this helped keep them interested and engaged.
iv. Program duration.
The primary theme emanating from the focus groups in relation to program duration was that it could have been longer than six months but with a lower message frequency after month six with illustrative quotations such as;
“Once it finished, I found myself failing again” (“D”, male, 61, FG3)
“One day they stopped, that was that last one. I would say I missed them in some way, because I was used to getting them” (“J”, male, 68, FG3)
“Maybe you can drop down to 2 messages a week for the second 6 months” (“S”, male, 67, FG2)
“Would it be possible to run the program for more than 6 months” (“S”, male, 67, FG2)
These above findings were consistent with the findings of the user survey where 79% felt the six month duration was appropriate, 16% felt it was too short and 5% felt it was too long. Considering the survey and focus group data together, there was a theme that program length of at least 6 months was ideal with a possible option for a longer duration.
v. One way communication.
There were mixed views about the one-way communication. Overall, there was consensus that it would be ideal to offer a two-way communication feature for those who are interested but two-way communication would not be required for most participants.
“I’m not a communicator, so I like the idea of being passive. I find it is fine like it is” (“D”, male, 61, FG3)
“I think it would be absolutely brilliant to be able to communicate with someone” (“D”, male, 65, FG2)
“Perhaps in a future program, like communicating back once in every 6 months or once in 3 months” (“A”, male, 50, FG4)
“Maybe every 9 or 6 months to have a phone call, maybe some volunteers who have had heart attacks and that you can talk to them if you have some questions” (“J”, male, 70, FG4)
Links with psychological theory and behavior change techniques
Three dominant theoretical frameworks were identified as potentially contributing to the observed behavior change and the TEXT ME program. The Information-Motivation-Behavior Skills Theoretical Model could primarily explain the observed behavior change associated with the TEXT ME program. This theory states that if a person is well-informed, motivated to act, and they have the skills and confidence to take action, they are more likely to initiate and maintain health-promoting behaviors that produce positive outcomes. In the context of our messaging program, a theme emerged that participants felt motivated and engaged by the program when it provided credible and meaningful information with illustrative quotations links with this theory including:
“So the messages prompt me to research all my medications” (“D”, male, 65, FG2)
“I actually researched how many calories you should eat in a day and what’s in everything and I set a spreadsheet and I started calculating what you should eat and what you shouldn’t eat.” (“A”, male, 50, FG4)
“Any info is good. After just having a heart attack I wanted more info about what I should be doing and what shouldn’t” (“T”, female, 49, FG1)
“It is good to receive some information about what the medications do, because sometimes I received information that the GP didn’t tell me about, which was really good” (“T”, female, 49, FG1)
“The messages gave me some insight into new things that I had never thought of, trying different foods” (“G”, male, 58, FG2)
Although the program was a one-way text message communication system, qualitative data suggest that Social-Cognitive Theory explains some of the observed behavior change. Social cognitive theory suggests that human behavior is determined by a combination of cognitive, personal and environmental influences. These reflect an interaction between thought, affect and action where expectations, beliefs, self- perceptions, goals and intentions give shape and direction to behavior. Further, behaviour is modified based on whether the individual has high or low self-efficacy toward the behavior, the response a person receives after performing the behavior and aspects of the environment that improve self-efficacy including the setting of achievable goals. In the current text messaging program, social recognition, improved self-efficacy, the provision of achievable task-setting suggestions, practical advice and positive reinforcement were all built into the program and quotes illustrating their successful implementation are below.
“I found the program encouraging” (“B”, female, 60, FG1)
“It gives a good feeling just knowing that there is someone else who cares and kept an eye on it/you like a good link” (“K”, male, 58, FG1)
“It made me feel good when sometimes I already did my exercise and then got a message about it. And then I was like ‘Yes, beat you’” (“B”, female, 60, FG1)
Operant and its predecessor Classical Conditioning are theoretical frameworks[34,35] for explaining behavior and this concept of ‘respondent condition’ appeared to emerge from the focus groups where participants responded to the repeated messaging within the program as a whole rather than the immediate content of individual messages. This type of conditioning was first reported by Ivan Pavlov where he demonstrated that dogs learned a stimulus-response connection in response to repeated stimuli such that a previously neutral stimulus eventually elicits a response. Our qualitative data suggest that over time participants became ‘conditioned’ towards healthy behaviors irrespective of the specific content of the individual messages received at a certain time. That is, through repeated presentation of a stimulus (in this case cardiovascular health related text messages) the participants eventually learned to associate the stimulus with overall cardiovascular health behaviour and consequently messages about diet also resulting in improved physical activity and vice versa. The notion that the program evoked a feeling of feeling “guilty” could also contribute to conditioning where the feeling of guilt is a noxious stimuli associated with ‘unhealthy’ behaviour. Illustrative quotes highlighting this concept include;
“It wasn’t particularly about what the message said that day” (“C”, female, 51, FG2)
“They always popped up before I was about to eat something naughty” (“W”, male, 68, FG1)
“I don’t actually go for context of the message, but look for the little nudge. The little things that nudge you to do the right things” (“D”, male, 65, FG 2)
“When you see the text message you might be a little bit more conscious about what you are eating” (“T”, male, 62, FG4)
“If I didn’t walk for the past 3 or 4 nights, because it was too hot or raining, you made a bit more effort to do it, you felt a bit guilty. And the same with checking your cholesterol and about what you are eating, think about the portions etc. It is an on-going thing, a general thing. But the reminder or the nag that was the key” (“S”, male, 76, FG2)
“I love chocolate and one time I was in the aisle of [shop name] thinking about getting a snack… then suddenly I received a message of TEXT ME saying: Hi [name], feeling like a snack? A banana or a piece of fruit! It really gave a big brother feeling to me, how could they know I was thinking about picking a snack? I thought well forget it lets go and walked away with a smile on my face. I got busted” (“B”, female, 60, FG1)
This process evaluation found that patients with CHD were engaged with and positive about a text message program aimed at supporting behaviour change. Results suggest that engagement was increased by several factors including the program being initiated close to the time of hospitalisation, the consistency of the messages with prior in-hospital education and management, having the opportunity for face-to-face contact at program commencement and the personalization of messages. We found features that impacted on behavioural change were the positive content, the sense of feeling supported, the delivery time being random and that most participants felt program duration could have been longer than six months. Through thematic analysis this research has identified several potential drivers of behavior change within a text messaging program and therefore facilitates understanding of components of the intervention that may increase effectiveness.
The primary goal of this evaluation was to help with understanding why text message-based programs for people with CHD are effective, the ingredients for success and how engagement can be optimized so as to inform development of other text message programs aimed at behavioural change in patients with other type of chronic disease. Overall, an important role for program evaluations is to examine the quantity and quality of what was actually implemented in practice and why.[20,21] Through understanding of barriers and enablers as well as reach and fidelity, program evaluations can inform future implementation of similar interventions and facilitate interpretation of intervention outcomes. It is well known that exploring issues such as acceptability and perceived usefulness from a consumer perspective enables program developers and those responsible for implementation to better understand intervention delivery and reach. The use of a mixed methods approach allows the integration of numerical data with perspective gained from a qualitative approach. In common with our results, other mixed methods studies of health-behavior interventions have found that most patients want disease specific information and self-management strategies. Our intervention was based on behavioural theory and was developed via a formal process involving consumers. The presented qualitative data provide a deeper understanding of the mechanisms associated with the observed program effectiveness. Such findings will be available for development of future programs targeting patients with CVD and other health conditions so as to inform policy and upscaling of similar programs designed to improve behaviour change.
Very few studies have reported qualitative evaluations alongside a RCT of a text message-based intervention for improving health-related behavior. One study tested the effectiveness of prompts sent via text message in terms of improving BP management via adherence to clinic visits and treatment. The qualitative evaluation was conducted via focus groups aiming to explore trial participants’ experiences and identified barriers to intervention delivery. The study concluded that the message program was acceptable, relevant, provided practical support for patients and that intervention success coincided with participant readiness for change. Similar to our study, Leon et al reported that most participants found the text messages with practical tips for healthy living were most useful, that the messages improved adherence, that participants were comfortable receiving the messages and that a polite and respectful tone was important.
Understanding the theory associated with behavior change allows consideration of hypothesized causal processes involved. In the case of the TEXT ME messaging program, the messages themselves were based on theory and understanding the factors influencing engagement and the mechanisms associated with the observed clinical outcomes is important for scalability, uptake and future implementation. Gaining an understanding of these mechanisms informs development of future content, delivery frequency and structure. Insight into the mechanisms of behavior change augments the clinical outcomes and ultimately enables a deeper understanding of the effectiveness. This level of analysis is important because ultimately a person’s behaviour is the result of their individual or collective action and it is a key determinant of health. This concept is particularly important in relation to CHD where lifestyle risk factors are leading contributors to morbidity and mortality. The behaviours that cause and contribute to CHD are common and changing them often means a person is required to make multiple and long-term changes to their everyday behavior. For example, a person may be recommended to stop smoking, increase their physical activity level and take daily medications for the remainder of their life. A National Institute for Health and Clinical Excellence guideline highlights the importance of identification and evaluation of mechanisms associated with behaviour change and that mixed method ethnographic research, longitudinal studies and qualitative approaches are needed. Gaining an understanding of underlying theoretical factors influencing behaviour help establish why an intervention is working so as to inform decisions relating to implementation and improvement .
While this mixed methods evaluation provides significant insight into the potential mechanisms of intervention effectiveness and opportunities for optimising engagement, the study is not without limitations. Firstly, the sample was recruited from within a RCT from a tertiary hospital which might limit the generalisability of the intervention, however this hospital has broad socioeconomic and multicultural representation. Secondly, our sample size for women was small and therefore examination of gender differences requires further investigation. It is acknowledged that we did not stratify for age and that there were more people with higher years of education participating in the focus groups and that our findings will vary between individuals and naturally certain groups should not be denied access to potential interventions. The qualitative nature of this research enables features to be examined in detail and allows consideration of human experience. Future research is needed to identify the effectiveness of various program design features. For example, to specifically determine whether it is ideal to offer one-way or two-way communication or whether this should be offered to participants by choice. In addition, future research could clearly define the ideal message delivery frequency, timing and the ideal program duration.
This process evaluation found that patients with CHD were engaged with and positive about a text message program aimed at supporting behaviour change. Exploring the theoretical frameworks provides insight into the causal pathway between the text-message intervention, behavior change and impact on risk factor measures. Gaining an understanding of the factors that influence engagement with the program as well as the advantages and disadvantages of various program characteristics can inform the development of other programs, identifies the key elements important to maintain in larger-scale programs and gives clues to the directions of future research to improve programs and support their sustained effectiveness.
S1 Table. Focus Group Demographic information.
We thank the Westmead Hospital Cardiologists for supporting the recruitment and TEXT ME program development at Westmead hospital: Robert Abraham, FRACP, Mikhail Altman, PhD,William Chik, PhD, James Chong, PhD, Mark Cooper, PhD, Lloyd Davis, PhD, A. Robert Denniss, PhD, Eddy Kizana, PhD, Pramesh Kovoor, PhD, Mahidi Mardini, PhD, Andrew Ong, PhD, Paul Russell, FRACP, David Ross, FRACP, Norman Sadick, PhD, Gopal Sivagangabalan, PhD, Michael Skinner, PhD, David Tanous, PhD, Timothy Tan, PhD, and Stuart Thomas, PhD.
- Conceptualization: JR MH JT AT CC.
- Data curation: JR KS GC MH.
- Formal analysis: JR KS GC MH JT AT CC.
- Funding acquisition: JR MH AT CC.
- Investigation: JR KS MH JT AT CC.
- Methodology: JR KS MH JT AT CC.
- Project administration: JR CC.
- Resources: JR AT CC.
- Software: JR JT AT CC.
- Supervision: JR CC.
- Validation: JR KS GC MH JT AT CC.
- Visualization: JR KS CC.
- Writing – original draft: JR KS GC MH JT AT CC.
- Writing – review & editing: JR KS GC MH JT AT CC.
- 1. World Health Organisation: The top 10 cause of death. Fact Sheet No 310. Updated May 2014. ed. 2014.
- 2. Ford ES, Ajani UA, Croft JB, Critchley JA, Labarthe DR, Kottke TE, et al. Explaining the decrease in U.S. deaths from coronary disease, 1980–2000. N Engl J Med 2007;356:2388–2398. pmid:17554120
- 3. Hardoon SL, Whincup PH, Lennon LT, Wannamethee SG, Capewell S, Morris RW. How much of the recent decline in the incidence of myocardial infarction in British men can be explained by changes in cardiovascular risk factors? Evidence from a prospective population-based study. Circulation. 2008;117:598–604. pmid:18212284
- 4. Briffa T, Hobbs M, Tonkin A, Sanfilippo FM, Hickling S, Ridout S et al. Population trends of current coronary heart disease event rates remain high. Circ Cardiovasc Qual Outcomes. 2011;4:107–113. pmid:21139089
- 5. Ades PA. Cardiac rehabilitation and secondary prevention of coronary heart disease. N Engl J Med 2001; 345: 892–902. pmid:11565523
- 6. Taylor RS, Brown A, Ebrahim S, Jolliffe J, Noorani H, Skidmore B, et al. Exercise-based rehabilitation for patients with coronary heart disease: systematic review and meta-analysis of randomized controlled trials. Am J Med 2004;116:682–692. pmid:15121495
- 7. Clark AM, Hartling L, Vandermeer B, McAlister FA. Meta-analysis: secondary prevention programs for patients with coronary artery disease. Ann Intern Med. 2005;143:659–72. pmid:16263889
- 8. Redfern J, Hyun K, Chew DP, Astley C, Chow C, Aliprandi-Costa B, et al. Prescription of secondary prevention medications, lifestyle advice and referral to rehabilitation among acute coronary syndrome (ACS) inpatients: results from a large prospective audit in Australia and New Zealand. Heart. 2014;100(16);1281–1288. pmid:24914060
- 9. Chow CK Jolly S, Rao-Melacini P, Fox KA, Anand SS, Yusuf S. Association of diet, exercise, and smoking modification with risk of early cardiovascular events after acute coronary syndromes. Circulation. 2010;121(6):750–758. pmid:20124123
- 10. Rasmussen JN, Chong A, Alter DA. Relationship between adherence to evidence-based pharmacotherapy and long-term mortality after acute myocardial infarction. J Am Med Assoc. 2007;297(2):177–186.
- 11. Fjeldsoe BS, Marshall AL, Miller YD. Behavior change interventions delivered by mobile telephone short-message service. Am J Prev Med. 2009;36(2):165–73. pmid:19135907
- 12. Free C Knight R, Robertson S, Whittaker R, Edwards P, Zhou W, et al. Smoking cessation support delivered via mobile phone text messaging (txt2stop): a single-blind, randomised trial. Lancet. 2011:378:49–55. pmid:21722952
- 13. Spohr SA, Nandy R, Gandhiraj D, Vemulapalli A, Anne S, Walters ST. Efficacy of SMS text message interventions for smoking cessation: A meta-analysis. J Subst Abuse Treat 2015;56:1–10. pmid:25720333
- 14. Scott-Sheldon LAJ, Lantini RC, Jennings EG, Thind H, Rosen R, Salmoirago-Blotcher E & Bock BC. (2016). Text messaging-based interventions for smoking cessation: A systematic review and meta-analysis. Journal of Medical Internet Research: mHealth and uHealth 2016;4(2), e49.
- 15. Patrick K, Raab F, Adams MA, Dillon L, Zabinski M, Rock CL, et al. A text message- based intervention for weight loss: randomized controlled trial. Journal of Medical Internet Research. 2011;11(1):e1.
- 16. Hurling R, Catt M, Boni MD, Fairley BW, Hurst T, Murray P, et al. Using internet and mobile phone technology to deliver an automated physical activity program: randomized controlled trial. Journal of Medical Internet Research. 2007;9(2): pmid:17478409
- 17. Bobrow K, Farmer AJ, Shanyinde M et al. Mobile Phone Text Messages to Support Treatment Adherence in Adults With High Blood Pressure (StAR): A Single-Blind, Randomized Trial. Circulation. 2016, online first DOI: pmid:26769742
- 18. Buis LR, Hirzel L, Turske SA, Des Jardins TR, Yarandi H, Bondurant P. Use of a Text Message Program to Raise Type 2 Diabetes Risk Awareness and Promote Health Behavior Change (Part II): Assessment of Participants' Perceptions on Efficacy. Journal of Medical Internet Research. 2013;15(12):e282. pmid:24356359
- 19. Smith SC Jr. Current and future directions of cardiovascular risk prediction. Am J Cardiol. 2006;97(2A):28A–32A. pmid:16442934
- 20. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J. 2015;350:h1258.
- 21. O'Cathain A, Thomas KJ, Drabble SJ, Rudolph A, Hewison J. What can qualitative research do for randomised controlled trials? A systematic mapping review. BMJ Open. 2013;3:e002889 pmid:23794542
- 22. Chow CK, Redfern J, Thiagalingam A, Jan S, Whittaker R, Hackett M, et al. Design and rationale of the tobacco, exercise and diet messages (TEXT ME) trial of a text message-based intervention for ongoing prevention of cardiovascular disease in people with coronary disease: a randomised controlled trial protocol. BMJ Open. 2012;2(1):e000606. pmid:22267690
- 23. Chow CK, Redfern J, Hillis GS, Thakkar J, Santo K, Hackett ML, et al. Effect of Lifestyle-Focused Text Messaging on Risk Factor Modification in Patients With Coronary Heart Disease: A Randomized Clinical Trial. J Am Med Assoc. 2015;314(12):1255–63.
- 24. Pawson R., & Tilley N. (1997). Realistic evaluation. London: Sage.
- 25. Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol. 2008; 27: 379–387. pmid:18624603
- 26. Austin JT, Vancouver JB. Goal constructs in psychology: Structure, process, and content. Psychological Review. 1996;120:338–375.
- 27. Redfern J, Thiagalingam A, Jan S, Whittaker R, Hackett ML, Mooney J, et al. Development of a set of mobile phone text messages designed for prevention of recurrent cardiovascular events. Eur J Prev Cardiol. 2014;21(4):492–499. pmid:22605787
- 28. Barbour R. Doing Focus Groups. 1st ed. Flick U, editor. Los Angeles: Sage Publications; 2007.
- 29. Ritchie J, Spencer L, editors. Qualitative data analysis for applied policy research. London: Routledge; 1994.
- 30. Curry LA, Nembhard IM, Bradley EH. Qualitative and mixed methods provide unique contributions to outcomes research. Circulation 2009;119(10):1442–52. pmid:19289649
- 31. Fisher JD. Fisher WA. Amico KR. Harman JJ. An information-motivation-behavioral skills model of adherence to antiretroviral therapy. Health Psychol. 2006;25:462–473. pmid:16846321
- 32. Bandura A. (1997). Self-efficacy: The exercise of control. NewYork: Freeman.
- 33. Bandura A. (1989). Social cognitive theory. In Vasta R. (Ed.), Annals of child development. Vol.6. Six theories of child development (pp. 1–60). Greenwich, CT: JAI Press.
- 34. Skinner B. F. (1938). The behavior of organisms. New York: Appleton-Century-Crofts.
- 35. Pavlov IP. (1955). Selected works. Moscow: Foreign Languages Publishing House.
- 36. Process evaluation for public health interventions and research. Steckler Allan (Ed); Linnan Laura (Ed) San Francisco, CA, US: Jossey-Bass. (2002).
- 37. Walker ER, Engelhard G, Barmon C, McGee RE, Sterk CE, Diiorio C, e al. A mixed methods analysis of support for self-management behaviors: perspectives of people with epilepsy and their support providers. Epilepsy Behav. 2014;31:152–159. pmid:24413284
- 38. Leon N, Surender R, Bobrow K, Muller J, Farmer A. Improving treatment adherence for blood pressure lowering via mobile phone SMS-messages in South Africa: a qualitative evaluation of the SMS-text Adherence SuppoRt (StAR) trial. BMC Family Practice. 2015;16:80. pmid:26137844
- 39. Michie S, Johnston M, Francis JJ, Hardeman W, Eccles M. From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Applied Psychology 2008:57(4);660–680.
- 40. Clark AM. What are the components of complex interventions in healthcare? Theorizing approaches to parts, powers and the whole intervention. Social Science and Medicine 2013:93(9):185–193.
- 41. National Institute for Health and Clinical Excellence (NICE) (2007). NICE Public Health Guidance 6 ‘Behaviour change at population, community and individual levels’. London: NICE.