Design

We undertook a mixed methods approach within the Tobacco, EXercise and dieT messages (TEXT ME) single-blind RCT that aimed to reduce cardiovascular risk in 710 patients with CHD.[22,23] Results reporting clinical effectiveness of the TEXT ME intervention are reported elsewhere.[23] The intervention group received four text messages per week for six months (further detail on the messaging program below) in additional to usual care and the control group received usual care without the text message program. In summary, at six months, the intervention group had significantly lower LDL cholesterol, systolic blood pressure (BP), body mass index (BMI), smoking rates and significantly higher physical activity levels, compared to the control group.[23] Patients provided written informed consent and ethical approval was obtained from the Western Sydney Local Health District Human Research Ethics Committee.

The evaluation plan is summarised in Table 1. This program evaluation was designed alongside the TEXT ME trial and therefore data were collected in parallel to the RCT. The chosen methods follow a realist evaluation model which seeks to understand human choices and actions, within a systems context.[24] Qualitative and quantitative data sources comprised; (i) analytic data extracted directly from the text message software system, (ii) survey of participants to identify usefulness and engagement, (iii) focus groups to identify features influencing user engagement as well as barriers and enablers to program implementation in order to understand context and mechanisms of change and; (iv) recruitment screening logs to examine generalizability and engagement. In the context of this research, the term ‘engagement’ refers to the concept of user interaction with the program where it is determined by a combination of user interaction with the program and their viewpoints regarding what program features made them more likely to read and follow the advice. The combination of survey (all intervention participants invited) and focus groups allowed for synthesis of specific participant viewpoints in addition to exploration of more detailed responses via focus groups. The evaluation examined fidelity of the intervention, usefulness and user engagement as well as delivery model and content of the intervention.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. TEXT ME Program evaluation plan.

https://doi.org/10.1371/journal.pone.0163929.t001

We also examined the data for evidence of behavior change techniques and links with psychological theory based on a published taxonomy.[25] As reported previously, development of the text message program was based on a range of behavior change techniques including: the provision of information and encouragement; prompting about consequences, intention formation, monitoring self-behavior, and barrier identification; advice about setting graded tasks; and strategies aimed at relapse prevention and the use of prompting and cues.[26] Within this present evaluation, we explored the focus group data, using thematic analysis, for links to these theoretical frameworks so as to explore mechanisms and context associated with behavior change.

Text messaging program

The text messaging program and its development process are detailed elsewhere.[22,23,27] The message program development[27] was based on behavior change techniques linked to the theoretical framework reported by Abraham and Michie.[25] In brief, it involved regular semi-personalised text messages providing advice, motivation, and information that aimed to improve general heart health, diet, physical activity and encourage smoking cessation (where relevant). Example messages are provided in Box 1. All participants received four messages per week for 24 weeks (smokers received fewer general heart health messages to accommodate smoking cessation messages). Messages were sent on four of five randomly selected week-days and arrived at random times during working hours. The messages were semi-personalized using baseline data about the participant’s preferred name and if the participant was vegetarian or a smoker. Participants were asked not to respond but they were informed the program would be managed by a computerized messaging engine and that they could reply with a message stating “STOP” if they wished to opt out of the program or they could contact the research coordinator listed on their patient information sheet.

Data sources

Analysis

Demographic information and categorical results were summarised as means and proportions. Surveys were analysed for themes and the results were used to guide the development of focus group discussion points. To develop a coding framework based on the focus group data, a constant comparative method was used.[29,30] Two researchers (JR, GC) independently coded transcripts, line by line, and through two iterative stages a coding framework was developed. The coders were experienced clinician (nurse and allied health) researchers with expertise in CVD management, behaviour change and mixed methods research. Data checking was achieved by discussion between members of the multidisciplinary research team who are experts in the development and evaluation of secondary prevention programs. Where disagreements existed over the interpretation of data, these were resolved by consensus. Themes were then discussed amongst the full research group for review and further analysis. Throughout the coding process we constantly compared the content with previously coded data. This iterative process including collapsing and expanding various codes and themes until no new concepts were emerging. Direct quotations from the focus groups were used to illustrate some key emergent themes. In accordance with focus group methods, analysis examined the perspective of individuals and as well as emerging consensus between the participants around the issues being discussed.[29]

Factors influencing user engagement with the text message program

Analysis of qualitative data collected in the focus groups confirmed a high level of user engagement. Seven themes emerged as factors influencing program user engagement (Fig 1). These included; (i) the ability to save and share messages, (ii) having the support of providers and family, (iii) a feeling of support through receipt of the program, (iv) the program being initiated close to the time of a cardiovascular event, (v) personalization of the messages, (vi) opportunity for initial face-to-face contact with a provider and (vii) that the message content was consistent with previous advice and from a credible source. Table 3 provides direct quotations from focus group participants illustrating the identified themes and factors influencing engagement. User engagement was also demonstrated by the fact that despite participants being asked not to respond to messages we received a total of 403 text message replies from 134 distinct mobile telephone numbers during the six month intervention period. Most of the replies received were related to saying thank you (30%), to provide feedback about a healthy behavior (14%) or to make a general comment (12%). As further evidence of engagement, in the user survey, 96% of participants reported that during the course of the program they read at least three-quarters of all text messages. In addition, 54% of survey participants and 55% of focus group participants reported that they saved or shared the text messages and 21% left the messages in their inbox.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Overview of factors influencing engagement with the text message program.

https://doi.org/10.1371/journal.pone.0163929.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Quotations illustrating factors influencing user engagement with text message program.

https://doi.org/10.1371/journal.pone.0163929.t003

Program delivery and perceived usefulness

Overall, the text message-based program was considered to be convenient, instant and private. Focus group and survey participants were consistently positive about the program and the potential for continuation with illustrative quotations as follows.

“Think it is a good choice to offer to everyone in every hospital” (“C”, male, 65, FG1)

“This should continue, this program is excellent” (“C”, female, 51, FG1)

“I think that the program has a great positive impact….it should be available to everyone” (“D”, male, 61, FG3)

These findings are supported by previously reported survey data demonstrating that most participants reported the program was useful (91%), easy to understand (97%) and motivating (77%).[23] Also, when asked about behavior related to specific risk factors as a result of the messages, 81% reported adopting a more healthy diet, 73% reported they increased their physical activity and 76% reported that the messages reminded them to take their prescribed medications.[23] These self-reported improvements directly align with significant quantitative improvements in cardiovascular risk factors including biomedical measures of BP and cholesterol as well as validated questionnaires assessing physical activity and nutrition.[23]

Using qualitative analysis, we specifically evaluated the program barriers and enablers in relation to characteristics of (i) message content, (ii) delivery frequency, (iii) message delivery timing, (iv) program duration and (v) one-way communication.

i. Message content.

In focus groups, the consensus was that the positive content, practical nature and variety of message content was good, easy to understand and included nothing offensive. In terms of content, there was a clear theme that the diet and physical activity messages were considered the most valuable (Table 4). However, participants found the medication and clinical information useful as well (Table 4). For improvement, participants suggested more content related to eating out and mental health and that the program could allow for variation to pre-defined data, for example, if they stopped smoking during the course of the program. In total, 96% of survey participants reported that the language of the text messages was appropriate with the minority reporting that the messages were too casual (2%) or too formal (2%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Quotations illustrating participant views relating to message content and perceived usefulness.

https://doi.org/10.1371/journal.pone.0163929.t004

Links with psychological theory and behavior change techniques

Three dominant theoretical frameworks were identified as potentially contributing to the observed behavior change and the TEXT ME program. The Information-Motivation-Behavior Skills Theoretical Model[31] could primarily explain the observed behavior change associated with the TEXT ME program. This theory states that if a person is well-informed, motivated to act, and they have the skills and confidence to take action, they are more likely to initiate and maintain health-promoting behaviors that produce positive outcomes.[31] In the context of our messaging program, a theme emerged that participants felt motivated and engaged by the program when it provided credible and meaningful information with illustrative quotations links with this theory including:

“So the messages prompt me to research all my medications” (“D”, male, 65, FG2)

“I actually researched how many calories you should eat in a day and what’s in everything and I set a spreadsheet and I started calculating what you should eat and what you shouldn’t eat.” (“A”, male, 50, FG4)

“Any info is good. After just having a heart attack I wanted more info about what I should be doing and what shouldn’t” (“T”, female, 49, FG1)

“It is good to receive some information about what the medications do, because sometimes I received information that the GP didn’t tell me about, which was really good” (“T”, female, 49, FG1)

“The messages gave me some insight into new things that I had never thought of, trying different foods” (“G”, male, 58, FG2)

Although the program was a one-way text message communication system, qualitative data suggest that Social-Cognitive Theory explains some of the observed behavior change. Social cognitive theory[32] suggests that human behavior is determined by a combination of cognitive, personal and environmental influences. These reflect an interaction between thought, affect and action where expectations, beliefs, self- perceptions, goals and intentions give shape and direction to behavior.[33] Further, behaviour is modified based on whether the individual has high or low self-efficacy toward the behavior, the response a person receives after performing the behavior and aspects of the environment that improve self-efficacy including the setting of achievable goals. In the current text messaging program, social recognition, improved self-efficacy, the provision of achievable task-setting suggestions, practical advice and positive reinforcement were all built into the program[27] and quotes illustrating their successful implementation are below.

“I found the program encouraging” (“B”, female, 60, FG1)

“It gives a good feeling just knowing that there is someone else who cares and kept an eye on it/you like a good link” (“K”, male, 58, FG1)

“It made me feel good when sometimes I already did my exercise and then got a message about it. And then I was like ‘Yes, beat you’” (“B”, female, 60, FG1)

Operant and its predecessor Classical Conditioning are theoretical frameworks[34,35] for explaining behavior and this concept of ‘respondent condition’ appeared to emerge from the focus groups where participants responded to the repeated messaging within the program as a whole rather than the immediate content of individual messages. This type of conditioning was first reported by Ivan Pavlov where he demonstrated that dogs learned a stimulus-response connection in response to repeated stimuli such that a previously neutral stimulus eventually elicits a response.[35] Our qualitative data suggest that over time participants became ‘conditioned’ towards healthy behaviors irrespective of the specific content of the individual messages received at a certain time. That is, through repeated presentation of a stimulus (in this case cardiovascular health related text messages) the participants eventually learned to associate the stimulus with overall cardiovascular health behaviour and consequently messages about diet also resulting in improved physical activity and vice versa. The notion that the program evoked a feeling of feeling “guilty” could also contribute to conditioning where the feeling of guilt is a noxious stimuli associated with ‘unhealthy’ behaviour. Illustrative quotes highlighting this concept include;

“It wasn’t particularly about what the message said that day” (“C”, female, 51, FG2)

“They always popped up before I was about to eat something naughty” (“W”, male, 68, FG1)

“I don’t actually go for context of the message, but look for the little nudge. The little things that nudge you to do the right things” (“D”, male, 65, FG 2)

“When you see the text message you might be a little bit more conscious about what you are eating” (“T”, male, 62, FG4)

“If I didn’t walk for the past 3 or 4 nights, because it was too hot or raining, you made a bit more effort to do it, you felt a bit guilty. And the same with checking your cholesterol and about what you are eating, think about the portions etc. It is an on-going thing, a general thing. But the reminder or the nag that was the key” (“S”, male, 76, FG2)

“I love chocolate and one time I was in the aisle of [shop name] thinking about getting a snack… then suddenly I received a message of TEXT ME saying: Hi [name], feeling like a snack? A banana or a piece of fruit! It really gave a big brother feeling to me, how could they know I was thinking about picking a snack? I thought well forget it lets go and walked away with a smile on my face. I got busted” (“B”, female, 60, FG1)