Staff.

We aimed to recruit five research nurses, based on the literature and our experience of conducting qualitative health research. We revised this target to 15–20 participants (expanded to include different roles)–this was in response to informal feedback and conversations with sites. We elected to recruit more staff members compared to families because they played a key role in implementing the algorithm recommendations. We also considered that there was a diversity and richness of experiences and insights to be gleaned from staff at different sites and performing diverse roles.

Email invites were distributed to a purposive sample of members within research teams who occupied various roles in the trial (e.g., Research Nurses, Consultants and Principal Investigators) across ten different UK sites. These sites were selected to obtain interview data from staff who worked with different levels of recruitment to the RAACENO trial (smaller vs. larger numbers) and based on trends that demonstrated the sites associated with low/high adherence to algorithm recommendations. However, due to the COVID-19 pandemic and the resultant low uptake of individuals responding to our initial invite at the time, we elected to adopt a convenience sampling method to improve recruitment figures. Reminder invites were subsequently distributed and staff at three additional sites were approached. Interviews with trial staff who agreed to participate were arranged and conducted by a researcher (LL) who was not known to the participants prior to data collection.

Fifteen trial staff members were included in the final sample size. An additional three individuals had initially expressed interest in participating but 1 was unavailable due to the pressures of the pandemic and 2 did not reply to further contacts. Data saturation became evident following the first 6 interviews [23], but data collection continued to ensure we obtained a more representative sample of staff occupying various roles within the trial and at different sites [22, 24]. The researcher introduced herself as a qualitative researcher with no involvement in clinical data collection. Interviews were conducted via telephone, audio-recorded and transcribed by an external transcription company. Field notes were generated to facilitate data analysis. Participants were interviewed from March 2020 till June 2020.

Families.

We set out to interview 20 families drawn from both arms of the trial. Families were first approached and invited by research nurses across seven trial sites. Site selection was initially based on recruitment levels at sites and the convenience of accessing the hospitals for face-to-face interviews based on geographical location (prior to the requirement for remote working due to COVID-19). Thereafter, potential interviewees identified by research nurses as interested in being interviewed were followed up by one researcher (HMM) by telephone during the study and then approached again by the research nurses at their 6-, 9- or 12-month assessments to ask whether they would still be happy to be interviewed following their 9-month (penultimate) or 12-month (final) assessment (see S1 Appendix for further information regarding participant follow-up procedures). Of the 17 families who initially expressed interest in participating, 6 families (mother-child pairs) were interviewed between March 2020 till May 2020. The remainder of those who initially expressed an interest in taking part did not reply to further invitations. Interviews were conducted by a researcher (LL) who had no previous correspondence with the families and who introduced herself as a qualitative researcher independent from the clinical team involved in the data collection and analysis for the host trial. Interviews were conducted via telephone, audio-recorded and transcribed by an external transcription company. Field notes were generated to facilitate data analysis. Recruitment continued for both trial staff and families until data saturation was reached.

Smart inhalers.

Technological challenges. Overall, smart inhalers were perceived to be generally appreciated, but with some technical problems relating to recording/uploading data (9RN1, 8RN1, 7RN1, 6CT2, 5RN1, 4RN1, 3CT1). Accuracy of the smart inhaler data indicating patient usage was questioned by one staff interviewee (3CT1). There was disappointment if a score had to be disregarded:

“In the beginning a few seemed to work, but towards the end I was disappointed not to be able to show the parents good results if they said their child was being compliant…So we had to you know, to ignore the [adherence] score even though the child… the child wanted to beat their last score and it could be disappointing…” (2RN1).

Monitoring adherence. Smart inhalers were considered good for monitoring adherence (8RN1) and providing an objective index of adherence (6RN1), as well as for setting targets:

“… The ones that were downloaded, they were really happy with them and it gave them a focus. One of my early recruits, hers always worked amazing and her mum was really good at encouraging her and saying, “Right, we want the number higher next time, come on, we want better there,” you know? So really used it as a tool to encourage them…” (5RN1)

Staff believed that adherence improved (6CT2) and thought that smart inhalers for non-adherent children (9RN1) were ‘a bit of a game’ with tech to monitor adherence (7RN1) and to manage some patients (6RN3), although it was acknowledged that some would probably not take their inhalers properly even if monitored (6RN3). For example:

“What you don’t want to be doing is to continually escalate doses down an algorithm when in fact the reality is, is they’re not taking their medicine at the required frequency.” (4CT2)

Some families were reportedly excited about seeing the results and their performance (6RN1), but it was identified as a potential reason for drop out of children who were not adherent (7RN1). However, even when adherence was low, families did not seem to mind being monitored (6CT2), although one clinician mentioned the ‘big brother’ factor:

“I think there is a big brother factor, they get a bit anxious about us scrutinising how well they take their medicines. But having something that brings it to the forefront so that you can have an honest discussion I think is very, very valuable.” (4CT2)

Enabling communication. The data were useful for facilitating ‘difficult conversations’, for example, helping to have conversations and improve adherence to see asthma under better control (3CT1); discussing when adherence had drifted off, e.g., during holidays (8RN1); and uncovering when a child had not been taking medication with parents assuming that they were (7RN1). Improved adherence of some was reported initially, but this was not always sustained (6RN3, 4CT2), although the smart inhalers could be used as an incentive to obtain a score (5RN1, 4CT2, 2RN1). For example:

“I think the children enjoyed that game you know, they always tried to guess what it was going to be and then they tried to beat their score the next time, and they understood how to achieve that and what the significance that inflammation can mean in combination with their lung function test. It helped them to understand it a little bit better and the importance of using their inhaler as per what the doctor said.” (2RN1)

Staff perceived that parents wanted access to their child’s data so that they could monitor them (8RN1) and suggested that it would be better if there was an opportunity for instant download at home rather than in clinic (2RN1). Factors that were concerning staff around data capture, also highlighted above in terms of missing data affecting the algorithm’s recommendation, were: weekends when children go to the other parent and forget the smart device (6CT2, 2RN1) and cut offs/timings automated within the device:

“… So, if you’ve got a teenager that’s getting up at 1.00 in the afternoon, they’ll get up, they’ll take their inhaler and then they’ll go and do whatever, and then they’ll take it again, but that was saying “No” for the morning dose because it had a cut off of 12.00…” (6CT2)

Otherwise, positive experiences were reported:

“I think it’s quite nifty, it’s not heavy, it’s not bulky, it’s quite easy. So, I think yeah, it’s not too bad to use, it’s quite easy, yeah.” (6CT2)

And being able to check on other aspects was also considered as helpful, e.g., expiry date (8RN1).

Familiarity with algorithms—staff perspectives.

There were a range of considerations regarding experiences and acceptability of the algorithm’s role in the diagnostic and decision-making process. Familiarity with using algorithms in treatment was a factor: one staff member said that they had never used algorithms before to determine treatment (4RN1). Another expressed a very clear interpretation of what should happen, suggesting that a ‘purist’ would follow the algorithm to the letter (9RN1). Staff seemed to express an open mind to using such an approach in clinical practice. But there were some caveats, for example, as long as it was considered ’reasonable’ (8RN1) or ‘appropriate’ (6CT2), staff said they would accept the recommendation, however, if not, staff explained how they would use it:

“If you were stepping down as you went into the school return in September it made you more anxious if you knew that in historical years that they’d had asthma attacks at that time. So that did influence a minority of patients in terms of whether the algorithm was followed or not.” (4CT2)

Concerns regarding step down recommendations. There was concern if a step-down was recommended in the context of continuing emergency department visits and needing to look more closely (1CT1) or causing emergency department visits through a step-down that led to exacerbation (6CT2). The use of the algorithm was considered to be beneficial when a step down meant that a patient could be returned to community care rather than being managed in secondary care (7RN1). For this participant, the algorithm recommendations provided additional evidence which harmonised with their clinical judgement. Internal ‘missing data’ problems and external contextual factors were also considered, for example one staff interviewee said that there may be a problem if the algorithm was not operating on a full dataset due to missing data (6RN1), also highlighted elsewhere where data were missing because the child was staying with their other parent and forgetting their smart inhaler (6CT2).

Using clinical judgement. There were perceptions that people–both staff and families–were not accepting of ‘technology’ on its own:

“I think most families who were accepting of the algorithm but only with the proviso that it’s never just the technology and the algorithm, and the doctor can always override that if necessary.” (2RN2)

“Reassuring them that the computer’s not dictating to them what’s going to happen, that the doctor will overall decide what’s safe for them.” (2RN1)

Clinician judgement was considered to be critical to applying the algorithm’s recommendation (6RN1) and the notion that understanding among trial participants that the algorithm recommends, and the clinician decides, was highlighted (7RN1). Trust in the consultant and families accepting clinical judgement, whether or not this involved adhering with the algorithm (9RN1), was raised. The notion of ‘computer says no’ and that algorithms are pretty limited was raised, alongside the idea that patients and their families come to see an expert in whom they trust and with whom they have a personal relationship, which cannot be replaced by a computer and algorithm (3CT1). Staff perceived that parents were sometimes reluctant to go ‘too low’ or alarmed about ‘how high’ (8RN1) the algorithm-recommended treatment, and if a parent was unhappy about a step down, staff believed that their feelings needed to be taken into account (6CT2). One staff member indicated:

“Yes, actually fairly frequently, we haven’t agreed with the algorithm. Well, for various reasons. Probably more often that parents have felt a bit more cautious when a step–down is suggested.” (2RN1)

Research nurses checking adherence with the algorithm rather than parents was noted by one site (1CT1) and not using the word ‘algorithm’ with families was highlighted at another (7RN1). Ultimately, providing the best care was the main factor:

“To them it was just like, at the end of the day, we’re going to give you the best care we can and the best treatment” (5RN1)

In terms of rates of agreement with algorithm recommendations, sites indicated that they had different experiences. For example, at one site, there was an estimate of disagreeing about 20% of the time due to the time of year and triggers (apprehensive about stepping down, 5RN1), whereas at another site, it was suggested that there was disagreement about half of the time (4RN1). One site perceived that the algorithm was ‘chopping and changing’ and preferred more settled treatment (9RN1), whereas another site considered that there were ‘too big jumps’ (8RN1). However, sites expressed satisfaction around obtaining data:

“One of our consultants, in particular, struggled with that a little bit, because you could have a child that would have… so step–downs were frequently suggested and when actually their lung function was decreasing, with those patients we had to override the algorithm.” (2RN1)

The need for GPs to have a better tool for use in primary care, i.e., using the algorithm, was noted (1CT1).

Smart inhalers.

Technological challenges. Smart inhalers were considered to be useful, but many encountered problems with their accuracy (Family 1, Family 2) and reported that the battery required regular charging (Family 4, Family 3). One parent said that it was difficult to use, even for adults:

“…Obviously from my experience, I work as support worker and when I give my residents the smart inhaler, they don’t know what to do with it, so some… even adult people don’t know what to do with this…” (Family 1)

There was a suggestion of a battery indicator being helpful (Family 5) and also that children are more tech savvy and would manage charging better than their parents (Family 4). One parent suggested having the option to download at home, perhaps weekly, to monitor their child would have been useful (Family 3). Another parent wanted to buy a smart inhaler:

“…As a parent, it’s been unquestionably my favourite thing about the study. I really echo what she said, I wish that we could buy it. I mean, I said I would buy it for somebody else in the study that maybe couldn’t afford it, I have no idea how much they are, but I know they said it would be difficult to get the software or something on a phone…” (Family 6)

Motivation and accountability for medication adherence. Families (parents/children) reported that they liked being monitored (Family 4) and the computer being able to account for adherence to generate recommendations after technical issues were fixed. Most families specifically mentioned that the monitoring was a motivator to be adherent with treatment, even when the asthma was well managed (Family 5, Family 4, Family 2, Family 3, Family 6). One parent said:

“… in my opinion it makes the child a little bit more conscious about making sure they take it knowing that it’s being recorded…” (Family 4)

Two parents described that a child feeling more responsible was a positive consequence of smart inhaler usage (Family 5, Family 6). Smart inhalers encouraged patients to adopt a degree of accountability in terms of their medication adherence. One parent said that their child had learned to manage their asthma better due to their study participation (Family 5).

Algorithm.

Perspectives on usefulness. One parent felt that the algorithm made good decisions (Family 5), and another mostly agreed with both the step-ups and step-downs in preventer treatment (Family 4): this was linked to their perception about how the condition was experienced by their child at the time. Another liked the close monitoring with recommendations about stepping up and down and ‘being on the right amount’:

“No, the doctor always just did what the computer recommended. But he always explained it and always said, you know, he always kind of justified it because her asthma had been fine, or because, you know, she hadn’t been on steroids for the previous three months or whatever. So, he always talked it through.” (Family 3)

The fluctuating nature of asthma symptoms (Family 4, Family 3, Family 6) were identified by parents as being relevant to how the algorithm was used. It was noted that the use of the algorithm required flexibility based on the family/patient lifestyle. For example, one family did not adhere to the treatment recommendations for a month, after consulting a clinician, because of travel plans to an area of poor air quality (Family 3). One parent cited that they respected decisions based on the algorithm because they trusted the clinical staff:

“It’s a programme that works out information. I don’t know anything about that, very technology behind, but yeah, they did explain it to me. I just trusted that they knew what they were doing.” (Family 2)

Some parents remembered the word ‘algorithm’ (Family 5) and one said that it was communication on multiple occasions (Family 4), whereas others said they did not remember the word, but felt that the language used by staff to describe the trial interventions was appropriate to their level of understanding (Family 3). In response to questions designed to elicit perceptions of the role of the algorithm in treatment decisions, one said it was initially daunting, but that this seemed acceptable for making recommendations about treatment and was described as ‘clever’:

“…the computer seemed to pick up on all of that, which I think was…I think it was really, really clever. But obviously having… knowing that there was a doctor overseeing it as well was that little bit of reassurance.” (Family 4)

Most parents reported that the algorithm had suggested changes, a step-up/down from current treatment recommendations, on one to three occasions (Family 1). Some liked the fact that a step-down was recommended as this meant taking less medication (or had been taking more than required) (Family 1), but others did not and were anxious about a step-down, even though they also wanted less steroid (Family 6), or a maintained level of treatment based on discussion between parents and clinicians, even where the algorithm suggested no treatment (Family 5). One parent considered algorithm recommendations to be reliable as they seemed to recommend a step-up when the asthma had felt bad but suggested that it would have never recommended no treatment (no medication) and as such was reliable:

“… She did have a bad asthma year the year of the study, but not particularly worse than it would have been anyway, I don’t think. She’s just got pretty bad asthma. But I liked that they were very responsive to everything.” (Family 3)

Enabling communication. Regarding a step-up decision, one parent liked to be able to have a conversation about the computer recommendation and was reassured that their clinician would not follow the algorithm if everyone perceived a step-up was too big a step (Family 4). Another felt similarly confident that, should they want to override it, for example because the child was not keen due to increased usage of steroids [and the perceived consequence of reduced growth], their consultant would step in:

“… she was really paranoid about the more steroids that she took, the shorter she would end up. So, every time the computer said she needed to increase her steroid dose she said, ‘Oh I don’t think so’.” (Family 2)

Comfort in clinician judgements. The majority of trial participants’ parents reported feeling comforted by the idea that they could ultimately depend on a doctor’s interpretation and overriding the algorithm’s recommendation based on knowledge of the case and adapting to season (Family 1, Family 5), or taking into account a previous bad reaction (Family 6). The fact that the algorithm was being used as a guide and not on its own (Family 5, Family 4, Family 6) was identified as important in the acceptability of the algorithm.

“…I’m fine with technology helping advise the doctors, but ultimately the doctor made the call, using the technology and the finding.” (Family 6)