Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Complex intervention programme to improve patient safety and facilitate deprescribing in frail older patients living at home (COFRAIL): A process evaluation of a cluster randomised controlled trial

  • Jens Abraham ,

    Contributed equally to this work with: Jens Abraham, Steffen Fleischer

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing

    jens.abraham@medizin.uni-halle.de

    Affiliation Institute of Health, Midwifery and Nursing Science, Medical Faculty of Martin Luther University Halle-Wittenberg, Halle (Saale), Germany

    ⨯
  • Steffen Fleischer ,

    Contributed equally to this work with: Jens Abraham, Steffen Fleischer

    Roles Conceptualization, Formal analysis, Funding acquisition, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Institute of Health, Midwifery and Nursing Science, Medical Faculty of Martin Luther University Halle-Wittenberg, Halle (Saale), Germany

    ⨯
  • Achim Mortsiefer,

    Roles Funding acquisition, Project administration, Writing – review & editing

    Affiliation Faculty of Health, Department of Medicine, Institute of General Practice and Primary Care, Witten/Herdecke University, Witten, Germany

    ⨯
  • Eva Drewelow,

    Roles Writing – review & editing

    Affiliation Clinic for Forensic Psychiatry, University Medical Centre Rostock, Rostock, Germany

    ⨯
  • Manuela Ritzke,

    Roles Writing – review & editing

    Affiliation Institute of General Practice, University Medical Centre Rostock, Rostock, Germany

    ⨯
  • Birgitt Wiese,

    Roles Writing – review & editing

    Affiliation WG Medical Statistics and IT-Infrastructure, Institute of General Practice, Hannover Medical School, Hannover, Germany

    ⨯
  • Charalabos Markos Dintsios,

    Roles Writing – review & editing

    Affiliation Faculty of Medicine, Institute for Health Services Research and Health Economics, Centre for Health and Society, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany

    ⨯
  • Veronika Bencheva,

    Roles Writing – review & editing

    Affiliations Faculty of Health, Center for Clinical Trials, Witten/Herdecke University, Witten, Germany, Chair of Clinical Pharmacology, Department of Medicine, Faculty of Health, Witten/Herdecke University, Witten, Germany

    ⨯
  • Petra Thürmann,

    Roles Funding acquisition, Writing – review & editing

    Affiliations Chair of Clinical Pharmacology, Department of Medicine, Faculty of Health, Witten/Herdecke University, Witten, Germany, Philipp Klee-Institute of Clinical Pharmacology, Helios University Hospital Wuppertal, Wuppertal, Germany

    ⨯
  • Stefan Wilm,

    Roles Funding acquisition, Writing – review & editing

    Affiliation Institute of General Practice, Medical Faculty, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany

    ⨯
  • Gabriele Meyer,

    Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

    Affiliation Institute of Health, Midwifery and Nursing Science, Medical Faculty of Martin Luther University Halle-Wittenberg, Halle (Saale), Germany

    ⨯
  • on behalf of the COFRAIL study group

    Members of the COFRAIL study group are mentioned in the Acknowledgments.

    ⨯

Abstract

Introduction

Frailty is associated with negative health outcomes in geriatric patients. A large proportion of frail patients is affected by polypharmacy, which in turn may be a possible cause of frailty. The cluster randomised controlled COFRAIL trial investigated the effects of family conferences to improve the care of frail patients. Deprescribing and communication about prioritising health goals between patients, relatives and general practitioners were key components of the intervention. An accompanying process evaluation was conducted to investigate whether the study intervention was implemented as intended and to describe the experiences of the target groups.

Methods

The process evaluation took place between February 2019 and October 2021 and followed international guidelines. Process parameters were collected using study documentation, standardised questionnaires and guided telephone interviews with a convenience sample of patients, relatives and general practitioners. Quantitative data were analysed descriptively, qualitative data through content analysis.

Results

Almost all general practitioners in the intervention group completed both mandatory trainings. Overall, 68% of the patients took part in all three planned family conferences, 85% in at least two. Patients, relatives and general practitioners reported positive experiences with the family conferences. Patients and relatives felt involved in the decision-making process and had predominantly no concerns when medication was discontinued. Only a few general practitioners considered the implementation of family conferences in regular care to be impractical. A supportive pharmacological hotline was not utilised frequently by the general practitioners. Due to the SARS-CoV-2 pandemic, some study procedures were adapted but did not have any negative impact.

Conclusions

While the COFRAIL study did not achieve its primary goals, we identified no major barriers in the implementation of the intervention programme. Overall, the target groups experienced the approach of family conferences positively. However, the process evaluation provides valuable lessons for designing and implementing similar interventions in the future.

Introduction

The frailty syndrome is a complex problematic health condition that is common in the older population. It is characterised by decreased physiological reserves and vulnerability to stressors [1–3]. Data on the prevalence of frailty vary widely, as studies are based on different operationalisations of frailty status [4]. A recent meta-analysis [5] with 57 studies and a total of 56,407 older people (mean age 78.6 years) found an overall prevalence of 26.8% and of 13.3% in community-based studies using a multidimensional approach for the definition of frailty (Multidimensional Prognostic Index (MPI) [6]).

An increased risk of falls, delirium, malnutrition, adverse drug events, inappropriate prescribing, hospitalisation as well as mortality is associated with this geriatric syndrome, especially in the context of polypharmacy [3,7–10]. A large proportion of older people is affected by polypharmacy, which in turn may cause or exacerbate frailty [1,11]. There is no consented definition of polypharmacy and a review identified 138 different definitions [12]. Most often, polypharmacy is defined as taking five or more drugs daily. The prevalence varies widely, not only depending on definitions that have been applied, but also due to the case mix of the population in terms of chronic conditions, demographic characteristics and the underlying setting [13,14].

The two-arm cluster randomised controlled trial (COFRAIL) [15,16] (trial registration: DRKS00015055) with a total of 113 general practitioners (GPs) in 110 practices (clusters) and 521 older, frail patients investigated the effectiveness of family conferences as an approach to improve the care of this target group. Identification of polypharmacy with jointly considered discontinuation of drugs (deprescribing) [17] and communication about prioritisation of health goals between patients, their relatives and/or the ambulatory care service and their GPs during family conferences were main components of this complex intervention. The deprescribing process is inherently complex and thus requires more effective communication between all involved interest-holders [18,19]. In community-dwelling older patients, family members frequently play a key role in managing medications [20–23]. Therefore, strategies aimed at deprescribing should be incorporated into a shared decision-making framework which actively involves GPs, patients and family members. Family conferences are a well-established instrument to improve communication in various healthcare settings [24–30]; their impact on drug safety in frail, older patients receiving home care was investigated for the first time in the COFRAIL study [15,16].

The GPs in the intervention group received a training programme over a period of six months. Two mandatory training sessions and a third optional training were provided. The training focused particularly on the topics of deprescribing and non-pharmacologic interventions, and included communication training to prepare the GPs for the structured family conferences. The patients in the intervention group, along with their family carers and/or the home care service, were offered three family conferences over a period of nine months, conducted either in the patient’s home or at the GP’s practice. The main focus of the family conferences was to review the current medication and deprescribe drugs if indicated, supported by a deprescribing manual [31], and, if needed, to consider further non-pharmacological interventions from a toolbox [32]. Furthermore, the GPs had the opportunity to get counselling from an external clinical pharmacologist/pharmacist via phone or email and to ask clinical and pharmaceutical questions regarding their study patients (pharmacological hotline). The participants of the control group received care as usual.

The COFRAIL intervention led to a significant decrease in the overall number of medications and the number of potentially inappropriate medications (PIM) after six months but did not reduce the number of hospitalisations, the number of medications, or the number of PIM compared to usual care after 12 months. The report on the effectiveness study and details about the development and content of the COFRAIL intervention including the underlying theoretical rationale are published elsewhere [15,16,32].

Due to the complex nature of the intervention and the study setting, a comprehensive process evaluation was carried out as recommended by the UK Medical Research Council’s (MRC) guidance on the development and evaluation of complex interventions [33]. According to the MRC, a strictly quantitative approach that relies solely on an experimental design, without incorporating elements like process evaluation, is rarely sufficient for researching complex interventions. A process evaluation enables understanding of fidelity and quality of an intervention’s implementation, mechanisms of change, and context. Moreover, it provides insights into why an intervention may fail unexpectedly and lead to unintended effects, or how the intervention might be improved or refined [33]. This paper reports on the process evaluation conducted alongside the COFRAIL study [15,16] and aims to evaluate the implementation of the intervention, including barriers and facilitators, as well as to explore the experiences of the participating target groups – patients, relatives and GPs.

Materials and methods

Design

The process evaluation was designed based on the recommendations for process evaluations for cluster randomised trials of complex interventions proposed by Grant et al. [34]. Different process parameters were collected at the cluster and individual level using various qualitative and quantitative methods. The process evaluation was carried out between 01st February 2019 and 31st October 2021.

Setting

This study was performed in a primary care setting in two areas of Germany (Düsseldorf (North Rhine-Westphalia) and Rostock (Mecklenburg-Western Pomerania)). The process evaluation involved the participating GPs, patients and relatives of the intervention group.

Data collection methods

To investigate the feasibility and optimise the intervention, semi-structured telephone interviews were conducted with GPs, patients and relatives after piloting the family conferences. The piloting has been reported elsewhere [32].

The recruitment process, including information provided on reasons for non-participation or drop-out, was documented at both the cluster and individual level. The results have been published together with the effectiveness data [15]. To determine intervention fidelity, structured documentation forms were completed by instructors after each mandatory educational session. Furthermore, the mandatory educational sessions were evaluated by participants after the second session using a standardised questionnaire (satisfaction with the training, attitudes towards deprescribing, acceptance of intervention, self-efficacy, expectations; rated by 4-point Likert scales (1 = strong agreement to 4 = strong disagreement)). For the third optional educational session, only the utilisation was documented. In addition, the family conferences were documented by the GPs using a semi-structured protocol immediately after each family conference. The utilisation of the individual medication reviews provided by a pharmacologist/pharmacist was also documented.

At the end of the study, the attitudes of GPs towards the COFRAIL intervention and the experiences, including barriers and facilitators, were assessed in guided telephone interviews. Guided telephone interviews were also conducted with patients and their relatives regarding families’ experiences (e.g., consideration of preferences, changes in physician-patient-communication, barriers and facilitating factors). The interviews were conducted by researchers not involved in the main study, and who had at least a Master`s degree in health and nursing science, and extensive experience in qualitative data collection. Both interview guides were pretested during the piloting phase of the COFRAIL study. The interview guides for the telephone interviews with GPs, patients and relatives are provided in additional S1 and S2 Appendix. Fig 1 illustrates the timeline of the process evaluation procedures. A comprehensive overview of the focus, methods and target groups of the process evaluation can be found in additional file S3 Appendix.

thumbnail
Fig 1. Timeline of the process evaluation procedures.

Measurement points: t0=baseline COFRAIL study, t1=after 6 months, t2=after 12 months. Abbreviations: IG=intervention group; GP=general practitioner.

https://doi.org/10.1371/journal.pone.0350664.g001

Sample

For the evaluation of the mandatory educational sessions, all participating GPs were asked to complete the evaluation questionnaires after the second session. The guideline-based telephone interviews with GPs at the end of the study were conducted with a convenience sample (all GPs willing to participate were included, with no specific sampling criteria applied) of at least ten GPs per region. At the end of the mandatory training sessions, the GPs were asked about potential interest in further interviews and were later invited to participate. No incentives were offered.

Initially, it was planned to conduct guideline-based telephone interviews with a convenience subsample (interested to participate, with no specific sampling criteria applied) of ten patient-relative-dyads per region. Since the relatives could not be reached through patients in many cases or joint interview appointments with relatives and patients were not feasible, the telephone interviews were mostly conducted separately with either patients or relatives. For this purpose, the study nurses at the two study centres in Düsseldorf and Rostock inquired about potential interest among eligible patients and relatives during the main data collection and, if necessary, sent the contact data to the study centre in Halle (Saale) for process evaluation. Subsequently, a written invitation to participate in the interview was sent out. Participants had to be physically and mentally able to take part in a telephone interview. An exclusion criterion of the COFRAIL study and therefore of its process evaluation was the inadequate German language proficiency of the patient and family caregivers, or the unavailability of an interpreter. There were no further inclusion or exclusion criteria.

Ethics

The COFRAIL study, including the process evaluation, was approved by the ethics committees of the study centres Rostock (no. A2018-0151) and Düsseldorf (no. 2018−283). Written informed consent was obtained from all interview participants. Participants in the evaluation of the educational programme gave their consent by returning the questionnaires.

Data analysis

The process evaluation was exploratory. The quantitative data were analysed descriptively, calculating mean values, standard deviation, minimum – maximum, absolute and relative frequencies. The qualitative data were analysed according to Maying’s qualitative content analysis [35]. All interviews were digitally audio recorded and transcribed verbatim based on the transcription rules by Dresing & Pehl [36]. The initial themes were coded deductively by one person (JA) using a set of categories derived from the research question and the interview guides applied to each interview. Based on the interview material, additional themes were developed inductively. Finally, findings were discussed and interpreted with another member of the study team (SF). The contents were summarised and narratively described. MAXQDA software (version 2020) was used for the analysis. The interviews were held in German, and participants’ quotes were translated into English by the authors (JA & SF) and double-checked by a native speaker.

Results

Implementation of educational programme

A total of 26 first training sessions, 21 second training sessions -both mandatory- and three third training sessions (optional) were provided. The first training lasted on average 137 minutes (median: 120 minutes; SD: 42.9; min: 70 minutes; max: 210 minutes), the second training 103 minutes (median: 90 minutes; SD: 31.6; min: 60 minutes; max: 195 minutes). Data on the duration of the third training is not available.

All 59 GPs (from 56 intervention practices) attended the first training. Only one GP was unable to take part in the second training session, but all the training materials were provided. A total of 26 GPs participated in optional training session three. Due to the SARS-CoV-2 pandemic, all training sessions were held online in groups or individually by telephone from March 2020 onwards (training session 1: n = 8; training session 2: n = 14; training session 3: n = 3).

The training sessions were mainly conducted according to the training manual. Deviations occurred in particular due to the pandemic-related changeover to telephone training or the online format, when for example, role plays or group work could not be carried out.

Evaluation of the educational programme

A total of 50 GPs completed the training evaluation after the second mandatory training session. Overall, the training was positively evaluated by the participants, both for presence sessions as well as the telephone/online training. The content was deemed relevant by almost all participants and the professional level was appropriate. Most GPs felt well prepared to conduct the family conferences.

Involving relatives in shared decision-making was considered important by most GPs but deemed even more important for patients. In addition, there were still GPs who were afraid that deprescribing would be associated with negative consequences for patients and could lead to conflicts with other specialists (see Fig 2).

thumbnail
Fig 2. Evaluation of the educational programme by GPs.

*Only valid answers counted. The overall evaluation of the training (analogous to German school grades, one to six, one being the best) was rated with a mean (SD) of 1.5 (0.5).

https://doi.org/10.1371/journal.pone.0350664.g002

Implementation of the family conferences

The first family conference was conducted in July 2019, the last in May 2021 by the 59 GPs. Family conferences were conducted with a total of 676 patients (first family conference: n = 244, second family conference: n = 232, third family conference: n = 200). Of the 272 patients in the intervention group, 185 patients (68%) received all three planned family conferences, 46 patients (17%) received a total of two family conferences, and 29 patients (11%) received at least one family conference. Twelve patients (4%) did not participate in a family conference at any time. The first family conference lasted on average 37 minutes (median: 30 minutes; SD: 15.57), the second family conference lasted 35 minutes (median: 30 minutes; SD: 12.94), and the third family conference lasted 32 minutes (median: 30 minutes; SD: 14.70). The duration of the family conferences ranged widely between ten and 120 minutes.

A total of 342 family conferences were conducted in the patients’ homes, 213 family conferences took place in GP practices, and 80 family conferences were held via telephone or videoconference (information is missing for a total of n = 41 family conferences).

In 410 (60.7%) family conferences at least one relative participated, both family members and representatives from the home care service attended 115 (17.0%) family conferences and in 29 (4.2%) family conferences only representatives from the home care service were involved. In a total of 120 (17.7%) family conferences, neither a family member nor a representative of the home care service participated (n = 2 missings).

Table 1 shows the medical and non-pharmacological measures and recommendations resulting from family conferences documented by the GPs. This comprised the adjustment of medication (including, e.g., dose reduction, prescription, discontinuing and change of medication) and other medical treatments or diagnostics like referrals to other specialists, flu vaccination or blood tests. Further measures and recommendations included, among others, the prescription of therapies (physiotherapy, logopaedics, or occupational therapy), advisory consultations (e.g., regarding other treatment options), encouragement of increased mobility, and the prescription of technical/care aids (e.g., walker). In the first family conference, no measures or recommendations (apart from adjustment of medication) were agreed upon for 103 patients (42%), in the second family conference this applied to 66 patients (32%), and in the third family conference to 62 patients (31%).

thumbnail
Table 1. Medical and non-pharmacological measures and recommendations resulting from family conferences.

https://doi.org/10.1371/journal.pone.0350664.t001

Use of the pharmacological hotline

The GPs in the intervention group had the opportunity to consult an external clinical pharmacologist/pharmacist via phone or email. This was utilised eight times during the course of the study. The questions were very heterogeneous and predominantly specific to individual patients.

Experiences of patients and relatives

A subsample of 24 patients and 18 relatives participated in the telephone interviews. The mean duration of the interviews was 22 minutes (min: 8 minutes; max: 54 minutes). The participants’ characteristics are presented in Table 2.

thumbnail
Table 2. Characteristics of patients and relatives.

https://doi.org/10.1371/journal.pone.0350664.t002

General experiences with family conferences.

The experiences of patients and relatives regarding the family conferences were largely positive. This was associated with a generally good relationship with the GP. A particularly positive aspect of the family conferences was that the GP took plenty of time to discuss all questions and concerns in the patient’s home.

Yes, I thought that was good. It wasn’t bad because you could also talk about other problems and so on. I mean, when you go to the GP, for instance, they often don’t have that much time to talk about any other problems. (Interview 5 (patient): pos. 42)

Both patients and relatives felt they had been sufficiently involved in the joint discussion with the GP during the family conferences.

Consideration of the preferences.

The adjustment of medication was perceived by the patients and relatives as the main focus of the family conferences, in addition to the general health and care situation. If indicated, drugs could be reduced or discontinued. In some cases, the drugs had to be restarted after patient follow-up, or no changes were made. According to the patients and relatives interviewed, they were able to contribute sufficiently to the joint discussion and the preferences were taken into account in the decision-making process. But it also became clear that patients and relatives were not often able to judge the recommendation to discontinue or reduce medication from a professional point of view and relied on the judgement of the GP.

[…] because I always had the feeling that A) I can always ask, and B) I can also influence things. And if I had said it was too risky for me, I don’t know if he [GP] would have got his way, but I didn’t have that feeling at all. (Interview 26 (relative): pos. 39)

However, some of the patients and relatives interviewed also reported concerns about the discontinuation of medication.

[…] I also felt a bit helpless, I have to say, because she [GP] said we were cancelling all that, she [patient] didn’t need it any more. And I had the feeling that it was for cost reasons or something, that it was all cancelled now, and I didn’t know really know the context or couldn’t make sense of it […]. (Interview 14 (relative): pos. 46)

In one case, a drug was continued by the patient not adhering to the agreement with the GP.

Lacking awareness of the intervention.

For some of the participants, the family conferences did not differ from usual home visits by GPs.

It was so normal, like always, she [GP] asked, and I answered […]. (Interview 4 (patient): pos. 40)

Sometimes it was apparent that the family conference itself was not perceived as an intervention, but rather the data collection as part of the COFRAIL study.

Pandemic-related influences.

For some participants, the family conferences took place after the onset of the SARS-CoV-2 pandemic, but neither patients nor relatives experienced any negative effects in terms of implementation.

[…] but the doctor still kept coming to visit me at home. So it all worked out wonderfully. (Interview 5 (patient): pos. 67)

Experiences of general practitioners

Telephone interviews were conducted with a subsample of 23 GPs out of a total of 59 GPs in the intervention group. The interviews lasted on average 16 minutes, with the shortest being eight minutes and the longest being 34 minutes. Table 3 provides the descriptive characteristics of the participants.

thumbnail
Table 3. Characteristics of general practitioners.

https://doi.org/10.1371/journal.pone.0350664.t003

General experiences with family conferences.

The GPs also reported predominantly good experiences with family conferences. From their perspective, the patients perceived it as particularly positive that they were cared for more intensively. Direct communication with the patients and their relatives was perceived as very helpful.

Yes, it was definitely a good appointment to have a conversation with the family without there being an acute crisis or anything like that. Which is something that doesn’t necessarily happen in normal therapy, so I think both sides definitely benefited from having a general discussion. (GP 3: pos. 12)

Involvement of patients and relatives.

The GPs reported that patients and relatives were mostly involved in the decision-making process and concerns were taken into account, and medication was not always discontinued. The relatives played a supporting role.

But what was very good was the involvement of the relatives, which I think gave the patients more confidence that they could make a joint decision, because they had the feeling that the family was behind it, or the husband or daughter, and then of course that also made some treatment decisions easier. (GP 9: pos. 12)

The main focus of the family conferences was on adapting the medication. In general, a positive attitude toward deprescribing was apparent among GPs. However, it was sometimes reported that patients or relatives refused to reduce the medication.

I was actually surprised at how unwilling patients are to change anything. So the deprescribing, that’s always a good idea, but even if you have something in mind for the conference, two days later you get a call from the relative saying, no, that’s not possible because so and so. So it was actually a bit frustrating in retrospect. (GP 5: pos. 22)

Usefulness of supporting components.

The deprescribing manual, used both as a decision-making and an argumentation aid in the family conferences, was considered particularly supportive.

So I kept checking again and again. It always reassured me that I had already cancelled certain things. (GP 2: pos. 28)

The pharmacological hotline was not often used, as little need for it was seen.

Barriers and facilitators for the implementation of family conferences.

The organisation and implementation of the family conferences were predominantly experienced as unproblematic, and the relatives could mostly be well integrated. In some cases, it was not possible to involve the relatives (e.g., if relatives living far away or due to lack of time). Furthermore, the scheduling was sometimes difficult.

Yes, it didn’t always work out directly. With one patient we had a conference without the relatives, but otherwise it worked out with them. (GP 15: pos. 16)

The time expenditure was perceived very differently. Most of the GPs interviewed considered the amount of time to be appropriate. However, some considered the implementation of family conferences in regular care to be impracticable and too time-consuming. In some cases, it was not considered necessary to conduct family conferences in the patients’ home or as comprehensively and structured as envisaged in the guideline developed for conduction of the family conference in the COFRAIL study.

[…] Patients who normally go shopping or come to our practice. And I think the effort is simply too high, because home visits are always very time-consuming. You have the journey, you have to find a parking space, you’re simply there for longer because the patients in their familiar surroundings naturally have other issues that they want to address and then the relatives also have ten questions. (GP 9: pos. 34)

Not with a standardised sheet every time, so I don’t have a formal conference at home with the patients, but I have already incorporated the content into my daily practice. (GP 17: pos. 54)

The SARS-CoV-2 pandemic was another relevant contextual factor, but the GPs, like the patients and relatives, did not experience any negative effects regarding implementation of the study intervention. In some cases, family conferences were conducted by telephone or online to avoid personal contact.

We had family conferences under corona precautions. In other words, I also wore a gown in two cases because relatives were of a certain age and it wasn’t safe because I had to deal with potentially contaminated people on a daily basis. I always wore a mask and occasionally wore gloves, but it wasn’t really a problem. (GP 19: pos 30)

Discussion

The results of the process evaluation revealed that the COFRAIL intervention programme was largely implemented as intended, with adequate fidelity and intensity. The training sessions were mainly conducted according to the training manual and almost all intervention GPs completed both mandatory training courses. Slightly less than half of the GPs also participated in the third optional training. Overall, the GPs were highly satisfied with the educational programme and felt well prepared for conducting the family conferences. However, some GPs were still concerned that deprescribing might be associated with negative consequences for patients and could lead to conflicts with other specialists. This finding has previously been identified as a common barrier to deprescribing [37].

More than two-thirds of the patients received all three family conferences, and 85% of the patients received at least two family conferences that included structured medication reviews. Two earlier cRCTs investigated intervention programmes including among others GP-led medication reviews and reported that 87% and 78%, respectively, of medication reviews were conducted [38–40]. Thus, the intensity is similar to that of other studies. The first intervention programme was effective in reducing potentially inappropriate prescribing (PIP), which refers to various suboptimal prescribing practices, particularly the use of medication that implies a negative benefit-harm balance [38,39]. The latter study found that the intervention programme resulted in a small but significant reduction in the number of medications, but no significant reduction in PIP. However, the study only included patients with a much higher number of prescribed medications than in COFRAIL (≥15 vs. ≥ 5), which suggests a higher deprescribing potential [15,40,41].

In contrast, another study described an implementation failure, with only 49% of patients receiving both intended medication reviews and 30% receiving a partial review, resulting in no significant effect on either the primary trial outcome (quality of life) or the secondary outcome (number of medications prescribed) [42].

The COFRAIL intervention was somewhat unique in involving family members in the decision-making process. This involvement is crucial, as it can provide additional support and reassurance to patients during the decision-making process, potentially enhancing the acceptability and effectiveness of deprescribing efforts. In about 80% of the family conferences, at least one relative participated. To our knowledge, no interventional study aimed at improving drug safety in older adults with frailty in primary care involved relatives in the intervention programme [43].

The duration of the COFRAIL family conferences varied widely between ten and 120 minutes. It is therefore questionable to what extent the patients’ needs have always been adequately addressed and prioritised. The GPs’ documentation indicated that the focus of the first conference seemed to be clearly on the adaption of medication and in the subsequent conferences on non-pharmacological measures and recommendations as was intended in the intervention concept. As described in the published main study [15], after six months the mean number of PIMs was significantly lower in the intervention group compared to the control group (1.30 [SD: 1.05] vs. 1.71 [SD 1.25]; P = 0.04). An additional exploratory analysis found that mostly proton pump inhibitors, urate-lowering medications, statins, and oral antidiabetic agents were discontinued. After twelve months, there was no significant difference [15]. An in-depth analysis of the medication changes will be published separately. However, there was also a large proportion of patients who did not receive any non-pharmacological measures or recommendations as a result of the family conferences. This may indicate that these patients already received appropriate care or that there were some inconsistencies in the delivery of the intervention.

Overall, the family conferences were experienced very positively by the interviewed patients, their relatives and the GPs. The patients and relatives usually felt sufficiently involved in the family conferences and the decision-making process. Earlier evidence suggests that it is otherwise an important barrier to deprescribing interventions [44]. The adjustment of medication was perceived by the patients and relatives as the main focus of the family conferences, and most patients and relatives relied on the judgment of the GP. Physician trust has been described as an important factor that also may influence the extent of acceptance with regard to deprescribing [45]. However, some concerns about discontinuation of medication were also expressed.

Interestingly, it emerged during the interviews that some patients and relatives primarily perceived the data collection of the main study as part of the intervention. A previous study suggested that the intensive study data collection led to a potentially substantial bias and had the same effect in the control group as the structured medication reviews in the intervention group [46]. However, in our study, there were no significant changes regarding the main study outcomes after 12 months, neither in the intervention group nor in the control group [15].

Another component of the COFRAIL intervention was a pharmacological hotline, where the intervention GPs had the opportunity to consult an external clinical pharmacologist/pharmacist via phone or email [15]. Several studies involved pharmacists and pharmacologists in the intervention programmes aimed at improving appropriateness of medication regimes and the quantity of drugs with inconclusive results [42,47–52]. However, this supportive component was rarely utilised as the GPs expressed little need for it. Pharmacological counselling or reassurance may already have been sufficiently covered by the deprescribing manual provided. In addition, emerging questions may also have been clarified as part of the COFRAIL training programme by means of peer discussions. Regardless of this, perhaps the additional opportunity of using a pharmacological hotline could have been promoted more intensively to the GPs. Furthermore, possible barriers on the part of the GPs could be discussed in more depth during the training programme.

Overall, we identified no major barriers in the implementation of the intervention programme. Only a few GPs considered the implementation of family conferences in routine care to be impractical and too time-consuming. Involving family members or representatives of home care service in family conferences was mentioned by some GPs but was not described as a major challenge. In contrast, in quite a lot of family conferences neither a family member nor a representative of the home care service attended.

Although some GPs indicated concerns about conflict with other specialists after the educational programme, it was not reported as a specific barrier. From February 2020, the COFRAIL study was affected by the onset of the SARS-CoV-2-pandemic in Germany. As in many other studies [53,54], some procedures in the COFRAIL study had to be adapted. Thus, some training sessions and family conferences were held via telephone or online. However, from the perspective of patients, relatives and GPs this change in study procedures did not appear to disturb the process of a family conference to a relevant extent.

Strengths and limitations

Our process evaluation was based on the recommendations for process evaluations for cluster randomised trials of complex interventions [34]. Quantitative and qualitative methods were rigorously applied at different stages of the main study. The perspectives of patients, their relatives and GPs regarding the intervention programme and its implementation were comprehensively covered. The interviews were conducted by interviewers who were not involved in the main study, reducing the likelihood of socially desirable responses from the interviewees.

Data saturation was achieved both in the interviews with GPs and in the interviews with patients and relatives. However, convenience sampling for the interviews should be considered, which might affect the transferability and credibility. It cannot be ruled out that particularly motivated GPs with a more positive attitude toward deprescribing as well as less frail, and ill, or cognitively impaired patients agreed to participate in the interviews. The characteristics of the interview samples do not support this assumption, since they are largely comparable with the overall population of the COFRAIL study. Furthermore, the interviews with patients and relatives were planned as dyad-interviews, which turned out to be difficult to realise. Relatives sometimes did not live in the same household, or the patients were willing to take part in an interview but the relatives were not, or vice versa. Therefore, the methods were adapted to the research context and the interviews were mainly conducted separately, which likely led to a different dynamic of the conversations.

Although the interviews provide some insights, it remains unclear in this process evaluation to some extent whether the documented medical and non-pharmacological measures and recommendations resulting from family conferences were actually implemented by the patients and relatives. It is also uncertain how consistent the quality of the family conferences among the GPs was, as envisaged by the training programmes. Furthermore, the process evaluation focused on the implementation of the intervention rather than on the usual care provided by the GPs in the control group. Therefore, it remains unclear whether and which recommendations were made to their patients during the study period. It cannot be ruled out that the perceived satisfaction with the increased time available with the GP, as expressed by patients in the interviews, may also reflect a bias due to the open trial design of the COFRAIL study without blinding towards the intervention. However, these process measures were not collected in the control group. Moreover, the process evaluation focused more on the process of implementing the intervention and the experiences of the target groups. It was not possible to fully understand the ineffectiveness of the COFRAIL intervention on the basis of this process evaluation, even though some relevant factors were taken into account.

Conclusions

The COFRAIL study sought to address the significant challenges of polypharmacy and frailty in older adults by implementing a structured intervention centred on family conferences. Findings of the COFRAIL trial and this process evaluation suggest that family conferences for shared decision-making can successfully initiate the process of discontinuing medication. Despite the comprehensive nature of the intervention, however, the study did not achieve its goals for the main outcomes of reducing hospitalisations, the number of medications or PIM.

Overall, the intervention was well implemented and no major barriers were found, suggesting that the intervention may not have had enough impact to effectively influence these outcomes.

However, the process evaluation of the COFRAIL study offers important insights into the implementation and reception of the intervention, which can inform future efforts in similar contexts. The training provided to the GPs was generally well received, but some GPs expressed ongoing concerns about deprescribing, particularly regarding the potential for negative consequences for patients and conflicts with other specialists. This highlights the need for more robust support systems and further education to address these barriers.

The family conferences themselves were largely implemented as intended, with a large proportion of patients receiving all sessions. The involvement of relatives in these conferences was positively perceived by both patients and their families. Variation in the duration of the family conferences and partly a lack of non-pharmacological recommendations suggest that there may be inconsistencies in the delivery of the intervention. Some patients and relatives did not perceive the family conferences as distinct from usual care, or associated the intervention with the data collection process of the study, indicating a need for clearer communication.

The underutilisation of the pharmacological hotline suggests that this support component was not fully integrated into the intervention or perceived as valuable by the GPs. This issue could be improved by better promotion of the hotline and a more direct involvement of clinical pharmacologists in the decision-making process during family conferences in future studies.

Supporting information

S1 Appendix. Interview guide for patients and relatives.

https://doi.org/10.1371/journal.pone.0350664.s001

(PDF)

S2 Appendix. Interview guide for general practitioner.

https://doi.org/10.1371/journal.pone.0350664.s002

(PDF)

S3 Appendix. eTable 1. Contents and procedures of process evaluation.

https://doi.org/10.1371/journal.pone.0350664.s003

(PDF)

Acknowledgments

We would like to thank all the participating general practitioners, patients and family members. We also thank Mareike Bake (University Hospital Halle (Saale), University Medicine Halle (Saale), Germany) for the support in the coding of the interviews. We thank Vivienne Krause for double-checking of the translation of interview quotes and the language check of the manuscript. We would like to acknowledge the members of the COFRAIL study group: Achim Mortsiefer (Lead author of the COFRAIL study group; E-mail: Achim.Mortsiefer@uni-wh.de, Institute of General Practice and Primary Care, Faculty of Health, Department of Medicine, Witten/Herdecke University, Witten, Germany), Susanne Löscher (Institute of General Practice, Medical Faculty, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany), Yekaterina Pashutina (Institute of General Practice, Medical Faculty, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany), Sara Santos (Institute of General Practice, Medical Faculty, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany), Attila Altiner (Department of General Practice and Health Services Research, University Hospital Heidelberg, Heidelberg, Germany), Eva Drewelow (Clinic for Forensic Psychiatry, University Medical Centre Rostock, Rostock, Germany), Manuela Ritzke (Institute of General Practice, University Medical Centre Rostock, Rostock, Germany), Anja Wollny (Institute of General Practice, University Medical Centre Rostock, Rostock, Germany), Petra Thürmann (Chair of Clinical Pharmacology, Department of Medicine, Faculty of Health, Witten/Herdecke University, Witten, Germany; Philipp Klee-Institute of Clinical Pharmacology, Helios University Hospital Wuppertal, Wuppertal, Germany), Veronika Bencheva (Center for clinical trials, Faculty of Health, Witten/Herdecke University, Witten, Germany; Chair of Clinical Pharmacology, Department of Medicine, Faculty of Health, Witten/Herdecke University, Witten, Germany), Matthias Gogolin (Department of Clinical Pharmacology, School of Medicine, Faculty of Health, Witten/Herdecke University, Witten, Germany), Gabriele Meyer (Institute of Health, Midwifery and Nursing Science, Medical Faculty of Martin Luther University Halle-Wittenberg, Halle (Saale), Germany), Jens Abraham (Institute of Health, Midwifery and Nursing Science, Medical Faculty of Martin Luther University Halle-Wittenberg, Halle (Saale), Germany), Steffen Fleischer (Institute of Health, Midwifery and Nursing Science, Medical Faculty of Martin Luther University Halle-Wittenberg, Halle (Saale), Germany), Andrea Icks (Institute for Health Services Research and Health Economics, Centre for Health and Society, Faculty of Medicine, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany), Joseph Montalbo (Institute for Health Services Research and Health Economics, Centre for Health and Society, Faculty of Medicine, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany), Birgitt Wiese (WG Medical Statistics and IT-Infrastructure, Institute of General Practice, Hannover Medical School, Hannover, Germany), Stefan Wilm (Institute of General Practice, Medical Faculty, Heinrich-Heine-University Düsseldorf, Germany), Gregor Feldmeier (Institute of General Practice, University Medical Centre Rostock, Rostock, Germany). We acknowledge the financial support of the Open Access Publication Fund of the Martin Luther University Halle-Wittenberg.

References

  1. 1. Gutiérrez-Valencia M, Izquierdo M, Cesari M, Casas-Herrero Á, Inzitari M, Martínez-Velilla N. The relationship between frailty and polypharmacy in older people: A systematic review. Br J Clin Pharmacol. 2018;84(7):1432–44. pmid:29575094
  2. 2. Chen X, Mao G, Leng SX. Frailty syndrome: an overview. Clin Interv Aging. 2014;9:433–41. pmid:24672230
  3. 3. Fried LP, Tangen CM, Walston J, Newman AB, Hirsch C, Gottdiener J, et al. Frailty in older adults: evidence for a phenotype. J Gerontol A Biol Sci Med Sci. 2001;56(3):M146-56. pmid:11253156
  4. 4. Buckinx F, Rolland Y, Reginster J-Y, Ricour C, Petermans J, Bruyère O. Burden of frailty in the elderly population: perspectives for a public health challenge. Arch Public Health. 2015;73(1):19. pmid:25866625
  5. 5. Veronese N, Custodero C, Cella A, Demurtas J, Zora S, Maggi S, et al. Prevalence of multidimensional frailty and pre-frailty in older people in different settings: A systematic review and meta-analysis. Ageing Res Rev. 2021;72:101498. pmid:34700009
  6. 6. Pilotto A, Ferrucci L, Franceschi M, D’Ambrosio LP, Scarcelli C, Cascavilla L, et al. Development and validation of a multidimensional prognostic index for one-year mortality from comprehensive geriatric assessment in hospitalized older patients. Rejuvenation Res. 2008;11(1):151–61. pmid:18173367
  7. 7. Fried TR, O’Leary J, Towle V, Goldstein MK, Trentalange M, Martin DK. Health outcomes associated with polypharmacy in community-dwelling older adults: a systematic review. J Am Geriatr Soc. 2014;62(12):2261–72. pmid:25516023
  8. 8. Wastesson JW, Morin L, Tan ECK, Johnell K. An update on the clinical consequences of polypharmacy in older adults: a narrative review. Expert Opin Drug Saf. 2018;17(12):1185–96. pmid:30540223
  9. 9. Davies LE, Spiers G, Kingston A, Todd A, Adamson J, Hanratty B. Adverse Outcomes of Polypharmacy in Older People: Systematic Review of Reviews. J Am Med Dir Assoc. 2020;21(2):181–7. pmid:31926797
  10. 10. Flaherty JH, Perry HM 3rd, Lynchard GS, Morley JE. Polypharmacy and hospitalization among older home care patients. J Gerontol A Biol Sci Med Sci. 2000;55(10):M554-9. pmid:11034227
  11. 11. Muhlack DC, Hoppe LK, Saum K-U, Haefeli WE, Brenner H, Schöttker B. Investigation of a possible association of potentially inappropriate medication for older adults and frailty in a prospective cohort study from Germany. Age Ageing. 2019;49(1):20–5. pmid:31732737
  12. 12. Masnoon N, Shakib S, Kalisch-Ellett L, Caughey GE. What is polypharmacy? A systematic review of definitions. BMC Geriatr. 2017;17(1):230. pmid:29017448
  13. 13. Pazan F, Wehling M. Polypharmacy in older adults: a narrative review of definitions, epidemiology and consequences. Eur Geriatr Med. 2021;12(3):443–52. pmid:33694123
  14. 14. Khezrian M, McNeil CJ, Murray AD, Myint PK. An overview of prevalence, determinants and health outcomes of polypharmacy. Ther Adv Drug Saf. 2020;11:2042098620933741. pmid:32587680
  15. 15. Mortsiefer A, Löscher S, Pashutina Y, Santos S, Altiner A, Drewelow E, et al. Family Conferences to Facilitate Deprescribing in Older Outpatients With Frailty and With Polypharmacy: The COFRAIL Cluster Randomized Trial. JAMA Netw Open. 2023;6(3):e234723. pmid:36972052
  16. 16. Mortsiefer A, Wilm S, Santos S, Löscher S, Wollny A, Drewelow E, et al. Family conferences and shared prioritisation to improve patient safety in the frail elderly (COFRAIL): study protocol of a cluster randomised intervention trial in primary care. Trials. 2020;21(1):285. pmid:32197631
  17. 17. Scott IA, Hilmer SN, Reeve E, Potter K, Le Couteur D, Rigby D, et al. Reducing inappropriate polypharmacy: the process of deprescribing. JAMA Intern Med. 2015;175(5):827–34. pmid:25798731
  18. 18. Jansen J, Naganathan V, Carter SM, McLachlan AJ, Nickel B, Irwig L, et al. Too much medicine in older people? Deprescribing through shared decision making. BMJ. 2016;353:i2893. pmid:27260319
  19. 19. Hansen CR, O’Mahony D, Kearney PM, Sahm LJ, Cullinan S, Huibers CJA, et al. Identification of behaviour change techniques in deprescribing interventions: a systematic review and meta-analysis. Br J Clin Pharmacol. 2018;84(12):2716–28. pmid:30129139
  20. 20. Travis SS, McAuley WJ, Dmochowski J, Bernard MA, Kao H-FS, Greene R. Factors associated with medication hassles experienced by family caregivers of older adults. Patient Educ Couns. 2007;66(1):51–7. pmid:17118616
  21. 21. Dixe MDACR, da Conceição Teixeira LF, Areosa TJTCC, Frontini RC, de Jesus Almeida Peralta T, Querido AIF. Needs and skills of informal caregivers to care for a dependent person: a cross-sectional study. BMC Geriatr. 2019;19(1):255. pmid:31533637
  22. 22. Gench M, Sawan MJ, Langford A, Gnjidic D. Tools to evaluate medication management for caregivers of people living with dementia: A systematic review. Health Expect. 2021;24(5):1570–81. pmid:34289214
  23. 23. Sawan MJ, Jeon Y-H, Bond C, Hilmer SN, Chen TF, Wennekers D, et al. Caregivers’ experiences of medication management advice for people living with dementia at discharge. J Eval Clin Pract. 2021;27(6):1252–61. pmid:33586327
  24. 24. Bångsbo A, Dunér A, Lidén E. Patient participation in discharge planning conference. Int J Integr Care. 2014;14:e030. pmid:25411572
  25. 25. Cahill PJ, Lobb EA, Sanderson C, Phillips JL. What is the evidence for conducting palliative care family meetings? A systematic review. Palliat Med. 2017;31(3):197–211. pmid:27492159
  26. 26. Guan T, Otis-Green S, Sharma A, Kuiken A, Chan BM, Fineberg IC, et al. Family meetings in palliative care: A systematic review. Patient Educ Couns. 2026;146:109499. pmid:41616646
  27. 27. Kesecioglu J, Rusinova K, Alampi D, Arabi YM, Benbenishty J, Benoit D, et al. European Society of Intensive Care Medicine guidelines on end of life and palliative care in the intensive care unit. Intensive Care Med. 2024;50(11):1740–66. pmid:39361081
  28. 28. McInerney BE, Cross AJ, Turner JP, Bell JS. Systematic Review of Psychotropic Adverse Drug Event Monitoring Tools for Use in Long-Term Care Facilities. J Am Med Dir Assoc. 2023;24(6):773-781.e5. pmid:37037347
  29. 29. Nascimento A, Brandão-Silva J, Cunha D, Ramos I, Pereira C, Semedo V, et al. Caregiver perceptions of the impact and effect of family conferences for patients with complex and high-level palliative care needs: a cross-sectional survey study. BMC Palliat Care. 2026;25(1):58. pmid:41514417
  30. 30. Zhang H, Wang N, Bai N, Yin M. Conducting family meetings on families with dementia: An integrative review. J Clin Nurs. 2024;33(4):1362–75. pmid:38317507
  31. 31. Mann N-K, Schmiedl S, Mortsiefer A, Bencheva V, Löscher S, Ritzke M, et al. Development of a deprescribing manual for frail older people for use in the COFRAIL study and in primary care. Ther Adv Drug Saf. 2022;13:20420986221122684. pmid:36091625
  32. 32. Drewelow E, Ritzke M, Altiner A, Icks A, Montalbo J, Kalitzkus V, et al. Development of a shared decision-making intervention to improve drug safety and to reduce polypharmacy in frail elderly patients living at home. PEC Innov. 2022;1:100032. pmid:37213749
  33. 33. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061. pmid:34593508
  34. 34. Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14:15. pmid:23311722
  35. 35. Mayring P. Qualitative content analysis: theoretical foundation, basic procedures and software solution. 2014. https://nbnresolving.org/urn:nbn:de:0168-ssoar-395173
  36. 36. Dresing T, Pehl P. Practice book transcription. Control systems, software and practical guidelines for qualitative researchers. 4th ed. Eigenverlag. 2013.
  37. 37. Anderson K, Stowasser D, Freeman C, Scott I. Prescriber barriers and enablers to minimising potentially inappropriate medications in adults: a systematic review and thematic synthesis. BMJ Open. 2014;4(12):e006544. pmid:25488097
  38. 38. Clyne B, Cooper JA, Hughes CM, Fahey T, Smith SM, OPTI-SCRIPT study team. A process evaluation of a cluster randomised trial to reduce potentially inappropriate prescribing in older people in primary care (OPTI-SCRIPT study). Trials. 2016;17(1):386. pmid:27488272
  39. 39. Clyne B, Smith SM, Hughes CM, Boland F, Bradley MC, Cooper JA, et al. Effectiveness of a Multifaceted Intervention for Potentially Inappropriate Prescribing in Older Patients in Primary Care: A Cluster-Randomized Controlled Trial (OPTI-SCRIPT Study). Ann Fam Med. 2015;13(6):545–53. pmid:26553894
  40. 40. McCarthy C, Pericin I, Smith SM, Kiely B, Moriarty F, Wallace E, et al. Patient and general practitioner experiences of implementing a medication review intervention in older people with multimorbidity: Process evaluation of the SPPiRE trial. Health Expect. 2022;25(6):3225–37. pmid:36245339
  41. 41. McCarthy C, Clyne B, Boland F, Moriarty F, Flood M, Wallace E, et al. GP-delivered medication review of polypharmacy, deprescribing, and patient priorities in older people with multimorbidity in Irish primary care (SPPiRE Study): A cluster randomised controlled trial. PLoS Med. 2022;19(1):e1003862. pmid:34986166
  42. 42. Mann C, Shaw ARG, Guthrie B, Wye L, Man M-S, Chaplin K, et al. Can implementation failure or intervention failure explain the result of the 3D multimorbidity trial in general practice: mixed-methods process evaluation. BMJ Open. 2019;9(11):e031438. pmid:31699734
  43. 43. Ibrahim K, Cox NJ, Stevenson JM, Lim S, Fraser SDS, Roberts HC. A systematic review of the evidence for deprescribing interventions among older people living with frailty. BMC Geriatr. 2021;21(1):258. pmid:33865310
  44. 44. Doherty AJ, Boland P, Reed J, Clegg AJ, Stephani A-M, Williams NH, et al. Barriers and facilitators to deprescribing in primary care: a systematic review. BJGP Open. 2020;4(3):bjgpopen20X101096. pmid:32723784
  45. 45. Reeve E, Wiese MD, Hendrix I, Roberts MS, Shakib S. People’s attitudes, beliefs, and experiences regarding polypharmacy and willingness to Deprescribe. J Am Geriatr Soc. 2013;61(9):1508–14. pmid:24028356
  46. 46. Muth C, Uhlmann L, Haefeli WE, Rochon J, van den Akker M, Perera R, et al. Effectiveness of a complex intervention on Prioritising Multimedication in Multimorbidity (PRIMUM) in primary care: results of a pragmatic cluster randomised controlled trial. BMJ Open. 2018;8(2):e017740. pmid:29478012
  47. 47. Salisbury C, Man M-S, Bower P, Guthrie B, Chaplin K, Gaunt DM, et al. Management of multimorbidity using a patient-centred care model: a pragmatic cluster-randomised trial of the 3D approach. Lancet. 2018;392(10141):41–50. pmid:29961638
  48. 48. Holland R, Desborough J, Goodyer L, Hall S, Wright D, Loke YK. Does pharmacist-led medication review help to reduce hospital admissions and deaths in older people? A systematic review and meta-analysis. Br J Clin Pharmacol. 2008;65(3):303–16. pmid:18093253
  49. 49. Birt L, Dalgarno L, Wright DJ, Alharthi M, Inch J, Spargo M, et al. Process evaluation for the Care Homes Independent Pharmacist Prescriber Study (CHIPPS). BMC Health Serv Res. 2021;21(1):1041. pmid:34600542
  50. 50. Geurts MME, Talsma J, Brouwers JRBJ, de Gier JJ. Medication review and reconciliation with cooperation between pharmacist and general practitioner and the benefit for the patient: a systematic review. Br J Clin Pharmacol. 2012;74(1):16–33. pmid:22242793
  51. 51. Howard R, Rodgers S, Avery AJ, Sheikh A, PINCER trialists. Description and process evaluation of pharmacists’ interventions in a pharmacist-led information technology-enabled multicentre cluster randomised controlled trial for reducing medication errors in general practice (PINCER trial). Int J Pharm Pract. 2014;22(1):59–68. pmid:23718905
  52. 52. Blum MR, Sallevelt BTGM, Spinewine A, O’Mahony D, Moutzouri E, Feller M, et al. Optimizing Therapy to Prevent Avoidable Hospital Admissions in Multimorbid Older Adults (OPERAM): cluster randomised controlled trial. BMJ. 2021;374:n1585. pmid:34257088
  53. 53. Bratan T, Aichinger H, Brkic N, Rueter J, Apfelbacher C, Boyer L, et al. Impact of the COVID-19 pandemic on ongoing health research: an ad hoc survey among investigators in Germany. BMJ Open. 2021;11(12):e049086. pmid:34872995
  54. 54. Schnack H, Lubasch JS, Zinkevich A, Pawel A, Uthoff SAK, Ansmann L, et al. Health Services research in Germany in Times of COVID-19: How Does the Pandemic Influence Research Processes, Methods and the Personal Situation of Researchers? An Online Survey. Gesundheitswesen. 2023;85(6):495–504. pmid:37315565