There have been growing recommendations to include education in multi-disciplinary interventions targeting chronic pain management. However, effects of this strategy on short- and long-term self-management of chronic pain, remain largely unexplored.
1. To provide an updated overview of studies that report on the impact of patient education in multi-disciplinary interventions, on self-management of chronic pain; 2. To explore associations between education and chronic pain self-management techniques; and 3. To identify the format and duration of suitable chronic pain interventions targeted at patient self-management.
Design: Narrative systematic literature review of randomised or controlled study designs. Data Sources: PubMed, CINAHL, EMBASE, PsycINFO. Participants: Adult patients with chronic pain of any aetiology participating in multi-disciplinary programs that included education. Main outcome measures: Assessments of level of pain, function, quality of life, self-efficacy, self-management, and any other relevant assessments. Study Appraisal and Synthesis Methods: PRISMA guidelines, Cochrane Risk of Bias tool, and TIDieR model.
Database searching identified 485 potential papers. After removal of duplicates, and irrelevant articles by title and abstract, 120 full-text articles were reviewed and 27 studies were included in this systematic review. Studies were predominantly from the United States (n = 8; 29.6%). Over one hundred outcome measures were identified across all studies, with significant variation also observed in terms of how chronic pain duration was defined, and how education was delivered to participants. Overall, positive benefits of education were reported.
Education, as part of multi-disciplinary programs, is likely to improve self-management and self-efficacy in people with chronic pain of any aetiology. Heterogeneity in terms of: chronic pain duration; educational resources; healthcare professionals; and outcome measures, were identified as limitations. Further research, in the form of Randomised Controlled Trials addressing these limitations, is recommended.
Citation: Joypaul S, Kelly F, McMillan SS, King MA (2019) Multi-disciplinary interventions for chronic pain involving education: A systematic review. PLoS ONE 14(10): e0223306. https://doi.org/10.1371/journal.pone.0223306
Editor: Robert S. Phillips, York University, UNITED KINGDOM
Received: January 10, 2019; Accepted: September 18, 2019; Published: October 2, 2019
Copyright: © 2019 Joypaul et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Chronic pain is pain that lasts or recurs for more than three months [1–3]. Globally, prevalence of chronic pain has been reported as 37.3% of the population in developed countries and 41.1% in developing countries . In Australia, chronic pain is the fourth most common chronic condition , affecting almost 20% of the population . Chronic pain is complex in terms of aetiology and management approaches [6–8], and despite the existence of pharmacological therapy [9, 10], remains highly resistant to treatment [5, 10]. Chronic pain significantly and negatively impacts upon individual lives, leading to physical disability, mental health problems and long waitlists for specialist health services, as well as economic costs to health services, patients and the community [1, 4, 11–14]. A sustainable solution is needed to reduce these negative impacts and promote effective self-management, with increased recognition of the role of public education around pain and its management, and co-ordinated multi-disciplinary (MD) care [1, 9].
Public education promotes patient awareness of the somatic, psychological and social aspects of chronic pain [1, 6–9], and supports self-management with the aim to improve quality of life . Education alone can have a limited impact on chronic pain [15–24], yet when combined with MD programs, there seems to be additional benefits for chronic pain management, such as increased patient confidence and self-efficacy [25–27]. MD care refers to the collaboration of healthcare professionals from a range of disciplines to deliver comprehensive patient care that meets the needs of the individual . MD programs designed for chronic pain management generally include patient education and other services or care, such as medication reviews, self-management programs, allied health and community-based services, in various delivery modes, such as face-to-face, teleconferencing, and/or internet-based sessions [1, 9]. Including education in MD programs has been associated with medication optimization, reduced pain catastrophizing, and reduced utilisation of secondary care services [9, 25, 26, 29]. However, research has highlighted the need for Randomised Controlled Trials (RCTs) to confirm the immediate and sustained impact of these programs on health outcomes [25, 26].
There has been limited reporting of empirical evidence of MD interventions; questions remain regarding the efficacy of individual versus group education and the optimal combination of educational topics [1, 27, 30–32]. Although there are some randomised and/or controlled studies [33–59] and reviews in this area [27, 30–32], they have specific limitations. The most detailed systematic review to date was published by Scascighini et al. in 2008 . MD programs, with or without education components, were reported to be more effective than no treatment or standard medical treatment, in participants with chronic, non-specific musculoskeletal pain . The authors concluded that incorporation of once-weekly patient education sessions in MD programs was a superior strategy compared to other medical treatment . A strength of the review was the exclusion of non-RCT studies, however, limitations included: an incomplete reporting of the search strategy; a narrow scope focussing solely on cognitive-behavioural and psychological graded interventions; and limited assessment of methodological quality and risk of bias assessments of included studies . More recent meta-analyses were restricted to small numbers of included papers and the investigation of education as a solitary intervention rather than as part of MD programs [31, 32]. Another narrative review around preventive interventions focused on post extremity trauma, that is, one pain type only .
Questions remain on how to best incorporate education into MD programs for chronic pain. Such knowledge remains critical in an era when the prevalence of chronic pain is predicted to rise with an ageing population [5, 29]. The aim of this systematic literature review was to update and extend Scascighini et al.’s review . This included: 1. Using a well-defined and comprehensive search strategy to identify studies that report the impact of patient education in a broad range of MD interventions; 2. Exploring for associations between education and chronic pain self-management techniques; and 3. Identifying format and duration of suitable chronic pain interventions targeted at self-management from methodological quality and risk of bias assessments. The findings of this review will inform researchers, healthcare professionals and/or people experiencing chronic pain about the current evidence of MD interventions, as well as highlight critical aspects to include in future programs alongside realistic expectations of effectiveness.
A systematic literature review was performed according to the methods outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines . PRISMA guidelines constitute an evidence-based protocol for developing and reporting narrative systematic reviews and meta-analyses , and are recommended by the Cochrane community and the EQUATOR network when reporting database searches in systematic reviews [61, 62]. Prior to commencing the systematic review, all authors agreed to relevant definitions and a robust search strategy and study selection process (see Supporting information); two attempts at contacting Scascighini et al.  for a detailed search strategy were unsuccessful. No protocol exists for this work.
For the purpose of this review, the definition of chronic pain was: pain of any aetiology persisting for more than three months . Interventions were deemed MD if they involved collaboration of healthcare professionals from a range of disciplines to deliver comprehensive care to the patient . Education was defined as instructions to inform participants about self-management and/or medication-taking techniques for their chronic pain. The education could be delivered via a range of modalities, including face-to-face, teleconferencing, internet-based sessions and/or use of other multimedia content. Participants included persons with chronic pain who consented to take part in the research, such as those who volunteered, persons who were referred to pain clinics or health centres, individuals who were on sick leave from work, or persons living in retirement communities. Study designs included in this review were RCTs and/or other randomised or controlled study designs (e.g. cluster randomised trials).
A specialist librarian was consulted on three occasions for advice around, and refinement of, the search strategy (see S2 Appendix). Health databases searched were CINAHL, EMBASE, PubMed and PsycINFO, using a combination of Medical Subject Heading (MeSH) terms (e.g. “chronic pain,” “health education,” “multidisciplinary care team,” and “interdisciplinary treatment approach”) and text words (e.g. “self care”). These databases represent the four most appropriate databases that were likely to produce a broad range of peer-reviewed literature across the disciplines of medicine, nursing, psychology and allied health. The search was conducted on five occasions from mid-2017 to August 2019, with the search strategy consistently adopted across all databases to ensure identification of eligible studies. The final list of studies was exported to a referencing management system (EndNote®) in August 2019.
Eligibility criteria and study selection
The following inclusion and exclusion criteria were established (Table 1).
Eligibility for inclusion/exclusion was first assessed via independent duplicate manual screening of article titles and abstracts (S.J. and S.M.). Disagreements were resolved by discussion and consensus with a third author (M.K.). The same process was undertaken when reviewing all relevant full-text articles. A supplementary search method, involving a review of reference lists of included full-text articles (i.e. snowballing), was adopted.
Information was extracted into a Microsoft Excel® spreadsheet and included characteristics such as country, sample size, number of chronic pain types and/or sites, and members of the MD team. The outcome measures were noted as total number of outcome measures used, those that showed significant results and whether such measures were validated. Information extracted into a separate Microsoft Word® document included more detailed description of the individual interventions including type and delivery mode, as well as study setting, patient details and age range, findings and follow-up. The objectives, randomisation and blinding procedures, as well as group allocation details, were also noted.
Risk of bias and quality of reporting
Risk of bias and quality of reporting assessments were undertaken by the first author (S.J.), following concurrent independent testing (S.J. and M.K.) of the first six included articles against a checklist developed from the Cochrane Collaboration Risk of Bias tool [63–65] and the Template for Intervention Description and Replication (TIDieR) [66, 67]. Where articles lacked information, the respective study protocols were consulted. These steps ensured the completeness of reporting and helped determine the reliability and replicability of included studies [63–67]. Any disagreements were resolved through discussion between all authors.
Given the heterogeneity of study methods and outcome measures across all included studies as well as the broad nature of the review, statistical comparisons between studies were not possible. Hence, a narrative description of the data was adopted as a suitable method to meet the review aims . Other narrative studies published in evidence-based journals and databases such as PLoSONE  and Cochrane , are known to have used the same methodology.
The authors systematically analysed all reported outcomes across all included studies. Thematic analysis was performed from all outcomes and only those findings that aligned with the objectives of the work, have been reported in this paper.
The search strategy identified 485 records of which 69 duplicates were removed. An additional 297 records were excluded through title and abstract screening, with a total of 120 full-text articles retrieved for further evaluation. Twenty seven studies met the inclusion criteria (see Fig 1) [33–59]. A concise overview of data from these included studies is provided in Tables 2 and 3.
Characteristics of identified randomised and/or controlled studies
Findings from the 27 included studies that aligned with the objectives of this work are summarised below.
The number of study participants ranged from 23  to 365  across the 27 included studies. The minimum reported age of participants was 18 years  and the maximum reported age was 88 years . One study did not specify participant age range of adult participants . Most studies (n = 18) had more female than male participants [33, 34, 36–40, 42–44, 47, 50–54, 56, 57]; two studies only involved female participants [46, 49] and one did not specify gender distribution .
Intervention participants reported a diverse duration of chronic pain and its location or site/s. Fourteen studies reported a minimum chronic pain duration of three months [34, 38, 41–44, 46, 50, 52–56, 59], while seven studies included participants reporting pain for at least six months [33, 45, 48, 49, 51, 57, 58]. Six studies specified chronic pain, without an explicit pain duration for all participants [35–37, 39, 40, 47]. Participants with one pain site were addressed in ten studies [33, 36, 38, 40, 42, 47, 48, 52–54]. Of these, seven studies visited back pain [33, 40, 47, 48, 52–54], two studies visited neck pain [38, 42], and one study visited headaches . The remaining 17 studies included participants with a mix of pain aetiologies and sites [34, 35, 37, 39, 41, 43–46, 49–51, 55–59].
Included studies were predominantly conducted in the United States (n = 8) [35, 40, 41, 43, 48, 56, 58, 59], Australia (n = 4) [39, 44, 51, 55], and Germany (n = 3) [36, 42, 47], with two studies each from Denmark [37, 46], Spain [38, 50], Norway [34, 49], and the United Kingdom [54, 57]. Education was didactic across all studies and was facilitated by a collaboration of healthcare professionals from at least two disciplines. Physiotherapists, psychologists and nurses constituted most MD teams across all included studies.
All 27 studies involved more than one educational topic either as part of the active intervention or usual care or both. Education around physical activity was most prominent (n = 23) [33–35, 37–40, 42, 43, 45–55, 57–59], followed by cognitive behavioural strategies (n = 22) [33–39, 43–52, 54–58]. While not explicitly stated, two other studies were likely to have involved exercise education, for example on “training to maintain gains”  or ergonomics , and three additional studies were likely to have used cognitive behavioural aspects [40, 41, 59]. Education about medications and optimisation of analgesic treatment were included in eight studies [35, 37, 49, 51, 55, 56, 58, 59], with advice on alternative pain management strategies also part of the medication management training in these studies. Education about nutrition was included in one study .
The didactic mode of education delivery across all studies was in the form of lectures or seminars, with 25 studies also reporting the provision of at least one supplementary educational tool to participants [33–49, 51, 52, 54–59]. These additional tools included personalised health plans (n = 9) [34, 37, 43, 48, 49, 55–58], regular (e.g. daily/biweekly) telephone calls to participants (n = 9) [35, 37, 41, 43, 44, 55, 56, 58, 59], written materials (n = 9) [33, 35, 38, 39, 42, 55, 57–59] and books (n = 7) [40, 43–45, 51, 54, 56]. Four studies provided recorded materials [40, 43, 47, 56], with two studies each providing tapes [40, 47] and CDs [43, 56], and one study providing videos . Three studies used emails as an additional educational tool [41, 44, 46], three studies used diaries [36, 42, 49] in the form of journals (n = 2) [36, 42] and smartphone diaries (n = 1) . Pictures and metaphors , diagrams and drawings , text messages , computerised registers , and webpages  were each used in a single study. Twenty-three RCTs provided homework activity or take-home materials to participants [33–36, 38–45, 47–51, 53–58].
Duration of education delivery varied from a minimum of three hours on a single day (n = 1)  to a maximum of once-weekly sessions over a period of 18 months (n = 1) . Eleven studies provided education on an individualised basis [35, 37, 38, 40, 41, 44, 46, 48, 49, 56, 58]; eight studies involved group sessions [36, 39, 45, 50–54]; and eight combined individual and group education [33, 34, 42, 43, 47, 55, 57, 59].
The team involved in the delivery of the education content across all studies was mostly comprised of physiotherapists (n = 17) [33, 34, 37–42, 45, 47, 48, 50–53, 55, 57] and psychologists (n = 17) [33, 35–37, 39, 41–43, 45–48, 50, 51, 55, 57, 58]. Nurses (n = 9) were the third most common health professionals across all studies [35, 37, 41, 43, 45, 48, 49, 51, 55]. Health professionals in studies involving medication training were: nurses [35, 37, 49, 51], anaesthesiologists [33, 36, 37], general practitioners (GPs) , pain medicine specialists [55, 56], pharmacists [35, 59], physician assistants , and internal medicine physicians . Only five studies described four or more different types of healthcare professionals as part of their MD team [33, 35, 37, 41, 55]; 11 studies were unclear on the actual roles/types of healthcare professionals involved [34, 43, 44, 46, 49, 52, 54, 56–59].
Participant check-ins or monitoring periods between active intervention sessions varied widely, ranging from at least daily checks  to outcome checks every two months . These checks all preceded actual follow-up periods. Three of the 27 studies did not involve follow-up periods [41, 42, 46]. Of the remaining 24 studies, the maximum duration of follow-up was 18 months , with three to 12 months follow-up most commonly reported. Follow-up results were generally positive or showed no change compared to results at intervention completion. Fourteen studies adopted an intention-to-treat analysis in their methodology [33–35, 38, 40, 42, 43, 45, 46, 48, 49, 51, 52, 56].
Intervention outcomes and effectiveness were generally evaluated immediately after intervention conclusion, followed by re-evaluation at follow-up. The majority of studies (n = 23) showed statistically significant results (p<0.05 and/or large effect size) for one or more outcome measures [33–40, 42–47, 49–54, 56, 58, 59].
A total of 111 outcome measures were identified across all studies. Forty-two measures appeared once only and could not be compared between studies due to lack of similarity and uncertainty around validity. More than half of the included studies (n = 15) [34–37, 39, 40, 43, 47, 51, 53–58] used combinations of standardised, validated outcome measures (e.g. Tampa Scale for Kinesiophobia; TSK) with non-standardised, non-validated measures (e.g. distance walked in six minutes along corridor). Six studies had outcome measures tailored specifically to the research [36, 40, 47, 51, 53, 56]. None of the included studies used solely non-validated measures.
The number of outcome measures used per study ranged from one , to a maximum of 11 [47, 56], with no two studies using the same combination. Improvement in pain and mobility were measured in all 27 studies. These were mostly reported as scores from the Rolland Morris Disability Questionnaire (RDQ; n = 7) [33, 35, 41, 43, 51, 53, 54], the Visual Analog Scale (VAS; n = 6) [33, 34, 37, 42, 49, 52], the Health Survey Short Form 36 (SF-36; n = 4) [34, 37, 42, 53], and the Pain Self-Efficacy Questionnaire (PSEQ; n = 4) [51, 54, 56, 59]. These four tools represent reliable and widely-used measures of low back pain disability , intensity and frequency of chronic pain , health status  and ability to cope and manage despite pain [73, 74] respectively.
Changes in medication use before and after interventions were reported by seven studies [35, 37, 40, 43, 53, 56, 57], with only three of these delivering medication education [35, 37, 56]. One study by Sullivan and colleagues (2017) was specifically aimed at tapering opioid use .
Risk of bias and quality of reporting
Risk of bias (Table 4 and Fig 2) was variable across all seven Cochrane Risk of Bias elements, with Kerns et al. 2014 , reporting the lowest risk of bias for all elements.
All but three [43, 47, 59] of the included studies showed adequate randomisation of study participants: 13 studies reported random number generation [34, 36, 38–41, 44–46, 50, 54, 57, 58]; eight studies used block randomisation [33, 35, 37, 42, 48, 49, 52, 55]; and one study each reported drawing lots , a mix of block randomisation and random number tables  and a mix of computer-generated randomisation, sealed envelopes and blocked randomisation . The three remaining studies involved biased randomisation in pain centres that could not be paired , did not specify how randomisation was achieved , or adopted a non-randomised design .
Blinding of participants and personnel (i.e. performance bias), blinding of outcome assessment (i.e. detection bias), and incomplete outcome data (i.e. attrition bias) showed the highest risk of bias across studies. For example, nine studies did not report complete outcome data as stated in their respective protocols and/or methodology [34, 36, 39, 44, 53–55, 57, 58]. Allocation concealment (i.e. selection bias) was deemed high risk in three included studies [57–59] and techniques used for concealment were unclear for nine studies [33, 34, 36, 41, 43–45, 47, 50].
Reporting bias and other bias were generally low across all 27 studies. Based on the information outlined in study protocols, three studies showed selective outcome reporting [33, 35, 55]. Other than a potential conflict of interest in one study  and design issues in one non-randomised study , no additional sources of bias were identified.
Generally, the quality of reporting was satisfactory when assessed against the TIDieR checklist [66, 67] (Fig 3). On average, eight items were reported across all 27 included studies. One study  reported all 12 checklist items, while another  reported all 11 checklist items as it did not involve any modifications during the course of study. All included studies described five of the 12 checklist items (i.e. brief names; why; procedures; how; and when and how much) in their methods sections. Intervention location was the next best reported checklist item, with only three studies [34, 53, 54] not providing details of where they were conducted. Modifications were made and reported by three studies [33, 56, 59]. Planned and actual fidelity assessments were the least reported checklist items, with 14 studies not describing plans for determining fidelity [34, 36–40, 46, 47, 50, 52–54, 57, 59] and 16 studies not reporting if any fidelity assessment plans were followed through [34, 36, 37, 39, 40, 46, 47, 49, 50, 52–55, 57–59].
Summary of findings
Overall, education as part of MD interventions, either as the active intervention or the comparator, seemed to show positive outcomes for participants. These positive outcomes generally appeared to be sustained or improved at follow-up. Topics around physical activity and cognitive-behavioural strategies were most common. The majority of included studies offered take-home materials in the form of written supplements and homework to help participants re-visit content at their own convenience. Interventions were conducted as one-on-one and/or group weekly didactic sessions, targeting any type of chronic pain. MD teams across all included studies consisted mostly of physiotherapists, psychologists and nurses. However, variable risks of bias and heterogeneity in outcome measures were observed across all 27 included studies.
This systematic review identified that education, as part of MD interventions, appeared to contribute to an immediate amelioration in chronic pain management, with benefits seemingly sustained or improved long-term. This was despite the observed heterogeneity in: participant type and duration of chronic pain; study methods; outcome measures; and quality of reporting. Commonalities identified across the included studies provide important insights to all relevant stakeholders about potentially successful strategies for future adoption.
Common aspects across included studies and deductions
Most education sessions were run in groups, for a minimum of two hours each, over a four- to ten-week period. They incorporated direct, face-to-face interactions between educators and participants, and provided take-home materials and homework activities. These characteristics all seemed to optimise group outcomes and brought positive changes to participants’ lifestyles and quality of life over time, a finding similar to that reported in a review of cancer pain management . Duration of education sessions also aligned with previous recommendations of effective instruction time of 2.5 hours for low back pain .
In terms of skill sets of recruited healthcare professionals, physiotherapists, psychologists and nurses were the most common intervention facilitators, potentially indicative of a need to focus on education around physical activity, mental health (notably, cognitive behavioural strategies) and general lifestyle changes. Communication between participants and these three types of healthcare professionals is likely to reshape beliefs and behaviour of people with chronic pain, an outcome identified in previous research involving various healthcare professionals and health conditions such as low back pain and palliative care [28, 77–80].
Directions for future research
Moving forward, researchers should incorporate all fore-mentioned education characteristics in future RCTs of MD programs. Additionally, future RCTs should: expand on the MD team; use only validated outcome measures; extend follow-up periods to investigate the longitudinal effects on participants and health systems; and analyse the effectiveness of chronic pain management strategies in developing and non-developing countries.
1. Expanding the MD team.
Almost half of the included studies (n = 11) did not report on the full composition of their MD teams, thereby creating a lack of transparency around the exact range and diversity of professionals involved. Chronic pain, by its very nature, is a complex medical condition that could require a broad array of healthcare professionals for optimal management. Research on the impact of bigger and more diverse MD teams than those used historically are recommended. The immediate and long-term impact of such teams on participants (e.g. the inclusion of GPs, pain specialists, pharmacists, psychologists, physiotherapists, nurses, nutritionists, occupational therapists and social workers), can only be hypothesised at this stage, as highlighted by previous preliminary studies [25, 26].
Involvement of pharmacists in only two interventions was surprising and might indicate that these professionals are overlooked in chronic pain management, despite them being strategically placed in primary care to monitor, advise, support and refer consumers with chronic pain . Future RCTs exploring the role of pharmacists in MD chronic pain management, especially in terms of medication management and counselling, are warranted to further investigate the positive contributions of these professionals [25, 26].
A number of other healthcare professionals were overlooked in MD chronic pain management; occupational therapists, nutritionists and social workers were involved in only one study each. These healthcare professionals have an important role in providing assistance to patients with chronic pain. Occupational therapists can assist patients to reach optimal health and well-being through participation in everyday activities . Nutritionists can educate patients on the impact of food in reducing chronic inflammation, a symptom associated with chronic pain, as well as assist with optimal weight management . Social workers can establish, monitor and improve practice and ethical standards within MD teams, as well as advise patients on relevant and available services, such as subsidised healthcare plans . Future RCTs investigating MD teams that also involve these healthcare professionals, are warranted.
2. Using only validated outcome measures.
Instead of combined use of validated and non-validated outcome measures (as observed across most studies in this review), it is recommended that only evidence-based measures, such as the Pain Self-Efficacy Questionnaire (PSEQ), Assessment of Quality of Life-6D (AQoL-6D) and Patient Global Impression of Change (PGIC), be utilised in future RCTs. These represent reliable and consistent ways of assessing participants’ ability to physically and emotionally cope with and manage pain over time [74, 84–90]. Given that VAS, RMDQ and SF-36 detected significant changes [33–35, 37, 42, 43, 49, 51–54] in this review, these tools are recommended for similar interventions in the future. Consistent use of only validated outcome measures will facilitate comparison of outcomes across studies. Only then, will other systematic frameworks, such as Grading of Recommendations Assessment, Development and Evaluation (GRADE) , be used effectively to make clinical practice recommendations.
3. Extending follow-up periods to assess longitudinal effects.
This review has only reported positive, short-term (less than two years) lifestyle benefits of education, as part of MD programs, in individuals experiencing multiple types of chronic pain. Yet, longitudinal lifestyle, behavioural changes and economic impact (e.g. in the form of savings incurred to participants, health systems and the economy) over multiple years, remain other aspects to be investigated. The authors propose an extension of follow-up periods to analyse these effects.
4. Conducting RCTs in developing and non-developing countries.
Despite a high proportion of the population in developing countries reporting chronic pain [4, 92], this review has mostly considered studies from developed nations (n = 25). Local traditions, cultural background and/or local policies tend to vary among developed, developing and non-developing countries, and can significantly influence diagnosis and treatment of chronic pain in these areas [10, 92]. With an inevitably ageing population and a predictable rise in prevalence of chronic pain globally [5, 29], investigating the provision of cost-effective MD programs that incorporate education in developing and non-developing countries is another avenue for future research. Only then, can chronic pain be truly and effectively managed on a global scale.
Strengths and limitations
The initial aim of this review was to update the work of Scascighini et al. . Despite multiple attempts to contact the authors, a detailed search strategy of the 2008 review was not obtained. Consequently, an alternative up-to-date search strategy, aimed at better identifying relevant articles, was developed. While the total number of studies included in this work (n = 27) was not dissimilar to the 35 and 31 studies analysed by Scascighini et al. and Berube et al. respectively [27, 30], this work set out with a broader focus of exploring the effects of any type of education, delivered as part of MD interventions, on any chronic pain type. Inclusion of all possible forms of education was a strength of this review and addressed the authors’ original objective of reporting on the ideal combination of education methods to optimise self-management of chronic pain.
Another strength was the use of several databases to identify relevant studies, a strategy previously identified as broadening the scope of research and improving the quality of systematic reviews . Setting language limits to English, French, Dutch and German, rather than English alone, added to this strength by capturing a wider range of articles and increasing the generalisability of conclusions. Comprehensive assessments of methodological quality and risk of bias, as well as analysis of longitudinal effectiveness in relation to follow-up periods , further consolidated conclusions of this review. These strategies were seldom reported in earlier work [27, 30–32].
Conversely, heterogeneity of included studies was a limitation as it made application of the Cochrane Collaboration Risk of Bias tool [63–65] and the TIDieR model [66, 67] difficult. The comparative process proved especially challenging due to the diversity in individual studies in terms of: 1. inclusion criteria for participants, especially in terms of duration of their pain; 2. education strategies adopted; 3. the array of validated and non-validated or customised assessment tools utilised to monitor participant progress; and 4. the variety of healthcare professionals involved. Additional inherent limitations of the TIDieR tool also imply potential uncertainty in the determination of quality of intervention reporting and reproducibility of specified methods . As stipulated by Cotterill et al. (2018), TIDieR limits its quality and reproducibility assessments only to the context or setting specified by the individual study analysed, thereby failing to provide an overview of quality and reproducibility of similar studies in different contexts or settings over time .
Another limitation was restriction of the authors’ search strategy to only four databases and four languages, which implied that other relevant studies may have been missed. No grey literature was searched and it is likely that studies identifying negative effects of education may not have been reported.
Gaps in existing literature
Interpretation of results from this review calls for caution as variable risks of bias, uncertainties around quality, and lack of consistency (e.g. in terms of MD teams and outcome measures), were reported as shortcomings of included studies. There is an increased need for greater homogeneity in research methodology, such as use of intention-to-treat, adequate concealment and consistent use of only validated outcome measures. Detailed reporting of interventions to ease reproducibility, for example in terms of: type of professional recruited; type of materials used; and intervention adherence measures, is also required. Clarity around participant characteristics such as exact duration of chronic pain, will further increase reliability and generalisability of any findings promoting self-management and self-efficacy in people with this condition. Lastly, given the high proportion of studies from developed nations, conclusions from this review may not be generalised to people with chronic pain in developing and non-developing nations.
Participant education remains an understudied aspect of MD interventions focused on chronic pain management. Compared to previous work, this review has attempted to present reliable and comprehensive evidence around the effects of including education in such programs. MD programs that include four- to ten-week education around topics of physical activity, cognitive behavioural strategies and general lifestyle, generally point towards significantly positive results, particularly in relation to self-management practices and self-efficacy. Such programs should be adopted as part of a patient’s pain management plan in the future. However, some research gaps remain, including a lack of consistency in reporting participant chronic pain duration; use of standardised and validated outcome measures in combination with non-validated measures; and lack of homogeneity in research methodology. A need for more research that expands on the MD team, uses only recommended outcome measures, extends follow-up periods and is implemented in various developing and non-developing countries, is warranted.
S1 Appendix. Preliminary database search strategy.
S2 Appendix. Database search strategy.
We thank our librarians for their invaluable advice regarding the search strategy for this review.
Access Economics Pty Limited. The high price of pain: the economic impact of persistent pain in Australia. MBF Foundation; 2007.
- 2. Blyth F, March L, Brnabicc A, Jormd L, Williamson M, Cousins M. Chronic pain in Australia: a prevalence study. Pain. 2001;89: 127–34. pmid:11166468
International Association for the Study of Pain. Chronic Pain has arrived in the ICD-11. 2019. Available from: https://www.iasp-pain.org/PublicationsNews/NewsDetail.aspx?ItemNumber=8340&navItemNumber=643
- 4. Tsang A, Von Korff M, Lee S, Alonso J, Karam E, Angermeyer M, et al. Common chronic pain conditions in developed and developing countries: gender and age differences and comorbidity with depression-anxiety disorders. J Pain. 2008;9(10): 883–91. pmid:18602869
Pain Australia. Painful Facts. 2017. Available from: http://www.painaustralia.org.au/about-pain/painful-facts
- 6. Flor H, Fydrich T, Turk DC. Efficacy of multidisciplinary pain treatment centres: a meta-analytic review. Pain. 1992;49: 221–30. pmid:1535122
- 7. Gatchel RJ, Okifuji A. Evidence-based scientific data documenting the treatment and cost-effectiveness of comprehensive pain programs for chronic non-malignant pain. J Pain. 2006;7: 779–93. pmid:17074616
- 8. Karjalainen K, Malmivaara A, van Tulder M, Roine R, Jauhiainen M, Hurri H, et al. Multidisciplinary rehabilitation for fibromyalgia and musculoskeletal pain in working age adults. Cochrane Database Syst Rev. 2000; 2: CD001984.
- 9. Australia Pain. National Pain Strategy Pain Management for all Australians: National Pain Summit initiative. 2011. Available from: http://www.painaustralia.org.au/improving-policy/national-pain-strategy
- 10. Breivik H, Collett B, Ventafridda V, Cohen R, Gallacher D. Survey of chronic pain in Europe: Prevalence, impact on daily life, and treatment. Eur J Pain. 2006;10: 287–333. pmid:16095934
- 11. Campbell G, Darke S, Bruno R, Degenhardt L. The prevalence and correlates of chronic pain and suicidality in a nationally representative sample. Aust NZ J Psychiatry. 2015;49(9): 803–11.
- 12. Henschke N, Kamper SJ, Maher CG. The Epidemiology and Economic Consequences of Pain. Mayo Clin Proc. 2015;90(1): 139–47. pmid:25572198
- 13. Phillips CJ. The cost and burden of chronic pain. Reviews in Pain. 2009;3(1): 2. pmid:26526940
- 14. Kerr S, Fairbrother G, Crawford M, Hogg M, Fairbrother D, Khor KE. Patient characteristics and quality of life among a sample of Australian chronic pain clinic attendees. Intern Med J. 2004;34: 403–9. pmid:15271174
- 15. Moseley GL, Nicholas MK, Hodges PW. A randomized controlled trial of intensive neurophysiology education in chronic low back pain. Clin J Pain. 2004;20(5): 324–30. pmid:15322439
- 16. Berwick DM, Budman S, Feldstein M. No clinical effect of back schools in an HMO: a randomized prospective trial. Spine. 1989;14: 338–44. pmid:2523581
- 17. Aberg J. Evaluation of an advanced back pain rehabilitation program. Spine. 1984;9: 317–8. pmid:6233720
- 18. Bergquist-Ullman M, Larsson U. Acute low back pain in industry: a controlled prospective study with special reference to therapy and confounding factors. Acta Orthop Scand. 1977;1–117.
- 19. Dehlin O, Berg S, Andersson GB, et al. Effect of physical training and ergonomic counselling on the psychological perception of work and on the subjective assessment of low-back insufficiency. Scand J Rehabil Med 1981;13: 1–9. pmid:6455734
- 20. Goossens ME, Rutten-van Molken MP, Leidl RM, Bos SG, Vlaeyen JW, Teeken-Gruben NJ. Cognitive-educational treatment of fibromyalgia: a randomized clinical trial. II. Economic evaluation. J Rheumatol. 1996;23: 1246–54. pmid:8823700
- 21. Klaber Moffett JA, Chase SM, Portek I, Ennis JR. A controlled, prospective study to evaluate the effectiveness of a back school in the relief of chronic low back pain. Spine. 1986;11: 120–2. pmid:2939571
- 22. Lankhorst GJ, Van de Stadt RJ, Vogelaar TW, Van der Korst JK, Prevo AJ. The effect of the Swedish Back School in chronic idiopathic low back pain: A prospective controlled study. Scand J Rehabil Med. 1983;15(3): 141–5. pmid:6195731
- 23. Triano JJ, McGregor M, Hondras MA, Brennan PC. Manipulative therapy versus education programs in chronic low back pain. Spine. 1995;20(8): 948–55. pmid:7644961
- 24. Stankovic R, Johnell O. Conservative treatment of acute low-back pain. A prospective randomized trial: Mckenzie method of treatment versus patient education in “mini back school.” Spine. 1990;15: 120–3. pmid:2139241
- 25. King MA, Sav A, McSwan J. A pilot study of a multidisciplinary clinical pain programme provided by the Gold Coast Medicare Local. IJPP. 2015;23(S1): 6.
- 26. Joypaul S, Kelly F, King MK. Turning Pain into Gain: Evaluation of a Multidisciplinary Chronic Pain Management Program in Primary Care. Pain Med. 2018(pny241): 1–9.
- 27. Scascighini L, Toma V, Dober-Speilmann S, Sprott H. Multidisciplinary treatment for chronic pain: a systematic review of interventions and outcomes. Rheumatology. 2008;47: 670–8. pmid:18375406
- 28. Mitchell GK, Tieman JJ, Shelby-James TM. Multidisciplinary care planning and teamwork in primary care. Med J Aust. 2008;188(8 Suppl): S61–4. pmid:18429739
- 29. Barlow J, Wright C, Sheasby J, Turner A, Hainsworth J. Self-management approaches for people with chronic conditions: a review. Patient Educ Couns. 2002;48: 177–87. pmid:12401421
- 30. Berube M, Choinière M, Laflamme YG, Gelinas C. Acute to chronic pain transition in extremity trauma: A narrative review for future preventive interventions (part 2). Int J Orthop Trauma Nurs. 2017;24: 59–67. pmid:27527536
- 31. Geneen LJ, Martin DJ, Adams N, Clarke C, Dunbar M, Jones D, et al. Effects of education to facilitate knowledge about chronic pain for adults: a systematic review with meta-analysis. Syst Rev. 2015;4: 132. pmid:26428467
- 32. Liegl G, Boeckle M, Leitner A, Pieh C. A meta-analytic review of brief guided self-help education for chronic pain. Eur J Pain. 2016;20(10): 1551–62. pmid:27161217
- 33. Abbasi M, Dehghani M, Keefe F, Jafari H, Behtash H, Shams J. Spouse-assisted training in pain coping skills and the outcome of multidisciplinary pain management for chronic low back pain treatment: A 1-year randomized controlled trial. Eur J Pain. 2012;16(7): 1033–43. pmid:22337646
- 34. Anderson B, Strand LI, Råheim M. The effect of long-term body awareness training succeeding a multimodal cognitive behavior program for patients with widespread pain. J Musculoskelet Pain. 2007;15(3): 19–29.
- 35. Bair MJ, Ang D, Wu J, Outcalt SD, Sargent C, Gerwig A, et al. Evaluation of stepped care for chronic pain (ESCAPE) in veterans of Iraq and Afghanistan: A randomized trial. JAMA Intern Med. 2014;29: S89.
- 36. Basler HD, Jäkle C, Kröner-Herwig B. Cognitive-behavioral therapy for chronic headache at German pain centers. Int J Rehabil Health. 1996;2(4): 235–52.
- 37. Becker N, Sjøgren P, Bech P, Olsen AK, Eriksen J. Treatment outcome of chronic non-malignant pain patients managed in a Danish multidisciplinary pain centre compared to general practice: A randomised controlled trial. Pain. 2000;84(2–3): 203–11. pmid:10666525
- 38. Beltran-Alacreu H, López-de-Uralde-Villanueva I, Fernández-Carnero J, La Touche R. Manual Therapy, Therapeutic Patient Education, and Therapeutic Exercise, an Effective Multimodal Treatment of Nonspecific Chronic Neck Pain: A Randomized Controlled Trial. Am J Phys Med Rehabil. 2015;94(10): 887–97.
- 39. Burke AL, Denson LA, Mathias JL. Does a Brief Educational Session Produce Positive Change for Individuals Waiting for Tertiary Chronic Pain Services? Pain Med. 2016;17(12): 2203–17. pmid:28025355
- 40. Cherkin DC, Eisenberg D, Sherman KJ, Barlow W, Kaptchuk TJ, Street J, et al. Randomized trial comparing traditional Chinese medical acupuncture, therapeutic massage, and self-care education for chronic low back pain. Arch Intern Med. 2001;161(8): 1081–8. pmid:11322842
- 41. Corson K, Doak MN, Denneson L, Crutchfield M, Soleck G, Dickinson KC, et al. Primary care clinician adherence to guidelines for the management of chronic musculoskeletal pain: Results from the study of the effectiveness of a collaborative approach to pain. Pain Med. 2011;12(10): 1490–501. pmid:21943325
- 42. Cramer H, Lauche R, Hohmann C, Lüdtke R, Haller H, Michalsen A, et al. Randomized-controlled trial comparing yoga and home-based exercise for chronic neck pain. Clin J Pain. 2013;29(3): 216–23. pmid:23249655
- 43. Ersek M, Turner JA, Cain KC, Kemp CA. Results of a randomized controlled trial to examine the efficacy of a chronic pain self-management group for older adults. Pain. 2008;138(1): 29–40. pmid:18086516
- 44. Gallagher L, McAuley J, Moseley GL. A randomized-controlled trial of using a book of metaphors to reconceptualize pain and decrease catastrophizing in people with chronic pain. Clin J Pain. 2013;29(1): 20–5. pmid:22688603
- 45. Heutink M, Post MWM, Bongers-Janssen HMH, Dijkstra CA, Snoek GJ, Spijkerman DCM, et al. The CONECSI trial: Results of a randomized controlled trial of a multidisciplinary cognitive behavioral program for coping with chronic neuropathic pain after spinal cord injury. Pain. 2012;153(1): 120–8. pmid:22100355
- 46. Jay K, Brandt M, Jakobsen MD, Sundstrup E, Berthelsen KG, Schraefel M, et al. Ten weeks of physical-cognitive-mindfulness training reduces fear-avoidance beliefs about work-related activity: Randomized controlled trial. Medicine. 2016;95(34).
- 47. Keller S, Ehrhardt-Schmelzer S, Herda C, Schmid S, Basler H. Multidisciplinary rehabilitation for chronic back pain in an outpatient setting: A controlled randomized trial. Eur J Pain. 1997;1(4): 279–92. pmid:15102393
- 48. Kerns RD, Burns JW, Shulman M, Jensen MP, Nielson WR, Czlapinski R, et al. Can we improve cognitive-behavioral therapy for chronic back pain treatment engagement and adherence? A controlled trial of tailored versus standard therapy. Health Psychol. 2014;33(9): 938–47. pmid:24295024
- 49. Kristjánsdóttir OB, Fors EA, Eide E, Finset A, Stensrud TL, van Dulmen S, et al. A smartphone-based intervention with diaries and therapist-feedback to reduce catastrophizing and increase functioning in women with chronic widespread pain: randomized controlled trial. J Med Internet Res. 2013;15(3).
- 50. Martín J, Torre F, Padierna A, Aguirre U, González N, García S, et al. Six-and 12-month follow-up of an interdisciplinary fibromyalgia treatment programme: results of a randomised trial. Clin Exp Rheumatol. 2012;30(6 Suppl 74): 103–11. pmid:23261008
- 51. Nicholas MK, Asghari A, Blyth FM, Wood BM, Murray R, McCabe R, et al. Self-management intervention for chronic pain in older adults: A randomised controlled trial. Pain. 2013;154(6): 824–35. pmid:23522927
- 52. Pires D, Cruz EB, Caeiro C. Aquatic exercise and pain neurophysiology education versus aquatic exercise alone for patients with chronic low back pain: A randomized controlled trial. Clin Rehabil. 2015;29(6): 538–47. pmid:25200879
- 53. Ribeiro LH, Jennings MF, Jones A, Furtado R, Natour J. Effectiveness of a back school program in low back pain. Clin Exp Rheumatol. 2008;26(1): 81–8. pmid:18328151
- 54. Ryan CG, Gray HG, Newton M, Granat MH. Pain biology education and exercise classes compared to pain biology education alone for individuals with chronic low back pain: A pilot randomised controlled trial. Man Ther. 2010;15(4): 382–7. pmid:20359937
- 55. Smith N, Jordan M, White R, Bowman J, Hayes C. Assessment of Adults Experiencing Chronic Non-Cancer Pain: A Randomized Trial of Group Versus Individual Format at an Australian Tertiary Pain Service. Pain Med. 2016;17(2): 278–94. pmid:26814305
- 56. Sullivan MD, Turner JA, DiLodovico C, D'Appollonio A, Stephens K, Chan YF. Prescription Opioid Taper Support for Outpatients With Chronic Pain: A Randomized Controlled Trial. J Pain. 2017;18(3): 308–18. pmid:27908840
- 57. Turner-Stokes L, Erkeller-Yuksel F, Miles A, Pincus T, Shipley M, Pearce S. Outpatient cognitive behavioral pain management programs: a randomized comparison of a group-based multidisciplinary versus an individual therapy model. Arch Phys Med Rehabil. 2003;84(6): 781–8. pmid:12808527
- 58. Uebelacker LA, Weisberg RB, Herman DS, Bailey GL, Pinkston-Camp MM, Garnaat SL, et al. Pilot Randomized Trial of Collaborative Behavioral Treatment for Chronic Pain and Depression in Persons Living with HIV/AIDS. AIDS Behav. 2016;20(8): 1675–81. pmid:27115400
- 59. Chao MT, Hurstak E, Leonoudakis-Watts K, Sidders F, Pace J, Hammer H, et al. Patient-Reported Outcomes of an Integrative Pain Management Program Implemented in a Primary Care Safety Net Clinic: a Quasi-experiemental Study. J Gen Intern Med. 2019;34(7): 1105–7. pmid:30783878
- 60. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Br Med J. 2009;339: 332–6.
- 61. The Cochrane Collaboration. Cochrane Community Searching: Recording and Reporting. 2019. Available from: https://community.cochrane.org/organizational-info/resources/resources-groups/information-specialists-portal/searching-recording-reporting
Equator network. Enhancing the QUAlity and Transparency Of health Research. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Available from: www.equator-network.org/reporting-guidelines/prisma/
- 63. Training Cochrane. Assessing risk of bias in included studies. Available from: https://pdfs.semanticscholar.org/b2e5/4b74b82ab219236c8a595be2e0f7eb9836e1.pdf
- 64. Higgins JPT, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343: 1–9.
Higgins J, Green SM. Cochrane handbook for Systematic Reviews of Interventions: The Cochrane Collaboration; 2011.
The TIDieR (Template for Intervention Description and Replication) Checklist. Available from: www.equator-network.org
- 67. Hoffmann TC, Glasziou PP, Boutron I. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J. 2014;348: g1687.
- 68. Popay J, Baldwin S, Arai L, Britten N, Petticrew M, Rodgers M, et al. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews. Lancaster University. 2006. Available from: http://www.ccsr.ac.uk/methods/publications/documents/Popay.pdf
- 69. Hunt T, Wilson CJ, Caputi P, Woodward A, Wilson I. Signs of current suicidality in men: A systematic review. PLoS ONE. 2017;12(3): e0174675. pmid:28355268
Roland Morris Disability Questionnaire. Available from: www.rmdq.org
Physiopedia. Visual Analogue Scale. Available from: https://www.physio-pedia.com/Visual_Analogue_Scale
- 72. Stewart M. The Medical Outcomes Study 36-item short-form health survey (SF-36). Aust J Physiother. 2007;53(3): 208. pmid:17899676
- 73. Nicholas MK. The pain self‐efficacy questionnaire: Taking pain into account. Eur J Pain. 2007;11(2): 153–63. pmid:16446108
Nicholas MK. Self-Efficacy and Chronic Pain. Annual Conference of the British Psychological Society. 1989.
- 75. Bennett MI, Bagnall A, Closs SJ. How effective are patient-based educational interventions in the management of cancer pain: Systematic review and meta-analysis. Pain. 2009;143: 192–9. pmid:19285376
- 76. Engers AJ, Jellema P, Wensing M, van der Windt DAWM, Grol R, van Tulder MWC, et al. Individual patient education for low back pain. Cochrane Database Syst Rev. 2008;1: CD004057.
- 77. Kenny DT. Constructions of chronic pain in doctor-patient relationships: bridging the communication chasm. Patient Educ Couns. 2004;52: 297–305. pmid:14998600
- 78. Darlow B, Fullen BM, Dean S, Hurley DA, Baxter GD, Dowell A. The association between health care professional attitudes and beliefs and the attitudes and beliefs, clinical management, and outcomes of patients with low back pain: A systematic review. Eur J Pain. 2012;16: 3–17. pmid:21719329
The Royal Australian College of General Practitioners. The RACGP Curriculum for Australian General Practice. 2016. Available from: http://curriculum.racgp.org.au/statements/multidisciplinary-care
- 80. Rodondi P-Y, Dubois J, Bill A-S, Koutaissoff D, Ros J, Aveni E, et al. Primary care physicians' attitude and reported prescribing behavior for chronic low back pain: An exploratory cross-sectional study. PLoS ONE. 2018.
- 81. Occupational Therapy Australia. About Occupational Therapy. 2019. Available from: https://www.otaus.com.au/about/about-ot
Harvard Medical School. Can diet heal chronic pain? United States of America: Harvard Health Publishing; 2018. Available from: https://www.health.harvard.edu/pain/can-diet-heal-chronic-pain
- 83. Allied Health Professions Australia. Australian Association of Social Workers. 2017. Available from: https://ahpa.com.au/our-members/australian-association-of-social-workers/
- 84. Asghari A, Nicholas MK. Pain self-efficacy beliefs and pain behaviour. A prospective study. Pain. 2001;94(2001): 85–100.
- 85. Nicholas MK, Asghari A, Blyth FM. What do the numbers mean? Normative data in chronic pain measures. Pain. 2007;134(2008): 158–73.
- 86. Tay TG, Willcocks AL, Chen JF, Jastrzab G, Khor KE. A Descriptive Longitudinal Study of Chronic Pain Outcomes and Gender Differences in a Multidisciplinary Pain Management Centre. Pain Studies and Treatment. 2014;2: 56–9.
- 87. Bandura A. Self-Efficacy: Toward a Unifying Theory of Behavioural Change. Psychol Rev. 1977;84(2): 191–215. pmid:847061
- 88. Miles CL, Pincus T, Carnes D, Taylor SJC, Underwood M. Measuring pain self-efficacy. Clin J Pain. 2011;27: 461–70. pmid:21317778
- 89. Sarda JJ, Nicholas MK, Pimenta CAM, Asghari A. Pain-related self-efficacy beliefs in a Brazilian chronic pain patient sample: a pyschometric analysis. Stress Health. 2007;23: 185–90.
- 90. Richardson JR, Peacock SJ, Hawthorne G, Iezzi A, Elsworth G, Day NA. Construction of the descriptive system for the assessment of quality of life AQoL-6D utility instrument. Health Quality Life Outcomes. 2012;10(1): 1.
BMJ Best Practice. What is GRADE? 2019. Available from: https://bestpractice.bmj.com/info/us/toolkit/learn-ebm/what-is-grade/
- 92. Johnson MI, Elzahaf RA, Tashani OA. The prevalence of chronic pain in developing countries. Pain Manage. 2013;3(2): 83–6.
- 93. Rathbone J, Carter M, Hoffmann T, Glasziou P. A comparison performance of seven key bibliographic databases in identifying all relevant systematic reviews of interventions for hypertension. Syst Rev. 2016;5(27).
- 94. Salkind NJ. Follow-Up. Encyclopedia of Research Design. 2010.
- 95. Cotterill S, Knowles S, Martindale A-M, Elvey R, Howard S, Coupe N, et al. Getting messier with TIDieR: embracing context and complexity in intervention reporting. BMC Med Res Methodol. 2018;18(12): 1–10.