https://orcid.org/0009-0005-4020-6112
Figures
Abstract
Objective
To compare the treatment effect of five electrical stimulation methods commonly used in the treatment of stroke patients with lower limb dysfunction.
Methods
We implemented a systematic search of 3915 studies published up to January 2023 from eight databases and two clinical trial registries. First, two independent reviewers critically evaluated trial eligibility according to the inclusion and exclusion criteria. Next, they selected and extracted data. Then, they assessed the risk of bias. Pairwise meta-analysis and Bayesian network meta-analysis were conducted to estimate the effectiveness and ranking of the five electrical stimulation methods.
Results
A total of 33 trials with a final total of 2246 subjects were included in the analysis. By combining the comprehensive Rehabilitation Treatment (RT), the treatment effects of using five electrical stimulation methods were surperior to those of using RT only. In the meantime, RT+transcranial Direct Current Stimulation(tDCS) and RT+Functional Electrical Stimulation(FES) could be the optimal electric stimulation schemes for restoring lower limb motor function(SMD 8.35, 95%CI [3.05, 13.34]/ SMD 5.64, 95%CI [3.68, 7.56]), improving balance (SMD 9.80, 95%CI [0.67, 20.93]/ SMD 6.54, 95%CI [3.85, 10.95]) and activities of daily living(SMD 18.95, 95%CI [0.401, 36.9]/ SMD 15.47, 95%CI [7.89, 22.75]), and the treatment effects would be even better using RT+FES+tDCS combination.
Citation: Fang Y, Li J, Liu S, Wang Y, Li J, Yang D, et al. (2023) Optimization of electrical stimulation for the treatment of lower limb dysfunction after stroke: A systematic review and Bayesian network meta-analysis of randomized controlled trials. PLoS ONE 18(5): e0285523. https://doi.org/10.1371/journal.pone.0285523
Editor: Victor Afamefuna Egwuonwu, Nnamdi Azikiwe University, NIGERIA
Received: February 15, 2023; Accepted: April 25, 2023; Published: May 11, 2023
Copyright: © 2023 Fang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: Supported by “Science and Technology Development Fund of Hospital of Chengdu University of TCM[No.19MZ18]” and “Science and Technology Development Fund of Hospital of Chengdu University of TCM[No.21YS02]”. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Stroke is a disorder of cerebral blood circulation that can lead to neurological deficits [1, 2]. Previous studies indicated that the proportion of stroke in the global burden of the disease will increase year by year [3]. Stroke holds the characteristics of high incidence rate, high disability rate, high mortality, high recurrence rate, high economic burden and so on [4], which has led to a growing number of elderly people being disabled [5]. Stroke-induced lower limb dysfunction mainly affects the walking ability of patients, 63% of patients lose walking ability in the early post-stroke period and 22% cannot walk independently even after clinical and comprehensive rehabilitative interventions [6]. Lower limb dysfunction not only seriously impacts patients’ daily life but also brings serious mental and economic stress to patients and their families [7]. Therefore, establishing scientific and effective rehabilitation treatment schemes are crucial for such patients.
With the increasing understanding of lower limb dysfunction after stroke, various rehabilitation therapies have been applied to restore motor function, including traditional rehabilitation therapies such as occupational therapy, exercise therapy, and mirror therapy [8]. Emerging technologies like virtual reality [9], brain-computer interface technology [10], and intelligent robot training [11] have also been adopted in rehabilitation therapy. Nevertheless, few studies have demonstrated the clinical effectiveness of the above new methods. In addition, due to the lengthy rehabilitation process, economic benefits have become a high priority for clinicians and patients when choosing treatment methods [12].
In recent years, electrical stimulation has been widely used to improve limb function after stroke through clinical practice [13]. Considerable clinical trials have been implemented on electrical stimulation for the treatment of lower limb dysfunction after stroke [13]. The frequently-used electrical stimulation methods in the treatment of lower limb dysfunction after stroke include transcranial Direct Current Stimulation (tDCS), Neuromuscular Electrical Stimulation (NMES), Functional Electrical Stimulation (FES), Transcutaneous Electrical Nerve Stimulation (TENS), and Transcutaneous Electrical Acupoint Stimulation (TEAS) [14]. TENS is an electrical stimulation method adopting transdermal output pulse current to efficiently relieve the pain and stimulate the sensory impulses, thus improving muscle strength and motor function, meanwhile reducing spasticity [15] TEAS is a method of electrical stimulation of acupuncture points by TENS under the guidance of the meridian and acupoint theory of Chinese medicine [16]. NMES typically applies higher frequencies (20–50 Hz) current to promote muscle strength and relieve spasm symptoms [17]. FES is the most commonly used electrical stimulation methods for the treatment of lower limb dysfunction, and its basic principle is the simultaneous or intermittent use of electrical stimulation combined with functional tasks [18], from which a series of FES-based rehabilitation therapies have been derived. The tDCS is the only electrical stimulation method whose stimulation site is in the head, and its mechanism may be that it has short-term and long-term effects on cortical excitability and neuroplasticity [19]. In our previous study [20], the effectiveness of the above-mentioned five electrical stimulation methods in the treatment of upper limb dysfunction after stroke has been evaluated. In the follow-up work, we found that considerable efforts have been devoted to systematic review and meta-analysis on the effects of the abovementioned methods in the treatment of lower limb dysfunction after stroke. Nascimento et al [21] compared the effects of ankle-foot orthosis and FES on patients’ walking speed. Bai et al [22] investigated the ability of tDCS to restore motor function in patients’ lower limbs. Hong et al [23] investigated the effects of NMES on lower limb motor function after stroke. So far, nevertheless, the previous studies have been dedicated to exploring the effectiveness of a single electrical stimulation method, and few effort has been made to the comprehensive comparative analysis of various electrical stimulation methods). To address this issue, we adopted network meta-analysis [24] to select subjects with ischemic or hemorrhagic stroke and lower limb dysfunction based on high-quality randomized controlled trials. Combined with routine comprehensive rehabilitation therapy (RT), 5 outcome measures (FMA-LE, BBS, MBI, CSS and 10mMWS) were applied to evaluate the treatmnet effect of five different electrical stimulation schemes (RT+FES, RT+NMES, RT+TENS, RT+TEAS, RT+tDCS) on lower limb dysfunction after stroke, and the guiding significance of the results to clinical practice was discussed.
Methods
This study followed the PRISMA-NMA guidelines [25] (shown in S1 Table). The study has been registered in the Open Science Framework (registration DOI:10.17605/OSF.IO/F3G5Q).
Search strategy
We conducted an exhaustive online search for eligible studies by setting the retrieval time from the establishing date of each database to January 7th, 2023. The literature language was limited within English and Chinese. We searched in eight electronic databases, including China National Knowledge Infrastructure (CNKI), VIP Database for Chinese Technical Periodicals (VIP), WAN FANG Database (WF), Chinese biomedical literature service system (SinoMed), PubMed, Web of Science (WOS), Embase, and Cochrane Library. The clinical trial registries consisted of the International Standard Randomized Controlled Trial Number Register (ISRCTN) and the Chinese Clinical Trial Registry (ChiCTR). The MeSH terms used in this study included: Hemiplegia, Paralysis, Clinical trials as topic, Stroke, Electric Stimulation et al. We set four categories of free words, including (1) Lower limbs, leg, foot; (2) Motor function, Hemiplegia, Dysfunction; (3) Randomized controlled trial, controlled clinical trial, clinical trials; and (4) Stroke, Cerebrovascular Accident, Brain Vascular Accident, etc. For an example, the PubMed search strategies is shown in Table 1, and the search iterms were appropriately adjusted to meet the requirements of each database in order to ensure the basic logical integrity of the search.
Inclusion and exclusion criteria
Study type.
We screened studies that strictly met the design requirements for randomized controlled trials, including peer-reviewed journals in Chinese or English, and master and doctoral theses. Conference articles, newspaper articles, or book abstracts were excluded. We tracked studies retrieved in the Clinical Trials Registry Platform and excluded studies that were still in progress or had incomplete trial data. In addition, the treatment effects of interventions were easily exaggerated and may result in false positives due to the lack of validity of small trials that included a small number of patients [26]. In contrast, some small sample pretests usually adjusted the trial protocol or even did not conduct a formal trial in case of unsatisfactory trial results. Hence, aiming at guaranteeing the overall quality of the included studies and decreasing the bias of the present study, we excluded studies with total sample size less than 30.
Type of subjects.
The stroke diagnostic criteria included in RCTs need to record, and the participant should meet the diagnosis of ischemic or hemorrhagic stroke with definite lower limb dysfunction. Other systemic diseases or various causes of lower limb dysfunction were excluded. In addition, this study did not restrict the age, sex, race, or course of disease of the patients.
Type of interventions.
Since the kinds of electrical stimulation and rehabilitation treatments are various, we established strict inclusion criteria for interventions to obtain accurate literature screening results: (1) Experimental group: electrical stimulation methods including TENS, TEAS, NMES, tDCS, and FES were used combined with RT. The five electrical stimulation methods can be used separately or in combination with no restrictions on the dose, frequency, duration, or site of stimulation. (2) Control group: five electrical stimulation methods (same as the experimental group), RT, and Sham Stimulation (SS) were adopted in control group. The above interventions can be used separately or in combination. Because of the uncontrolled progression of stroke patients during rehabilitation and the complexity of their condition, the rehabilitation therapies used in each RCT were not identical. Based on the authoritative guidelines [8, 27] and clinical practice experience, the scope of RT was determined as: (1) conventional comprehensive rehabilitation techniques: exercise therapy, occupational therapy, rehabilitation training, rehabilitation education, and functional exercise; (2) conventional stroke care; (3) conventional stroke pharmacotherapy. It should be noted that we excluded RCTs using a single rehabilitation technique (e.g., mirror therapy, walking training, etc.) as an intervention, because the clinical efficacy of using different electrical stimulation methods based on comprehensive RT was the research priority of this study.
Type of outcome measures.
The primary outcome measure of this study was the Fugl-Meyer Assessment for Lower Extremity (FMA-LE), and secondary outcome measures included the Modified Barthel Index (MBI), Berg Balance Scale (BBS), 10m Maximal Walking Speed (10mMWS), and Composite Spasticity Scale (CSS). The FMA-LE is a widely recommended and used reliable scale for assessing lower limb motor deficits after stroke and can be utilized as a reliable tool for comprehensive evaluation of lower limb motor function [28]. BBS is a reliable and valid clinical scale that is often employed to assess patients’ balance. MBI is used to evaluate the improvement of patients’ daily living ability and the ability to function independently. The scales frequently-used to evaluate patients’ increased muscular tension include MAS and CSS. For lower limbs, CSS can effectively reflect the state of ankle plantar flexor tension compared with MAS [29]. Considering that walking ability is an important factor in measuring lower limb motor function [30], 10mMWS was included as an assessment index.
Study selection
First, two trained reviewers (YF and JL) independently screened the titles and abstracts according to inclusion criteria. Then, two reviewers assessed the full texts of potentially eligible studies. We marked the studies that were not accessible or the data was incomplete and we contacted the researcher by email. Finally, we excluded these studies after being unable to contact the researchers for three consecutive times or confirming that the studies were not accessible. Additionally, if there were divergences between the two reviewers, the third professional reviewer (QW) would intervene in for further evaluation.
Data process and analysis
Data extraction was conducted by two independent reviewers (YW and SL). The third reviewer (QW) would intervene in if there were divergences in the data extraction process. The extracted data included study title, author name, publication date, country, disease course, sample size, gender, age, intervention, treatment period, and outcome indicators (mean and standard deviation), etc. If the interventions of different groups in the multi-arms study are the same (different courses of treatment, frequencies, etc.), we will combine the data of the two groups. Meanwhile, the two reviewers also sorted out the parameter setting information of different electrical stimulation methods in each study.
Review Manager (Revman V5.3) and Aggregate Data Drug Information System (ADDIS V1.16.8) were used to conduct the meta-analysis. Pairwise Meta-Analysis was employed to compare two interventions comprehensively. In this study, Standard Mean Differences (SMD) was used for continuous outcomes, 95% represented confidence intervals (CIs). If there was no significant heterogeneity (I2<50%), the fixed effect model would be used. Whereas if there existed obvious heterogeneity between studies (I2≥50%), the random effect model would be adopted. NMA was performed in ADDIS using the Markov chain Monte Carlo method. NMA network plots for the five outcome measures were generated in Stata software (V16.0 MP). In a network, the larger the node represents the more studies using this intervention method. The thicker the lines between nodes represents the more comparative studies involving the two intervention methods. For any possible scenario, NMA was performed only when different interventions were connected in a network (There were direct or indirect links between different interventions). Furthermore, the node-splitting method was conducted to divide the evidence for each comparison of different interventions into direct and indirect evidence to evaluate local inconsistency. The internal consistency of the evidence network determined the validity of the NMA results, and the sources of direct and various indirect evidence should be consistent [24]. The segmented node method was used in this study to test the inconsistency in the NMA. Finally, the different electrical stimulation methods were ranked using the surface under the cumulative ranking curve (SUCRA).
Risk of bias assessment
The Cochrane risk of bias tool was used to assess the bias risk in RCTs by two reviewers (DY and JL). The assessment items included: (1) Allocation concealment (selection bias); (2) Random sequence generation (selection bias); (3) Blinding of outcome assessment (detection bias); Blinding of participants and personnel (performance bias); (4) Selective reporting (reporting bias); Incomplete outcome data (attrition bias); (5) Other bias. The assessment results would be classified into three categories, including high risk (H), low risk (L), and unclear risk (N). If two or more items were assessed as high risk, the trial would be considered as high risk. If all items of one trial were assessed as low risk or less than three items were of unclear risk, then the trial would be considered as low risk. The rest of the trials were classified as unclear risk [31].
Other assessment and analysis
We evaluated the publication bias by funnel plots generated by Stata software. At the same time, we evaluated the robustness of each result by sensitivity analysis. In addition, we adopted the Grade approach to evaluate the certainty of evidence. Finally, we confirmed that patients were not involved in the design, implementation, reporting, or dissemination of this study.
Results
Literature study
In this study, we identified and screened titles and abstracts of 3915 studies from 8 databases and 2 clinical trial registries. After the duplication check and preliminary screening by two reviewers, a total of 225 studies meeting the basic requirements were reviewed in full text. Through further screening, 33 RCTs [29, 32–63] meeting all the inclusion criteria were final included in this study. The screening process is illustrated in Fig 1.
In the 33 RCTs included for further analysis, a total of 2246 subjects participated in the trials, of which 51 subjects withdrew for various reasons. Most studies clearly described the type of stroke (Cerebral Infarction or Cerebral Hemorrhage) in the baseline data [29, 32, 33, 35–42, 44–52, 54, 56–61] and the course of stroke [29, 33–44, 46–48, 50, 51–62]. Only 10 trials described the specific stroke stages [29, 36, 42, 50, 52] and Brunnstrom stage [32, 35, 46, 47, 59]. Most of trials provided an accurate description of the age of the subjects (mean age range: 45.10–75.64 years) as well as the gender ratio at the baseline period. Of all 33 trials, a total of 5 trials [29, 32, 37, 48, 61] had three intervention groups with a sample size ratio of 1:1:1, and the remaining studies had two intervention groups with a sample size ratio of 1:1. The details of interventions in included studies can be found in S2 Table.
We compared five different electrical stimulation methods in this study. At present, there is no uniform standard for the parameter setting of electric stimulation therapy, so the treatment dose, treatment frequency, treatment course and stimulation site of electric stimulation were not limited in the inclusion standard. The detailed parameter settings of different electrical stimulation methods are shown in S3 Table. Seven studies [33, 35, 41, 45–47, 52] did not record specific parameters, and most of studies had a dose range of 30–100 Hz for electrical stimulation. One study [50] increased the stimulation dose of NMES to 200 Hz, and another study [40] used a low stimulation dose of NMES at 1 Hz. The stimulation site in all studies was in the affected lower limb, except for four studies [32, 54, 58, 61] using tDCS in which the stimulation site was in the hemiplegic lower limb representative area of the motor cortex on the surface of the head, and one study [45] using TENS in which the stimulation site was on the fourth lumbar vertebra. The course of electrical stimulation in most studies was 20 mins or 30 mins, and the treatment frequency was mainly once a day, and 5 times per week.
Risk of bias assessment
The results of bias risk assessment are depicted in Figs 2 and 3, and the detailed assessment results are presented in S4 Table. The results involved 9 studies that had high-risk items, 3 studies of which [53, 56, 62] did not describe random sequence generation methods, 4 studies of which [29, 48, 57, 61] were not blinded to subjects and study personnel, one study of which [62] was not blinded to the result evaluation and analysis personnel, and one study of which was considered otherwise biased due to missing baseline data. Moreover, it can be clearly seen that none of the high-risk bias trial was related to the following 3 assessment items: Allocation concealment (selection bias), Incomplete outcome data (attrition bias), and Selective reporting (reporting bias). In general, only one study [62] was classified as high-risk overall bias, accounting for approximately 3% of all studies. In addition, the results of the certainty of evidence (the Grade approach) are shown in S5 Table.
Pairwise Meta-Analysis
The comparison results between two interventions using Pairwise Meta-Analysis are presented in S1 Fig. Meanwhile, the results of 5 outcome measures are summarized in Table 2 with the intervention groups having meaningful comprehensive effects highlighted in bold. We found that no matter what outcome measures was used, the treatment effect of FES, TENS, TEAS, and NMES combined with RT was superior to that of RT or RT combined with SS. When we used FMA-LE to evaluate the trial results, RT combined with TENS was less effective than RT combined with TEAS. Furthermore, the comparison results adopting BBS indicated that the treatment effect of RT combined with both FES and tDCS surpassed that of RT combined with both FES and SS on restoring the balance function.
Network meta-analysis
Fig 4(A)–(4E) depict the network structures of five outcome measures (FMA-LE, BBS, MBI,10mMWS, CSS).
(A) FMA-LE, (B) BBS, (C) MBI, (D) 10mMWS, and (E) CSS.
A total of 25 studies [32–42, 44–47, 49, 51–57, 59, 60] involving 1815 subjects used the FMA-LE; BBS was the measure for 11 trials [32, 35, 37, 39, 41, 42, 47, 56–58, 60] with 784 patients; MBI was adopted to 13 trials [32, 35, 37, 39, 41, 42, 47, 56–58, 60] with a total of 878 patients; 10mMWS was the outcome measure for 8 trials [35, 39, 43, 44, 58, 61, 62, 63] involving 466 patients; CSS was the outcome measure used for 5 trials [29, 37, 48, 57, 60] with 237 patients. As shown in Fig 4(A) and 4(B), the network structures among the interventions evaluated by FMA-LE and BBS were similar, while FMA-LE established one more association between TEAS, NMES, and FES. The results indicated that FMA-LE and BBS were the most frequently-used outcome measures for the evaluation of 8 interventions. As can be seen in Fig 4, no matter what outcome measure was used, the number of trials comparing RT with FES+RT was the largest. The CSS (shown in Fig 4(E)) only involved the comparison of 4 interventions and only covered two electrical stimulation methods (FES and TEAS). Additionally, there was no complete association established in the network plot of 10 mMWS (shown in Fig 4(D)), and there was no direct or indirect association between the two sets of interventions ((RT+FES, RT+TENS, RT) and (RT+tDCS, RT+FES+SS, RT+tDCS+FES)). Therefore, the network of the two sets of interventions were analyzed separately when performing the NMA analysis.
The NMA results of the inconsistency test showed that the direct or indirect comparison of each segment node was not statistically significant(P>0.05), further indicating that there was no evidence of design inconsistency. The convergence of the model was verified under the condition that the potential scale reduction factor was 1 (shown in S6 Table).
Figs 5–7 show the NMA results, and the intervention groups with significant combined effects are highlighted in bold. As illustrated in Fig 5 (FMA-LE and BBS), the treatment effects of FES, NMES, TEAS, tDCS, and tDCS+FES combined with RT were superior to that of RT in terms of improving motor function of the subjects’ lower limbs. Meanwhile, the treatment effect of FES+ tDCS was better than that of SS. FES, tDCS, and tDCS+FES combined with RT outperformed RT in improving the patients’ balance. When it comes to improving MBI, the treatment effects of FES, NMES, tDCS, tDCS+FES combined with RT were more desirable than that of RT, and the treatment effect of tDCS was significantly excellent than that of TEAS (shown in Fig 6). Moreover, the treatment effect of the combination of FES and TEAS with RT was better than that of RT in relieving patients’ lower limb spasticity (CSS). It can be seen from Fig 7 (10mMWS) that only the combination of FES and RT improved walking ability better than RT, while no remarkable difference has been shown in the treatment effect among other interventions.
Notes: The bold values indicates a statistical difference.
Notes: The bold values indicates a statistical difference.
Notes: The bold values indicates a statistical difference.
In this study, we used a consistent model and adopted ADDIS to comprehensively rank the various interventions included in NMA. The ranking value for each intervention indicated its probability. As shown in S2 Fig, tDCS+FES combined with RT was most effective both in terms of improving FMA-LE scores, BBS and MBI, followed by tDCS and FES. FES performed best in relieving lower limb spasticity. Since the six interventions involving 10mMWS did not form a complete network structure, the two independent links in the network were ranked separately. The results indicated that based on RT, FES was superior to RT+TENS in improving patients’ walking ability, whereas tDCS outperformed FES+SS and tDCS+FES. The SUCRA scores are presented in S7 Table.
Sensitivity analysis
After excluding the studies with high-risk bias and the studies with sample sizes less than 40 or drop-out rate more than 15%, we carried out sensitivity analysis of all pairwise meta-analyses, and the results remained unchanged. When we excluded the studies with drop-out rate exceeding 15%, the results of sensitivity analysis showed that the treatment effect of FES+RT was better than that of tDCS+RT in improving FMA-UE(SMD 0.03, 95%CI [-0.61,0.67]) and MBI(SMD -0.10, 95%CI [-0.74,0.55]).
Publication bias
We employed funnel plots to evaluate the publication bias. Owing to the small number of included studies, only CSS and 10mMWS were not evaluated by using funnel plots. As shown in these figures (Figs 8–10), most of the studies were distributed in the funnel (95% confidence interval). There were a small number of studies out of the 95% confidence interval, demonstrating that the potential heterogeneity did exist in these studies. (Figs 8 and 10). Due to the limited sample size of the included RCTs, most studies were distributed in the lower-middle part of the funnel plot. In addition, the missing angle of BBS (Fig 9) on the left side of the red vertical line (odds ratio = 0) may be relevant to the unpublished studies with negative results.
Adverse events
In this paper, a total of 4 RCTs had adverse events. Among these RCTs, 3 RCTs [29, 48, 63] had adverse events of patient recurrent stroke, involving 5 patients and 4 interventions (RT, RT+SS, RT+FES, RT+TEAS). In one trial [29], one patient had gastric bleeding after using RT+FES. In another RCT [51], one patient had right lower limb thrombosis using RT alone.
Discussion
Main findings
In this study, we comprehensively searched studies from 8 databases and 2 clinical trial registries. Then, we performed a Pairwise Meta-Analysis and NMA on the 33 RCTs included in this study. Five frequently-used electrical stimulation methods for patients with limb dysfunction after stroke were selected in the analysis, including FES, TENS, TEAS, NMES, and tDCS. This is the first systematic review and NMA on different electrical stimulation methods in the treatment of lower limb dysfunction after stroke.
The NMA results demonstrated that the electrical stimulation methods combined with RT were more effective than RT. It can be indicated that electrical stimulation methods were capable of improving motor function of the lower limbs. Nevertheless, the treatment effect varied with different electrical stimulation methods. It can be deduced from the results that tDCS combined with FES was more effective in improving FMA-LE, BBS as well as MBI. Meanwhile, the treatment effect of tDCS or FES alone was second only to that of tDCS+FES, implying that these two electrical stimulation methods were optimal in treating lower limb dysfunction after stroke, and their combination can achieve more desirable results. FES is an electrical stimulation method by activating skeletal muscle with a constant frequency stimulation sequence [64], which has been developed as a popular treatment for lower limb dysfunction. FES has gradually been developed from a treatment method using electrophysiologically assisted devices to a treatment procedure that was capable of improving muscle control and residual motor nerve function [65]. tDCS is the only one of the five electrical stimulation methods with the stimulation site in the head. tDCS is a non-invasive neuromodulation technique [66], and its mechanism is promoting adaptive neuroplasticity [67]. Some researchers claimed that tDCS was beneficial for the recovery of motor function of patients with acute, subacute, or chronic stroke [68] under the premise of ensuring safety [69]. However, recent evidence indicated that tDCS cannot make any difference to the leg function, muscle strength, and cognitive function of patients after stroke [70]. In conclusion, although tDCS combined with RT showed significant advantages, the treatment effect of tDCS still needed to be further validated in large sample clinical controlled trials, considering the strict limitations of the interventions in this study.
In this paper, except for the stimulation areas of TENS and TEAS, the interventions of TENS and TEAS were almost the same. TENS stimulated the skin in the area of motor dysfunction, while TEAS stimulated acupuncture points. The previous studies have shown [71, 72] that TENS was beneficial to improving spasticity, muscle strength and gait capacity of stroke patients by regulating the spasticity based on various mechanisms, such as by increasing presynaptic inhibition or decreasing the excitability of the stretch reflex [73]. The results of NMA and FMA-LE ranking (S2 Fig) indicated that the treatment effect of TEAS was better than that of TENS, implying that the combined use of acupuncture treatment can improve the efficacy of electrical stimulation. Although the specific mechanism has not been found yet, the exploration of the effect of combined use of other electrical stimulation methods based on meridian-acupoint theory would be a very promising research direction. Furthermore, NMES was superior to TEAS in improving both FMA-LE and MBI scores, but inferior to TEAS in improving balance function (shown in S2 Fig). Some researchers believed that the actual clinical effectiveness of NMES depended on the systematic treatment scheme [74]. Additionally, the limitations of NMES for the recovery of motor function may be related to the recruitment of motor units during stimulation [75, 76]. It is worth noting that TEAS was not as good as SS in reducing CSS scores, which may result from the bias caused by only including one relevant study [48] (shown in Fig 6).
Overall quality of evidence
In this study, 33 RCTs were included through careful selection, with a total of 2246 patients participating in the trial among these RCTs. Among these trials, only 4 trials reported adverse events, and the main adverse event was recurrent stroke, involving a total of seven patients and two electrical stimulation methods. However, there was no evidence that the adverse events were induced by electrical stimulation. Generally, electrical stimulation methods were safe. The appropriate population and the specific side effects still need to be explored in long-term clinical practice and high-quality clinical trials.
Due to the specificity of the electrical stimulation method, it is hard to realize double blindness in the intervention process. Unlike the drug RCTs, the lack of blindness is an inherent limitation of non-pharmacological studies involving interventions with physiotherapy methods [77] and is also considered as one of the main limitations of this study. We found that most of the trials missed assessment items after being evaluated through the Cochrane Collaboration risk of bias tool. The uncertain risk of bias may lead to insufficient effectiveness [78]. Furthermore, due to the limitation of the number of studies, the Grade approach may not be able to fully reflect the quality of evidence. Moreover, we evaluated the publication bias by funnel plots (shown in Figs 8–10), while we did not evaluate CSS and 10mMWS due to limitations in the number of trials. The horizontal lines of the funnel plots for the three outcome measures (depicted in Figs 8–10) were skewed, indicating the publication bias. On the one hand, the asymmetric funnel plot may be related to unpublished negative results [79, 80]. On the other hand, this result may also cause by some ongoing trials with unpublished data. Meanwhile, if some trial data can hardly be determined to be normal distribution, the quartile and the median or the minimum/maximum of the median would not be converted into the mean and standard deviation, thus affecting the publication bias to some degree. Base on the above reasons, we carried out the sensitivity analyses, and the results showed that most of the results were reliable.
Strengths and limitations
The strengths of the present study can be summarized as follows. First, we conducted a comprehensive search and selection on 8 databases and 2 clinical trial registries according to PRISMA-NMA [25] and PRISMA guidelines and checklist [81]. Moreover, the rigorous data analysis approaches adopted in this study including NMA and Pairwise Meta-Analysis ensured the reliability of the final results. Secondly, this is a continuation study in which the methodology was extended and improved from the previous study [20]. In addition, this was the first academic research to compare and investigate the treatment effect of different electrical stimulation methods combined with RT for lower limb dysfunction after stroke.
Nevertheless, the limitations of this study can be concluded as follows. First of all, it can be seen from the baseline data of RCTs included, the course of some patients was significantly different. The same interventions may have different treatment effect on the stroke patients at different stage, thus influencing the comparison results of various electrical stimulation schemes. Secondly, the 5 three-arm trials [29, 32, 37, 48, 61] were divided into three comparison groups to compare different electrical stimulation schemes, but the sample size of each group was small, which may increase the risk of bias in this study. Finally, the differences in parameter settings may make a difference in the final treatment effect, so we sorted out the parameter details of various electrical stimulation methods in our preliminary work (S3 Table) and described the parameter details in results part Since there is still no parameter standard for different electrical stimulation methods on different diseases, this issue cannot be resolved temporarily.
Conclusions
Compared with the method only using RT, the comprehensive treatment schemes combined with electrical stimulation methods presented remarkable superiority in the treatment of lower limb dysfunction after stroke. Meanwhile, the relatively high comprehensive ranking of TDCS or FES could provide new ideas for clinical treatment. Moreover, TEAS combined with acupuncture points exhibited greater treatment potential than conventional TENS. The results of this study provided a basis for further application of electrical stimulation methods. Owing to the limitation of quality and quantity of the included studies, high-quality RCTs are extremely need to offer powerful evidence to further support the results.
Supporting information
S1 Table. The PRISMA network meta-analysis checklist.
https://doi.org/10.1371/journal.pone.0285523.s001
(DOCX)
S2 Table. Main characteristics of included RCTs.
https://doi.org/10.1371/journal.pone.0285523.s002
(DOCX)
S6 Table. Assessment of convergence of the model results.
https://doi.org/10.1371/journal.pone.0285523.s006
(DOCX)
S7 Table. Treatment ranking and SUCRA for outcome data.
https://doi.org/10.1371/journal.pone.0285523.s007
(DOCX)
S1 Checklist. The PRISMA network meta-analysis checklist.
https://doi.org/10.1371/journal.pone.0285523.s010
(DOCX)
References
- 1. Ding J, Zhang Y, Wang CX, et al. Dual-modality imaging of endothelial progenitor cells transplanted after ischaemic photothrombotic stroke. Life Sci. 2019;239:116774. pmid:31689438
- 2. El-Koussy M, Schroth G, Brekenfeld C, Arnold M. Imaging of acute ischemic stroke. Eur Neurol. 2014;72(5–6):309–316. pmid:25323674
- 3. Menken M, Munsat TL, Toole JF. The global burden of disease study: implications for neurology. Arch Neurol. 2000;57(3):418–420. pmid:10714674
- 4. GBD 2019 Stroke Collaborators. Global, regional, and national burden of stroke and its risk factors, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Neurol. 2021;20(10):795–820. pmid:34487721
- 5. Dorrance AM, Fink G. Effects of Stroke on the Autonomic Nervous System. Compr Physiol. 2015;5(3):1241–1263. pmid:26140717
- 6. Jang SH. The recovery of walking in stroke patients: a review. Int J Rehabil Res. 2010;33(4):285–289. pmid:20805757
- 7. Ashford S, Brown S, Turner-Stokes L. Systematic review of patient-reported outcome measures for functional performance in the lower limb. J Rehabil Med. 2015;47(1):9–17. pmid:25314949
- 8. Winstein CJ, Stein J, Arena R, et al. Guidelines for Adult Stroke Rehabilitation and Recovery: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association [published correction appears in Stroke. 2017 Feb;48(2):e78] [published correction appears in Stroke. 2017 Dec;48(12): e369]. Stroke. 2016;47(6):e98–e169. pmid:27145936
- 9. Kim WS, Cho S, Ku J, et al. Clinical Application of Virtual Reality for Upper Limb Motor Rehabilitation in Stroke: Review of Technologies and Clinical Evidence. J Clin Med. 2020;9(10):3369. Published 2020 Oct 21. pmid:33096678
- 10. Mane R, Chouhan T, Guan C. BCI for stroke rehabilitation: motor and beyond. J Neural Eng. 2020;17(4):041001. Published 2020 Aug 17. pmid:32613947
- 11. Zhang X, Yue Z, Wang J. Robotics in Lower-Limb Rehabilitation after Stroke. Behav Neurol. 2017;2017:3731802. pmid:28659660
- 12. Stinear CM, Lang CE, Zeiler S, Byblow WD. Advances and challenges in stroke rehabilitation. Lancet Neurol. 2020;19(4):348–360. pmid:32004440
- 13. Sharififar S, Shuster JJ, Bishop MD. Adding electrical stimulation during standard rehabilitation after stroke to improve motor function. A systematic review and meta-analysis. Ann Phys Rehabil Med. 2018;61(5):339–344. pmid:29958963
- 14. Doucet BM, Lam A, Griffin L. Neuromuscular electrical stimulation for skeletal muscle function. Yale J Biol Med. 2012;85(2):201–215. pmid:22737049
- 15. Sluka KA, Walsh D. Transcutaneous electrical nerve stimulation: basic science mechanisms and clinical effectiveness. J Pain. 2003;4(3):109–121. pmid:14622708
- 16. Meng D, Mao Y, Song QM, et al. Efficacy and Safety of Transcutaneous Electrical Acupoint Stimulation (TEAS) for Postoperative Pain in Laparoscopy: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Evid Based Complement Alternat Med. 2022;2022:9922879. Published 2022 Jan 15. pmid:35075367
- 17. Nussbaum EL, Houghton P, Anthony J, Rennie S, Shay BL, Hoens AM. Neuromuscular Electrical Stimulation for Treatment of Muscle Impairment: Critical Review and Recommendations for Clinical Practice. Physiother Can. 2017;69(5):1–76. pmid:29162949
- 18. Ring H, Weingarden H. Neuromodulation by functional electrical stimulation (FES) of limb paralysis after stroke. Acta Neurochir Suppl. 2007;97(Pt 1):375–380. pmid:17691399
- 19. Santos Ferreira I, Teixeira Costa B, Lima Ramos C, Lucena P, Thibaut A, Fregni F. Searching for the optimal tDCS target for motor rehabilitation. J Neuroeng Rehabil. 2019;16(1):90. Published 2019 Jul 17. pmid:31315679
- 20. Tang Y, Wang L, He J, Xu Y, Huang S, Fang Y. Optimal Method of Electrical Stimulation for the Treatment of Upper Limb Dysfunction After Stroke: A Systematic Review and Bayesian Network Meta-Analysis of Randomized Controlled Trials. Neuropsychiatr Dis Treat. 2021;17:2937–2954. Published 2021 Sep 15. pmid:34552328
- 21. Nascimento LR, da Silva LA, Araújo Barcellos JVM, Teixeira-Salmela LF. Ankle-foot orthoses and continuous functional electrical stimulation improve walking speed after stroke: a systematic review and meta-analyses of randomized controlled trials. Physiotherapy. 2020;109:43–53. pmid:33120054
- 22. Bai X, Guo Z, He L, Ren L, McClure MA, Mu Q. Different Therapeutic Effects of Transcranial Direct Current Stimulation on Upper and Lower Limb Recovery of Stroke Patients with Motor Dysfunction: A Meta-Analysis. Neural Plast. 2019;2019:1372138. Published 2019 Nov 16. pmid:31827495
- 23. Hong Z, Sui M, Zhuang Z, et al. Effectiveness of Neuromuscular Electrical Stimulation on Lower Limbs of Patients With Hemiplegia After Chronic Stroke: A Systematic Review. Arch Phys Med Rehabil. 2018;99(5):1011–1022.e1. pmid:29357280
- 24. Rouse B, Chaimani A, Li T. Network meta-analysis: an introduction for clinicians. Intern Emerg Med. 2017;12(1):103–111. pmid:27913917
- 25. Hutton B, Salanti G, Caldwell DM, et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162(11):777–784. pmid:26030634
- 26. Pocock SJ, Stone GW. The Primary Outcome Is Positive—Is That Good Enough?. N Engl J Med. 2016;375(10):971–979. pmid:27602669
- 27. Gittler M, Davis AM. Guidelines for Adult Stroke Rehabilitation and Recovery. JAMA. 2018;319(8):820–821. pmid:29486016
- 28. Hernández ED, Forero SM, Galeano CP, Barbosa NE, Sunnerhagen KS, Alt Murphy M. Intra- and inter-rater reliability of Fugl-Meyer Assessment of Lower Extremity early after stroke. Braz J Phys Ther. 2021;25(6):709–718. pmid:33358073
- 29. Yan T, Hui-Chan CW, Li LS. Functional electrical stimulation improves motor recovery of the lower extremity and walking ability of subjects with first acute stroke: a randomized placebo-controlled trial. Stroke. 2005;36(1):80–85. pmid:15569875
- 30. Cleland BT, Arshad H, Madhavan S. Concurrent validity of the GAITRite electronic walkway and the 10-m walk test for measurement of walking speed after stroke. Gait Posture. 2019;68:458–460. pmid:30599332
- 31. Wang L, Yin Z, Zhang Y, et al. Optimal Acupuncture Methods for Nonspecific Low Back Pain: A Systematic Review and Bayesian Network Meta-Analysis of Randomized Controlled Trials. J Pain Res. 2021;14:1097–1112. Published 2021 Apr 20. pmid:33907457
- 32. Chen HB, Zheng XY, LV X, et al. Effects of tDCS synchronized FES on lower limb motor function of stroke patients with hemiplegia: a randomized controlled study. Chinese Journal of Rehabilitation Medicine. 2021;36(10): 1227–1232.
- 33. Huang XD, Zheng YL, Xie ZW, Zhou S, et al. Effect of transcutaneous electrical stimulation of Taichong and Qiuxu points on ankle dorsiflexion function in elderly stroke patients. Journal of Emergency in Traditional Chinese Medicine. 2020;29(07):1260–1263.
- 34. Wang JT, Lv WL, Liu D. Clinical observation of low frequency acupoint electrical stimulation combined with rehabilitation training on ankle dorsiflexion dysfunction after stroke. Biped and Health. 2019;28(16):43–44+52.
- 35. Wang S, Tian J, Li ZF, et al. The clinical research of rehabilitation training combined with functional electrical stimulation treatment on foot drop caused by stroke. Chinese Journal of the Frontiers of Medical Science(Electronic Version). 2017;9(12):89–92.
- 36. Xu JY, Ru KY, Yang F, et al. A randomized controlled study of low frequency electrical stimulation on motor function in patients with hemiplegia after acute stroke. Journal of Bingtuan Medicine. 2015;46(04):23–24.
- 37. You GQ, Liao L, Liang HY, et al. Effectiveness of Functional Electrical Stimulation on Motor and Functional Recovery of the Lower Ex- tremity in Subject with Early Stroke: A Randomized Controlled Trial. China Practical Medicine. 2013;8(03):4–7.
- 38. GongY Shi GF, Wang TL. Effect of acupoint electrical stimulation on lower limb motor function of patients with poststroke foot prolapse Based on the theory of governing the paralysis take Yang Ming. Journal of Modern Medicine & Health. 2021;37(20):3434–3437.
- 39. Li GL, Zhu XM, Li X, et al. Effects of Functional Electrical Stimulation on Lower Limbs Motor Function of Stroke Patients with Hemiplegia. Medical Innovation of China. 2019;16(11):130–133.
- 40. Wen XP, Zou YM, Cao LH. Application effect of neuromuscular electrical stimulation therapy instrument on stroke patients with hemiplegia. Acta Medicinae Sinica. 2021;34(02):82–85.
- 41. Sun B, Zhou ZM, Zhong YJ. Clinical effect of functional electrical stimulation on hemiplegia in patients with cerebral apoplexy. Chinese Journal of Practical Nervous Diseases. 2020;23(12):1095–1099.
- 42. Li XZ, Li HZ, Liang CM. Effect of neuromuscular electrical stimulation therapy on gait spatiotemporal parameters and symmetry indexes among hemiplegic patients with stroke. Academic Journal of Guangzhou Medical University. 2021;49(01):37–40.
- 43. Liang ZX. Effect of functional electrical stimulation combined with rehabilitation training on stroke hemiplegia. Health Horizon. 2019(6):55.
- 44. Ma YY, Wang WB, He QS, et al. The efficacy of transcutaneous electrical nerve stimulation on lower limb spasticity and movement function in stroke patients with hemiplegia. Chinese Journal of Brain Diseases and Rehabilitation(Electronic Edition). 2011,1(02):183–186.
- 45. Cheng AL, Shen XY, Yang J. Effects of transcutaneous electric nerve stimulation combined with rehabilitative training on the recovery of memory and motor function in stroke patients. Chinese Journal of Clinical Rehabilitation. 2005(37):15–17.
- 46. Huang Y, Wan XL, Pan CH, et al. Effects of Functional Electrical Stimulation on Walking Ability in Stroke Patients with Foot Drop. Neural Injury and Functional Reconstruction. 2014;9(03):231–232+248.
- 47. Zhang XY. Effect of functional electrical stimulation and rehabilitation training on patients with foot ptosis caused by stroke. Contemporary Medical Symposium. 2016;14(22):47–48.
- 48. Yan TB, Hui-Chan CWY. Transutaneous electrical nerve stimulation on acupuncture points decreases spasticity and improves muscle strength of the affected lower extremity in subjects with acute stroke. Chinese Journal of Physical Medicine and Rehabilitation. 2007(01):25–28.
- 49. Chen RD, Chen A, Chen JJ. Effect of neuromuscular electrical stimulation on limb function rehabilitation in patients with cerebral infarction. Practical Clinical Journal of Integrated Traditional Chinese and Western Medicine. 2020;20(06):113–114+124.
- 50. Wen X. Effect of neuromuscular electrical stimulation on lower limb dysfunction in patients with cerebral infarction. Journal of Frontiers of Medicine. 2021;11(19):77–78.
- 51.
Tao X. Clinical study on the treatment of lower limb motor dysfunction of apoplexy hemiplegia by transcutaneous acupoint electrical stimulation. Yunnan University of Chinese Medicine. 2020. https://doi.org/10.27460/d.cnki.gyzyc.2020.000160
- 52. Liu ZL, Guan S, Song L, et al. Effects of functional electrical stimulation on the integral of motor function of lower limbs in patients with stroke hemiplegia. Chinese Journal of Clinical Rehabilitation. 2004(31):6824–6825.
- 53. Sabut SK, Sikdar C, Kumar R, Mahadevappa M. Functional electrical stimulation of dorsiflexor muscle: effects on dorsiflexor strength, plantarflexor spasticity, and motor recovery in stroke patients. NeuroRehabilitation. 2011;29(4):393–400. pmid:22207067
- 54. Zhang XH, Gu T, Liu XW, et al. The Effect of Transcranial Direct Current Stimulation and Functional Electrical Stimulation on the Lower Limb Function of Stroke Patients. Front Neurosci. 2021;15:685931. Published 2021 Sep 21. pmid:34621150
- 55. Huang RL, Yao SF. Multi-Channel Functional Electrical Stimulation Improving Activities of Daily Living in Patients with Early Stroke Hemiplegia: A Randomized Controlled Trial. Chinese Manipulation and Rehabilitation Medicine. 2018;9(02):12–14.
- 56. Huang T, Fan QH, Wu TY, et al. Functional electrical stimulation for improving motor function in elderly patients with cerebral apoplexy hemiplegia. Chinese Journal of Physical Medicine and Rehabilitation. 2010(05):370–372. 2010. 05. 013.
- 57. You GQ, Yan TB, Christina WY HUI-CHAN. Effectiveness of functional electrical stimulation on functional recovery of the lower extremity in subject with early stroke: a randomized controlled trial. Chinese Journal of Rehabilitation Medicine. 2007(10):867–870.
- 58. Zheng XY, Chen HB, Lv X, et al. The effect of tDCS synchronized FES on balance and walking function in stroke patients with hemiplegia. Chinese Journal of Rehabilitation Medicine. 2021;36(10):1220–1226.
- 59. Chen C, Li S, Tang CZ, et al. Effects of transcutaneous electrical acupoint stimulation on lower limb motor function and activities of daily living in stroke patients. Chinese Journal of Rehabilitation. 2016;31(03):168–170.
- 60. Peng Y, Zhang J, Zhang TB, et al. Effect of transcutaneous accupoint electrical stimulation on motor function recovery of the lower extremity of stroke patients. Guangzhou Medical Journal. 2015;46(01):19–22.
- 61. Mitsutake T, Sakamoto M, Nakazono H, Horikawa E. The Effects of Combining Transcranial Direct Current Stimulation and Gait Training with Functional Electrical Stimulation on Trunk Acceleration During Walking in Patients with Subacute Stroke. J Stroke Cerebrovasc Dis. 2021;30(4):105635. pmid:33517032
- 62. Sabut SK, Sikdar C, Mondal R, Kumar R, Mahadevappa M. Restoration of gait and motor recovery by functional electrical stimulation therapy in persons with stroke. Disabil Rehabil. 2010;32(19):1594–1603. pmid:20210592
- 63. Burridge JH, Taylor PN, Hagan SA, Wood DE, Swain ID. The effects of common peroneal stimulation on the effort and speed of walking: a randomized controlled trial with chronic hemiplegic patients. Clin Rehabil. 1997;11(3):201–210. pmid:9360032
- 64. Sujith OK. Functional electrical stimulation in neurological disorders. Eur J Neurol. 2008;15(5):437–444. pmid:18394046
- 65. Dimitrijevic MR. Clinical practice of functional electrical stimulation: from "Yesterday" to "Today". Artif Organs. 2008;32(8):577–580. pmid:18782124
- 66. Woods AJ, Antal A, Bikson M, et al. A technical guide to tDCS, and related non-invasive brain stimulation tools. Clin Neurophysiol. 2016;127(2):1031–1048. pmid:26652115
- 67. Hordacre B, Moezzi B, Ridding MC. Neuroplasticity and network connectivity of the motor cortex following stroke: A transcranial direct current stimulation study. Hum Brain Mapp. 2018;39(8):3326–3339. pmid:29655257
- 68. Fregni F, El-Hagrassy MM, Pacheco-Barrios K, et al. Evidence-Based Guidelines and Secondary Meta-Analysis for the Use of Transcranial Direct Current Stimulation in Neurological and Psychiatric Disorders. Int J Neuropsychopharmacol. 2021;24(4):256–313. pmid:32710772
- 69. Bikson M, Grossman P, Thomas C, et al. Safety of Transcranial Direct Current Stimulation: Evidence Based Update 2016. Brain Stimul. 2016;9(5):641–661. pmid:27372845
- 70. Elsner B, Kugler J, Pohl M, Mehrholz J. Transcranial direct current stimulation (tDCS) for improving activities of daily living, and physical and cognitive functioning, in people after stroke. Cochrane Database Syst Rev. 2020;11(11):CD009645. Published 2020 Nov 11. pmid:33175411
- 71. Mahmood A, Veluswamy SK, Hombali A, Mullick A, N M, Solomon JM. Effect of Transcutaneous Electrical Nerve Stimulation on Spasticity in Adults With Stroke: A Systematic Review and Meta-analysis. Arch Phys Med Rehabil. 2019;100(4):751–768. pmid:30452892
- 72. Hesse S, Werner C, Bardeleben A, Brandl-Hesse B. Management of upper and lower limb spasticity in neuro-rehabilitation. Acta Neurochir Suppl. 2002;79:117–122. pmid:11974976
- 73. In TS, Jung JH, Jung KS, Cho HY. Effectiveness of Transcutaneous Electrical Nerve Stimulation with Taping for Stroke Rehabilitation. Biomed Res Int. 2021;2021:9912094. Published 2021 Aug 25. pmid:34485529
- 74. Maffiuletti NA, Gondin J, Place N, Stevens-Lapsley J, Vivodtzev I, Minetto MA. Clinical Use of Neuromuscular Electrical Stimulation for Neuromuscular Rehabilitation: What Are We Overlooking?. Arch Phys Med Rehabil. 2018;99(4):806–812. pmid:29233625
- 75. Bickel CS, Gregory CM, Dean JC. Motor unit recruitment during neuromuscular electrical stimulation: a critical appraisal. Eur J Appl Physiol. 2011;111(10):2399–2407. pmid:21870119
- 76. Maffiuletti NA. Physiological and methodological considerations for the use of neuromuscular electrical stimulation. Eur J Appl Physiol. 2010;110(2):223–234. pmid:20473619
- 77. Stein C, Fritsch CG, Robinson C, Sbruzzi G, Plentz RD. Effects of Electrical Stimulation in Spastic Muscles After Stroke: Systematic Review and Meta-Analysis of Randomized Controlled Trials. Stroke. 2015;46(8):2197–2205. pmid:26173724
- 78. Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. Published 2011 Oct 18. pmid:22008217
- 79. Mlinarić A, Horvat M, Šupak Smolčić V. Dealing with the positive publication bias: Why you should really publish your negative results. Biochem Med (Zagreb). 2017;27(3):030201. pmid:29180912
- 80. Lin L. Hybrid test for publication bias in meta-analysis. Stat Methods Med Res. 2020;29(10):2881–2899. pmid:32290810
- 81. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. Published 2009 Jul 21. pmid:19622552