Introduction

Patients admitted to intensive care units (ICUs) are affected by both their illness and certain factors associated with their treatment, such as environmental changes, fear of death, passive compliance, and potentially permanent loss of function; consequently, a series of mental, physical, and psychological clinical syndromes may develop in such patients, such as depression, anxiety, delirium, and post-traumatic stress disorder [1]. Intensive care unit-acquired weakness (ICU-AW) is one such condition that may develop, which is generalized limb weakness caused by neuromuscular dysfunction that develops during critical illness. It involves a series of clinical symptoms, such as difficulty in weaning from mechanical ventilation, paresis or quadriplegia, decreased reflexes, and muscle atrophy [2]. It has been reported that the incidence of ICU-AW among ICU patients is 25% to 50%, and ICU-AW is associated with a prolonged duration of mechanical ventilation, a prolonged stay in an ICU, and increased mortality, all of which seriously affect the recovery of patients [3–5]. Currently, there are no specific methods and medicine for the treatment of ICU-AW, therefore, medical staff should pay attention to the prevention of ICU-AW. Early identification of ICU-AW risk groups and targeted intervention measures are of great significance for preventing the occurrence of ICU-AW [6].

Presently, ICU-AW is diagnosed by assessing the strength of a patient’s limb muscles. Generally, the Medical Research Council (MRC) scale is used to evaluate the strength of six pairs of bilateral muscles in a patient, including the muscles involved in wrist extension, forearm flexion, shoulder abduction, foot dorsiflexion, knee extension, and thigh bending [7]. However, due to the impaired consciousness or attention of a majority of critically ill patients who are admitted to an ICU, which may be caused by coma, sedation, or delirium, muscle-strength assessment is usually not possible in the early stages after a patient’s admission to an ICU. In addition, there is currently no consensus regarding a gold-standard method that can be used for the diagnosis of ICU-AW [8]. These factors that cause uncertainty with respect to ICU-AW identification and diagnosis may cause medical staff to ignore ICU-AW or delay the diagnosis of ICU-AW. Therefore, as soon as a patient is admitted to an ICU, in order to prevent a delay in ICU-AW diagnosis, it is important to quantify the risk of ICU-AW with the use of predictive models.

In previous studies, researchers have adopted different types of study design including single-center or multi-center for the development of ICU-AW risk prediction models. The risk prediction model of ICU-AW considers the multiple causes of ICU-AW, and through the use of statistical models, such a model may be employed to predict the probability of ICU-AW occurrence in ICU patients. On one hand, an ICU-AW prediction model can be used by medical staff to effectively screen high-risk patients for ICU-AW, improve their awareness of the risk of ICU-AW occurrence, and take corresponding preventive measures to reduce the risk of ICU-AW occurrence in ICU patients. On the other hand, it can also enable patients and their families to clearly understand the risk of ICU-AW occurring during a period of ICU admission and improve their understanding of and cooperation with work related to risk prediction and prevention of ICU-AW.

The purpose of this systematic review was to comprehensively search and review for studies regarding ICU-AW risk prediction models, in which such models had been developed and used to determine the risk of ICU-AW for ICU patients. It is worth noting that one previous study conducted a systematic review of 8 studies on risk prediction model of ICU-AW [9], and our study was a further exploration based on this systematic review, so there was the overlap in the two systematic reviews. However, the difference is that we have increased the amount of literature and analyzed the basic content of the model, method of development, the form of model, the applicability and limitations of the model in more detail, and provided an exhaustive summary of the characteristics, effectiveness and the differences between different ICU-AW risk prediction models that have been previously developed and applied. The findings of this study could consequently be used to make informed decisions regarding the use of one or more of these prediction models for the prediction of ICU-AW in ICU patients.

Methods

Results

Description of search and eligible studies

A total of 2,111 studies were searched, and 1,760 studies remained after duplicates were removed. A total of 1,717 studies were excluded due to the obvious irrelevance of the topics of those studies, as determined by evaluating the titles and abstracts of the study papers. We identified 43 studies for further evaluation by reviewing the full-text of the articles; 30 studies were excluded because in those studies, the performance of prediction models had not been measured, and the method of development of the models was considered unreasonable; two studies were excluded because there was insufficient data in those studies for extraction. Finally, 11 studies that met the inclusion criteria for this systematic review were included. The detailed search steps have been described using the PRISMA 2009 Flow Diagram (Fig 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Flow chart of study selection.

https://doi.org/10.1371/journal.pone.0257768.g001

Study characteristics and assessment of quality

A total of 11 studies involving 11 different ICU-AW risk prediction models with 5744 patients were included in this systematic review; of these, nine were English-language studies and two were Chinese-language studies. In terms of study design, five were cohort studies, two were observational studies, two were case-control studies, and two were retrospective studies. The NOS scores of the included literature ranged from 5 to 8 points, with indicates that the quality of the included literature was high. The basic characteristics and quality assessments of the included studies are presented in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics and quality of the included studies.

https://doi.org/10.1371/journal.pone.0257768.t001

Development of predictive models

The number of candidate predictors in each study ranged from 8 to 25, and in one study, continuous variables had been converted into dichotomous variables [14]. In the included studies, the total sample size ranged from 56 to 4157 cases, the number of result events ranged from 25 to 190, and the incidence of ICU-AW ranged from 2.7% to 68.5%. Four studies [4, 5, 16, 21] reported missing data, with the number of cases with missing data ranging from 8 to 12. For the development of the prediction models, logistic regression had been used in all studies. In five studies [4, 5, 14, 18, 21], backward stepwise selection had been used for the selection of predictor variables, and forward stepwise selection had been used in two studies [16, 19]. The details are listed in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Development of predictive models for ICU-AW.

https://doi.org/10.1371/journal.pone.0257768.t002

Performance and predictive factors of models

In eight studies [5, 14, 15, 17–20, 22], the discriminative performance of the prediction models was reported, with AUROC values ranging from 0.7 to 0.923, and the prediction performance reported in these studies was good. Seven studies reported the degree of calibration according to Hosmer-Lemeshow test [4, 5, 14, 15, 19, 20, 22], which the goodness of fit test of four models P> 0.05 [14, 15, 20, 22], and one of which was presented in the form of a calibration graph [19]. The classification ability of the models was reported in two studies, with a sensitivity of 74%–83.3% [16] and specificity of 88%–88.8% [21]. In terms of model verification, in three studies, internal verification was used to verify the established prediction models, and in one study, a combination of internal and external verification was used to evaluate the predictive effectiveness of the prediction model [19]. The final prediction models considered two to six predictors, and the predictor most commonly included in the models were age and the administration of corticosteroids. In five studies [5, 17, 19, 20, 22], formulas had been developed to determine scores for the risk of ICU-AW. The performance and predictors of all models are shown in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Performance and predictive factors of models for ICU-AW.

https://doi.org/10.1371/journal.pone.0257768.t003

Risk of bias and applicability evaluation of models

After the evaluation, we found that with respect to the domain of the participant, the risk of bias for all included studies was low, which the selected data sources were appropriate, and participants had been selected according to the inclusion and exclusion criteria. With respect to predictors, a high risk of bias was identified for two of the ten studies [4, 21], and the remaining studies had a low risk of bias. Ballve et al. [14], Wieske et al. [5] and Witteveen et al. [19] pointed out that the evaluator was blinded for the predictors, and the other of studies are "no information.” With respect to study results, we identified that two studies had a high risk of bias [4, 17], and nine studies had a low risk of bias. For a model-development study, if the number of events per variable (EPV) is < 10 there may be overfitting [23], and if the number of EPV is > 20, the study would be convincing [24, 25]. For model validation, if the number of EPV is < 100 [26], this may cause bias. Except for the sample size of the study by Witteveen et al. [19], the sample sizes of the other studies were insufficient and did not meet the requirements. In terms of variable selection, Liu et al. [20] and Miao et al. [22] directly performed multivariate analysis after univariate analysis without using appropriate variable-selection methods. In addition, the competition risk and time analysis of a prediction model had been considered in only one study [15], and the complexity of the data may have been overlooked in the other studies, in which no information had been provided in this regard. Except for the Witteveen et al. model [19], the other models had a high overall risk of bias. With respect to the applicability of the models, the applicability of all models was found to be good, and the study participants, predictors, and outcomes were highly consistent with those specified in the systematic-review questions. The evaluation of the risk of bias and applicability of the models, performed using PROBAST, is presented in Table 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Risk of bias and applicability evaluation of models.

https://doi.org/10.1371/journal.pone.0257768.t004

Discussion

Due to the various adverse effects that ICU-AW may have on patients who are critically ill, ICU medical staff pay attention to the risk prediction of ICU-AW and interventions for its prevention and treatment. Recently, multiple studies regarding the development and verification of ICU-AW risk prediction models have been consecutively performed. However, there are differences between the final prediction models developed, verified, and used in the different studies; these differences may be caused due to the differences in the areas considered, participants included, and methods used in the different studies. Therefore, it is necessary to systematically evaluate the existing prediction models to provide a theoretical basis for medical staff to choose high-quality models that can be used as risk-screening tools for ICU-AW. Liu et al. [9] included eight studies about the risk prediction model of ICU-AW for systematic review, and the author believed that the current prediction model of ICU-AW had good predictive performance and applicability, but suggested that the whole process of model development and verification should be reported in a standardized way. The results of our study are similar to those of Liu et al. [9], but the difference is that we believe that although the prediction performance of most models is good, due to lack of reports, the prediction performance of some models is still uncertain. In addition, most of the model lack external verification, which the stability is unclear, and the further study is needed. However, based on the current results of this system review, which can provide readers with more information about the prediction model of ICU-AW, so that they can better choose a model suitable for the current clinical context.

Through this systematic review, we comprehensively searched for relevant studies regarding ICU-AW risk prediction models using evidence-based methods and conducted an objective quality evaluation. After screening, 11 studies were finally included, including nine model-development studies and one model-validation study. The AUROC values for seven models were ≥ 0.7, which shows that those seven models can effectively predict the risk of ICU-AW in ICU patients. In seven studies, the calibration performance of the developed prediction models had been assessed by using the Hosmer-Lemeshow test, which allowed for a more scientific development of the ICU-AW risk prediction models. However, most of the models were biased due to the lack of blindness, lack of reporting, insufficient sample size, ignoring of missing data, and lack of evaluation of the performance and calibration of the risk prediction models. Internal verification methods had been used in only three studies, and in one study, a combination of internal and external verification had been used to verify a prediction model. Therefore, it appears that research regarding ICU-AW risk prediction models is still in a developmental stage.

In terms of diagnostic tools for ICU-AW, in eight studies [4, 5, 14, 15, 19–22], the MRC scale was used to assess ICU-AW, and in the three remaining three studies [16–18], different methods to diagnose ICU-AW had been used, including electrophysiological testing, quadriceps rectus femoris (QRF) ultrasonography, and definition based on expert consensus. The use of the MRC scale for the diagnosis of ICU-AW was recommended by the official clinical-practice guidelines of the American Thoracic Association; if the MRC-scale score for an individual is lower than 48, a diagnosis of ICU-AW can be made [27]. However, it is usually difficult to control factors related to the evaluation of ICU-AW, including the assessor and assessment time, because there is a lack of professional rehabilitation specialists who can measure the muscle strength of the muscles of different muscle groups for patients. Furthermore, most patients who are critically ill are unable to cooperate with such evaluations due to severe disease, sedation, and delirium. In addition, electrophysiological examination cannot be used to distinguish whether the cause of muscle weakness is myogenic or neurogenic, and the relationship between muscle weakness and its causes is unclear. Moreover, the clinical practice applicability of an ultrasound scan in this context is unclear. However, the diagnostic tool of ICU-AW was only used as an evaluation content in this review, which was aimed to understand the current international trend of diagnosis of ICU-AW. Although there is uncertainty about the use of other diagnostic methods, the use of different diagnostic methods in different studies has no effect on the predictive performance of the model itself. Therefore, even if only using MRC score as a diagnostic criterion, and excluding studies that did not use MRC as a diagnostic methods, which the existing study results and conclusions also will not be changed.

Excluding duplicate predictors, in this systematic review, 28 predictors were considered; the predictors most commonly included in the studies reviewed were the administration of corticosteroids, age, highest lactate levels, and acute renal failure. The different studies considered different predictors because of the differences in the diseases types of the participants included, conceptual definitions, and diagnostic tools used in the studies. Corticosteroids, which are commonly used in ICUs, mainly refer to glucocorticoid drugs; an excessive intake of corticosteroids causes muscle dysfunction and nerve damage, promotes the decomposition of muscle-tissue protein, and leads to increased protein loss [20]. Moreover, the side-effect of corticosteroid use is lipodystrophy, and corticosteroids may increase both the uptake and turnover of fatty acids in adipose tissue, which are closely related to the occurrence of ICU-AW [28]. Age is an important risk factor for ICU-AW [29]. In elderly patients, muscle protein synthesis decreases and decomposition increases with age. A decrease in the amount of muscle proteins can directly cause muscle weakness, which may gradually develop into sarcopenia, thereby increasing the risk of ICU-AW [30]. Blood lactate level is another important predictor. On one hand, an increase in blood lactate levels reduces the pH of blood and makes it acidic, which can lead to the stimulation of muscle nerve endings and cause damage [20]. On the other hand, a high blood lactate level causes a decrease in ionized calcium (Ca²⁺) concentrations, which affects the release and reuptake of Ca²⁺ by the sarcoplasmic reticulum during the process of muscle excitation-contraction coupling [20]. This may cause a decrease in the excitability of muscle nerves and lead to the occurrence of ICU-AW. Previous studies have shown that increased levels of arterial blood lactate will cause a certain degree of damage to myocardial cells [31], and aggravating histiocellular ischemia and hypoxia [32], which will cause neurological disorders. In addition, the incidence of acute kidney injury (AKI) among ICU patients is high due to multiple risk factors such as sepsis, surgery, shock, diabetes, hypertension, heart failure, use of nephrotoxic drugs, etc., [33]. AKI can cause electrolyte disorders in patients, among which an elevated concentration of blood potassium is the most common. An elevated serum potassium concentration causes the depolarization of the cell membrane and leads to limb weakness and reflex disappearance. Thus it increases the likelihood of ICU-AW occurring in a patient.

The models considered in this study have certain advantages and disadvantages. The model by Witteveen et al. [19] has been externally verified and has good extrapolation. The models by Hernández-Socorro et al. [17] and Liu et al. [20] have better discrimination than the other models. However, some models have certain shortcomings. The predictors included in some models are not present when the patient is admitted to ICU, such as MV duration> 5 days and hyperglycemia > 3 days included in Ballve et al. [14]; hospital length of stay included in Wolfe [15]; duration of MV and days with organ dysfunction included in De Johnghe et al. [4], etc. Therefore these models can identify risk factors associated with the outcomes of ICU-AW, but may not be appropriate for early risk prediction of ICU-AW on ICU admission. In the study by De Jonghe et al. [4], only the degree of calibration of the model used was reported, and in another two studies [16, 21], only the specificity and sensitivity of the modes were reported. Hernández-Socorro et al. [17] used QRF ultrasonography to diagnose ICU-AW; however, it is not easy to measure the QRF muscle area and QRF tendon thickness, as compared to the ease of determining other predictor values. Garnacho-Montero et al. [16] and Weber-Carstens et al. [21] used electrophysiological testing to diagnose ICU-AW, but electrophysiological testing is susceptible to interference from factors such as a patient’s disease condition and electromagnetic interference caused by treatment equipment. When choosing an appropriate model for an actual clinical situation, medical staff should comprehensively consider the predictive performance of a model, the availability of predictive factors, and the convenience of outcome measurement.

Conclusions

In this study, we considered a total of 11 ICU-AW risk prediction models, and systematically evaluated the model performance, methodological quality, method of development of a model, predictive factors, etc. The study results show that the predictive performance and applicability of most models are good, but the models lacked validity verification. This suggests that research regarding ICU-AW risk prediction models is still in a developmental stage, and there is no model that can be directly applied to the Chinese population. We suggest that prediction models with excellent performance and strong feasibility in all aspects should be developed in the future and their use for different populations from different regions should be verified.

Supporting information

Acknowledgments

We would like to thank Editage (www.editage.cn) for English language editing.