Identification of research question

This scoping review aims to address the following questions: (1) What types of predictive models are currently available for predicting the risk of PICC-CRBSI? (2) Which predictive factors are most frequently incorporated into these risk prediction models? (3) What modeling methods are currently used for PICC-CRBSI risk prediction, and how effective are they? (4) What are the limitations of existing studies on PICC-CRBSI risk prediction models, and what directions do these suggest for future research?

Search strategies and eligibility criteria

A comprehensive literature search was conducted across multiple databases, including the China National Knowledge Infrastructure, Wanfang Database, VIP Chinese Journal Database, PubMed, Embase, and Web of Science. The search period covered from the inception of these databases up to November 5, 2024. Both subject headings and free-text terms were used in the retrieval process. For example, in PubMed, the search formula for the English database was: (vascular access devices [MeSH Terms] OR peripherally inserted central catheter [Title/Abstract] OR central venous catheters [MeSH Terms] OR PICC [Title/Abstract] AND (catheter-related infect [Title/Abstract] OR CRBSI [Title/Abstract] OR CLABSI* [Title/Abstract] OR bloodstream infect* [Title/Abstract] OR blood poison* [Title/Abstract])** AND (risk predict [Title/Abstract] OR risk score [Title/Abstract] OR risk assess [Title/Abstract] OR score [Title/Abstract] OR prognos* [Title/Abstract] OR nomogram model [Title/Abstract])** [13]. Details of eligibility criteria were included in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Scoping review eligibility criteria.

https://doi.org/10.1371/journal.pone.0333466.t001

Data extraction

Two researchers (the first and third authors) independently conducted the literature screening according to the predefined inclusion and exclusion criteria, and extracted and integrated the relevant data. Any disagreements were resolved by a third reviewer (Yanhui Liu). Extracted data included: publication year, country, research design, research type, subject, sample size, incidence rate, modeling methods, candidate variables, variable screening methods, model presentation, model validation status, missing data, model predictive factors, and predictive performance.

Bias risk and applicability evaluation

Two researchers (the first and third authors) evaluated the quality of the included literature using the Prediction Model Risk of Bias Assessment Tool (PROBAST) [14]. The bias risk assessment of predictive models using this tool covers four domains: research object, predictive factors, outcomes, and statistical analysis. Additionally, the assessment of adaptability focuses on three domains: research object, predictive factors, and outcomes. The risk of bias and concerns regarding applicability in each domain were rated as high, low, or unclear.

Characteristics of selected studies

Eight studies [15–21] were included in this study. Among these, one was a multicenter prospective cohort study [15], while the remaining seven were retrospective studies [16–21]. Six studies conducted both model development and validation [15,17,19–21], one study focused exclusively on model development [16], and another solely on model validation [18]. In terms of patient populations, one study specifically involved hospitalized medical adults [15], four were applicable to tumor patients [17,19,20,22] and three were targeted to hospitalized adults [16,18,21]. The basic characteristics of the included studies are detailed in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Characteristics of the included studies (n = 8).

https://doi.org/10.1371/journal.pone.0333466.t002

Assessment of literature bias risk and applicability

The PROBAST evaluation for bias risk, covering four aspects of model development—study population, predictive factors, outcomes, and analysis—indicated a higher risk of bias in the predictive factors and analysis sections. This elevated risk may be attributed to the selection of statistical methods for predictive factors and the consideration of complex statistical issues during model development. The results of the evaluation are summarized in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Evaluation of bias risk and applicability of included studies.

https://doi.org/10.1371/journal.pone.0333466.t003

Construction and validation of the model

Basic information on model construction.

The study population for this model predominantly included elderly individuals, cancer patients, and hospitalized patients, with sample sizes ranging from 505 to 23,088. The modeling techniques utilized were Cox regression and logistic regression, involving between 5 and 24 candidate variables and 4–8 predictive factors in the analysis. The model was illustrated using scoring scales and line graphs, with both internal and external validations, see Table 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Construction and validation of risk prediction model (n = 8).

https://doi.org/10.1371/journal.pone.0333466.t004

Model prediction content.

The results were summarized and categorized. Ultimately, the predictors were classified into five categories: general information, catheter-related factors, disease-related factors, treatment factors, and other factors. The most frequently observed predictors included catheter retention time, maintenance cycle/frequency, multilumen, PICC parenteral nutrition, diabetes, and malignant tumors. The classification of model predictive factors is shown in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Classification of model predictive factors.

https://doi.org/10.1371/journal.pone.0333466.t005

Model validation and performance.

Among the eight models evaluated, four models [15–17,21] underwent internal validation, while the other four models were externally validated [18–20,22]. In terms of model performance, two models [19,22] employed the Hosmer-Lemeshow goodness-of-fit test, which indicated a good fit with X² = 8.905, P = 0.350 for the modeling set, and X² = 8.693, P = 0.365 for the validation set. Six models [15,16,19,22] reported the area under the curve (AUC) of the receiver operating characteristic curve, and two models [16,17] calculated the C-index to demonstrate their discriminative power. Additionally, five models [16–20] assessed model calibration. Two models [18,20] conducted decision curve analysis (DCA), with one noteworthy example by Hirotaka et al. [18] performing external validation on the Michigan Peripheral Insertion Center Catheter-Related Blood Flow Infection Rating Model [15] developed in 2017, demonstrating DCA’s superiority over the original rating scale. It is important to note that sensitivity and specificity were not reported for some models. In summary, the evidence indicates that each model is most reliable when applied to the patient group for which it was originally derived and externally validated: the Tang QY nomogram offers the strongest discrimination for general medical inpatients, the Hao QY nomogram performs best in lung-cancer patients undergoing chemotherapy, the Yu QQ nomogram and the ELM model are suitable for heterogeneous mixed-tumor cohorts, the Guo J nomogram is calibrated for immunocompromised elderly individuals with acute leukemia, and the large-scale MPC score provides a pragmatic baseline across diverse hospital settings. Together, these data underscore that model choice should be guided by alignment between derivation cohort and local case-mix, available clinical resources, and the extent of external validation in the intended population. Details of model validation and performance are presented in Table 6. Forest plot of models were drawed (Fig 2), two studies could not extract complete data, and no forest plot was generated [15,18].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Model validation and performance.

https://doi.org/10.1371/journal.pone.0333466.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Forest plot of different model performances.

The length of the blue line represents the width of the confidence interval, the red square represents the area under the curve (AUC) of the model, * corresponds to the development set of the model, and ** represents the validation set.

https://doi.org/10.1371/journal.pone.0333466.g002

Characteristics and validation needs of risk prediction models for PICC-CRBSI

Recent years have seen increasing scholarly interest in PICC catheter-related bloodstream infections, leading to a significant rise in studies aimed at their risk prediction [15–22]. This study systematically reviewed such research, both domestically and internationally, culminating in the inclusion of eight studies comprising six distinct risk prediction models. Only one of these was a multicenter prospective cohort study, with the others being retrospective, encompassing sample sizes from 505 to 23,088. The methodologies employed included Cox and logistic regression, with the number of candidate variables ranging from five to twenty-four, and predictive factors from four to eight. The models were presented using scoring scales and column charts. The AUC for six models ranged between 0.67 and 0.930 [15–17,19–21], while two models reported a C-index between 0.825 and 0.929 [16,17], indicating robust predictive performance. Key predictors in these models included catheter retention time, maintenance cycle/frequency, multilumen, PICC parenteral nutrition, diabetes, and malignancy. Clinically, it is essential to raise awareness and train medical personnel to promptly identify and address risk factors for PICC catheter-related bloodstream infections to prevent complications.

In terms of model construction, only one model utilized LASSO regression [21], and there are currently no reports in Chinese or English on using machine learning algorithms for developing risk prediction models for PICC catheter-related blood flow infections. LASSO regression plays a critical role in reducing dimensionality and selecting variables, effectively addressing multicollinearity when numerous influencing factors are involved [23]. Future developments in predictive modeling should consider employing LASSO regression and machine learning algorithms for feature selection, thereby enhancing predictive accuracy. Notably, 87.50% of the models included in this study were retrospective, highlighting the need for more multicenter, large-sample, prospective studies in the future.

Regarding model validation, four models [15–17,21] underwent internal validation, while four models [18–20,22] were externally validated. Internal validation is critical to prevent model overfitting, while external validation establishes a basis for model optimization and broader application. Upon completing model development, it is crucial to conduct both internal and external validation simultaneously and to perform multiple model comparisons to boost predictive accuracy and stability [24,25]. All models underwent discrimination testing, however, only two model [19,22] demonstrated goodness of fit. Five models [16–20] evaluated model calibration, and two models [18,20] assessed clinical utility, however, none of the studies addressed model sensitivity or specificity. For future construction of a risk prediction model for PICC catheter-related bloodstream infections, adherence to established clinical prediction model guidelines is recommended [24]. This should encompass evaluations of the model’s discrimination, calibration, goodness of fit, and clinical utility, as well as including specificity and sensitivity in research findings.

The risk of bias in the PICC-CRBSI risk prediction model is significant and requires further optimization

PROBAST is an invaluable tool for evaluating the risk of bias and the applicability of predictive models [14]. This study’s findings reveal that while the included models showed good adaptability, they were associated with a high risk of bias, with only two models exhibiting a low risk. Among the eight studies included, only Erica et al. [15] employed a prospective approach, retrospective studies are especially prone to bias from existing outcomes, thereby increasing the risk of bias. Of the eight studies, only two conducted the Hosmer-Lemeshow goodness-of-fit test [19,22]. Furthermore, in the study by Guo J et al. [20], a limited number of five candidate variables and fewer than ten dependent events significantly increased the risk of model overfitting [26]. Some researchers increase bias risk by converting continuous variables into categorical ones in logistic regression modeling. For instance, Hirotaka et al. [18] used age as a binary variable, while Li W et al [21] transformed age, catheter retention time, and catheter insertion precursor temperature into binary variables. Furthermore, diagnostic criteria varied, Hao Qiyan et al. [19] adhered to the “Guidelines for the Prevention and Treatment of Endovascular Catheter-related Infections (2007)” [27], whereas Tang QY et al. [16] and Li W et al [21] followed the criteria published by the Centers for Disease Control and Prevention in the United States [28]. Additionally, some studies [17,18,21] failed to provide detailed data processing descriptions, indicating that comprehensive reporting is necessary during model development and validation processes.

Impact of sample-size variability on model reliability and generalizability

Sample-size heterogeneity across the eight included studies influenced both the stability of individual prediction models and the precision of their reported performance metrics. Derivation cohorts with limited events-per-predictor parameter (EPP < 10)—observed in 6 of the 8 studies—are inherently prone to over-fitting, which can widen confidence intervals around discrimination (e.g., AUROC) and calibration statistics and thus raise concerns about internal validity [29]. Conversely, the two studies with EPP ≥ 10 benefited from reduced statistical noise and narrower intervals, although sheer size alone did not guarantee superior model accuracy. From a translational perspective, generalizability depends less on absolute cohort size than on population representativeness and the inclusion of external validation. In this review, studies spanning the full sample-size spectrum that incorporated external validation reported consistent performance between derivation and validation cohorts, indicating that adequate EPP and rigorous validation can offset potential drawbacks of modest cohort sizes. Therefore, while variable sample sizes introduce differential uncertainty, their impact on the collective evidence base is attenuated when methodological quality—particularly appropriate parameter-to-event ratios and robust external validation—is maintained [30].