Phase 1: NuBE Scale development.

Item development procedure: Phase 1 focused on the conceptualization of the NuBE Scale and the evaluation of face validity [34]. Item development was informed by two primary sources: a systematic review on nursing behaviors promoting patient engagement [5] and a qualitative study involving oncology patients and nurses [7]. The review, covering six databases (PubMed, CINAHL, Embase, Scopus, Web of Science, and Cochrane Library) from 2005 to 2021, included 24 eligible studies (RCTs, quasi-experimental, and pre-post designs in English, Spanish, French, or Italian) and was appraised using the Joanna Briggs Institute checklists. Three core dimensions—cognitive, emotional, and motivational—were identified as central to nursing behaviors fostering engagement, while a fourth dimension, related to informal caregivers, emerged from qualitative findings. The qualitative study involved semi-structured interviews with six cancer patients and two focus groups with 17 nurses from oncology wards and outpatient clinics. Sampling was purposive to ensure maximum socio-demographic variability. Patient interviews preceded the focus groups and informed their discussion guides. Data were transcribed verbatim and analyzed using Braun and Clarke’s thematic analysis approach [35], with methodological rigor ensured through Guba and Lincoln’s criteria for qualitative research trustworthiness [36]. The study was conducted between January and June 2021. Further methodological details are available in Tolotti et al. [7].

Based on the findings from Bonetti et al. [5] and themes emerging from the qualitative study [7] a preliminary pool of items was developed to capture nursing behaviors perceived by both patients and nurses as critical for promoting patient engagement in cancer care. The items were originally drafted in Italian.

Consensualisation procedure and face validity: Following item development, a consensualization process was conducted using the member checking technique [35,37], whereby participants from the qualitative phase reviewed the items to verify alignment with their experiences. Subsequently, face validity was assessed [34], focusing on content relevance, acceptability, and clarity. Based on feedback from patients and nurses, minor revisions were made to enhance comprehensibility and ensure the items accurately captured engagement-promoting nursing behaviors.

Phase 2: Nube Scale content validity – Modified e-Delphi procedure.

Content validity was assessed using a modified e-Delphi procedure [33], a structured consensus method widely used in instrument development. The panel included experts in patient engagement, questionnaire design, linguistics and oncological nursing. Three Delphi rounds were conducted between December 2022 and February 2023. Experts evaluated each item’s relevance to engagement-promoting nursing behaviors using a 4-point Likert scale (1 = totally irrelevant to 4 = totally relevant), following established guidelines [38,39]. Items rated as 3 or 4 were considered acceptable. Participants also provided suggestions to improve semantic clarity and content relevance. Revisions were integrated iteratively and re-evaluated in the second and third round [33,38,39].

Content Validity Index (CVI) calculations: Content validity metrics were computed for each Delphi round. The Item-Level Content Validity Index (I-CVI) represented the proportion of experts rating an item as 3 or 4. The Scale-Level CVI (S-CVI) indicated the proportion of items rated 3 or 4 across the full scale, while the Average S-CVI (S-CVI/Ave) was calculated as the mean I-CVI across all items [38,39]. Thresholds for adequacy were set at ≥0.83 for I-CVI and ≥0.90 for both S-CVI and S-CVI/Ave. In the third round, inter-rater agreement was further assessed using Cohen’s Kappa [40].

Phase 3. NuBE Scale piloting and psychometric testing.

The NuBE Scale developed in Phase 1 and validated for content by experts in Phase 2 was tested with patients. Before data collection, the scale was pilot tested with a subgroup of ten patients to assess readability and comprehensibility. No changes to the items were requested, and participants reported that the scale was easy to use. Data collection with patients took place from May 2023 to September 2023.

Questionnaire structure: The first section of the questionnaire included the NuBE Scale, in which patients were asked to rate the importance of each described nursing behavior in enhancing their engagement in disease management. Responses were recorded on a 7-point Likert scale, ranging from “not important at all” to “very important”.

The second section collected demographic information, including sex, age, level of education, type of oncological disease, presence of other chronic conditions, and frequency of hospitalizations in the past year.

To facilitate participation, we administered paper-based questionnaires, as most patients were older and had limited proficiency with digital devices.

Instruments for assessing convergent validity: To evaluate the convergent validity of the NuBE Scale, the questionnaire administered to patients included several validated instruments (see below). The selection of instruments for convergent validity was guided by established theoretical models of patient engagement, which conceptualize engagement as a multidimensional construct encompassing relational, cognitive, emotional, and motivational components. Each instrument was selected to capture a construct theoretically adjacent to engagement-promoting nursing behaviors, while remaining conceptually distinct from the dimensions assessed by the NuBE Scale.

Healthcare Climate Questionnaire (HCCQ) [41,42]: The Health Care Climate Questionnaire (HCCQ) was used to assess patients’ perceptions of the relational quality with their primary healthcare provider, serving as a proxy for patient engagement. The 14-item scale evaluates patient-caregiver interactions using a 7-point Likert scale (“totally disagree” to “totally agree”). Although an officially validated Italian version of the Health Care Climate Questionnaire (HCCQ) has not yet been published, the instrument has been translated into Italian and successfully used in previous studies conducted with Italian samples. For instance, Graffigna et al. [43] and Barello et al. [44] adopted the Italian-translated version of the HCCQ to assess perceived autonomy support in healthcare settings, reporting excellent internal consistency and confirming the unidimensional structure of the scale. These findings support the use of the instrument in the Italian context, even in the absence of a formally validated version.

The Cronbach’s alpha for this instrument in our study was 0.771, indicating good internal consistency.

Patient Health Engagement Scale (PHE-S) [45]: The PHE-S measures patients’ engagement in their healthcare journey, based on the Patient Health Engagement Model. This model conceptualizes engagement as a psychosocial process that progresses from disempowerment and vulnerability to full proactive involvement in healthcare. The Patient Health Engagement Scale (PHE-S) has been originally developed and validated in the Italian context, showing solid psychometric properties [45].

The scale consists of five self-reported items, each rated on a 7-point Likert scale (1 = low engagement; 7 = high engagement). Lower scores indicate a passive or emotionally distressed approach to healthcare, whereas higher scores reflect active and empowered health management. The total score is obtained by averaging all responses, providing an overall measure of patient engagement (Cronbach’s alpha = 0.863).

First Question of the SF-12 [46,47]: To assess quality of life, we included the first question from the SF-12 questionnaire: “In general, would you say your health is…?”. We used the Italian version of SF-12 [46,47]. Patients responded using a 5-point Likert scale, ranging from “Excellent” to “Very Poor”. This item was included as a global self-rated health indicator rather than as a comprehensive measure of quality of life. The use of single-item self-rated health measures is well established in the literature [48] and widely adopted in health research as a valid proxy for overall health perception. In the present study, this item was included to explore associations between engagement-promoting nursing behaviors and patients’ overall health perception, rather than to capture the multidimensional construct of quality of life.

Newcastle Satisfaction with Nursing Scale (NSNS) – Selected Items [49,50]: To evaluate satisfaction with nursing care, we selected two general items (QN1 and QN2) from the Newcastle Satisfaction with Nursing Scale (NSNS) in Italian version [49,50]. Given the length of the full questionnaire, we focused on these two key questions, which assess:

1. The overall quality of the patient’s clinical journey
2. The quality of nursing care received

These items were selected because they capture global evaluations of the care experience and satisfaction with nursing care, which are theoretically aligned with engagement-promoting nursing behaviors. The Newcastle Satisfaction with Nursing Scale has been validated in the Italian context [49,50], and the selected items reflect high-level satisfaction judgments rather than specific procedural aspects of care.

Patients rated these items on a 7-point Likert scale, ranging from “Very Bad” to “Excellent.”

Adherence to Refills and Medications Scale (ARMS) [51]: The ARMS was used to measure therapeutic adherence. This scale consists of 12 questions assessing how often patients forget or intentionally skip aspects of their treatment regimen. Example items include:

• How often do you forget to take your medicine?
• How often do you decide not to take your medicine?
• How often do you forget to get prescriptions filled?

Responses are recorded on a 4-point Likert scale, ranging from “None” to “All.” The Adherence to Refills and Medications Scale (ARMS), although not formally validated in Italian, has been previously used in studies involving Italian patient populations, supporting its applicability in this context [52,53].

Inclusion and exclusion criteria.

Patients were eligible for inclusion if they:

• Were 18 years or older
• Were proficient in Italian
• Were able to complete a paper-based questionnaire

There were no restrictions regarding the oncological disease. Only patients receiving palliative care or those with cognitive or mental health issues, that could interfere with their ability to complete the questionnaire, were excluded.

Recruitment and data collection.

Patient recruitment was conducted by five nurses working in inpatient wards and outpatient clinics during the data collection period. All nurses received training on data collection procedures from the first and second authors, who also supervised the entire process to ensure consistency and adherence to the protocol.

To ensure a balanced representation of patients from different clinical contexts, we predefined the number of questionnaires for each ward and outpatient clinic. This allocation was based on the size of each unit and the patient turnover rate, ensuring proportional sampling across different settings.

Sample size.

According to Tabachnick et al. [54] and Watson [55] a minimum of 200 participants is recommended for Principal Component Analysis (PCA) and Exploratory Factor Analysis (EFA). Additionally, a ratio of five participants per item is generally considered sufficient.

Gorsuch [56] and Heckler [57] suggest a minimum subject-to-item ratio of 5:1 for EFA, but they emphasize that this ratio is only acceptable under specific conditions. Both authors note that higher ratios are generally preferable for more robust factor analysis. Gorsuch [56] further specifies that the 5:1 ratio is appropriate only when variable communalities are high, meaning that the selected variables share a substantial amount of variance with others and that each factor has multiple highly loaded variables. Our preliminary analyses confirmed that these conditions were met.

Based on these recommendations and considering that the initial version of the NuBE Scale consisted of 48 items, we selected a sample size of 250 participants to ensure adequate statistical power for psychometric testing.

Descriptive statistics.

To describe the sample characteristics, we calculated frequencies and percentages for categorical variables and means with standard deviations for continuous variables. Moreover, we performed an item-level analysis to assess the quality of each item. Descriptive statistics were computed.

Exploratory factor analysis.

The objective was to develop a refined scale with fewer items than those identified in the e-Delphi procedure, ensuring that each factor was supported by an adequate number of items to allow for stable estimation and reliable interpretation. In line with common psychometric recommendations, a minimum of approximately five items per factor was considered a desirable guideline rather than a rigid criterion, as factors defined by a larger number of well-loading items are generally more stable and reliable, particularly in exploratory analyses [32,58,59].

Nevertheless, decisions regarding factor retention were ultimately based on a combination of statistical criteria, conceptual coherence, and interpretability, rather than on item count alone. So, scale refinement was guided by the identification of factors supported by a sufficient number of items to ensure stability and interpretability, with salient factor loadings (≥.40) and satisfactory internal consistency, as reflected by high reliability indices.

Given the exploratory nature of the study, multiple exploratory factor analyses (EFA) with an oblique Promax rotation were conducted using the principal axis factoring extraction method [59,60] which is appropriate when the goal is to identify underlying latent constructs and when the data may not meet the assumption of multivariate normality [58].

The number of factors was determined using the eigenvalue > 1 criterion [61] At each step, items were eliminated if they:

• Had low factor loadings (< 0.4), consistent with commonly adopted interpretative thresholds [59,62]
• Showed poor correlations with other items
• Had high cross-loadings on multiple factors, defined as secondary loadings ≥ .30 or differences between primary and secondary loadings smaller than.20, indicating insufficient factorial discrimination [54,59]
• Exhibited inconsistencies in shape indices

Additionally, factors were reduced if they did not have at least three or four items with significant loadings.

The appropriateness of EFA was assessed using:

• Kaiser–Meyer–Olkin (KMO) measure (considered excellent if > 0.90) [63]
• Bartlett’s test of sphericity (to confirm factorability of the correlation matrix)
• Explained variance, factor loadings, and eigenvalue >1 to define the final factorial structure

Reliability analysis.

Internal consistency reliability was determined using Cronbach’s alpha, with the following thresholds:

•  ≥ 0.70 = Acceptable
•  ≥ 0.80 = Good
•  ≥ 0.90 = Excellent [64]

Descriptive statistics were computed for each item in the final factor solution and for the overall factor scores. To further assess scale adequacy and conduct item analysis, we calculated:

• Item-total correlations
• Cronbach’s alpha if item deleted

Convergent validity.

To evaluate convergent validity, factor scores were correlated with scores from:

• Healthcare Climate Questionnaire (HCCQ) [41,42]
• Patient Health Engagement Scale (PHE-S) [45]
• First question of the SF-12 scale (for quality of life) [46,47]
• Two items from the Newcastle Satisfaction with Nursing Scale (NSNS) [49,50] (assessing satisfaction with nursing care)

Pearson correlation coefficients were computed, with a significance level (α) of 0.05.

Consensualisation procedure and face validity.

Face validity was assessed with six patients and six nurses from the qualitative phase. Participants confirmed that the items accurately reflected their experiences and were appropriate for capturing engagement-promoting nursing behaviors. Only minor revisions were made to improve clarity and comprehensibility.

Sample’s description.

A total of 250 oncology patients participated in the study (58.8% female; age range: 16–88 years; M = 63.2, SD = 14.3). All had a confirmed cancer diagnosis, and 29.2% reported additional chronic conditions. Overall, 44% had experienced at least one hospitalization in the previous year (M = 2.3). Educational attainment varied, with the majority having completed secondary (16.4%) or high secondary education (49.6%). Further socio-demographic details are provided in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Characteristics of participants (n = 250).

https://doi.org/10.1371/journal.pone.0342693.t002

Psychometric properties on the NuBE scale.

Structural and construct validity: The aim was to develop a concise yet psychometrically robust scale; the initial version included 48 items. Descriptive statistics for all items are reported in Supplementary materials. To refine the scale and optimize its psychometric structure, successive exploratory factor analyses (EFA) with Promax rotation were conducted on the complete dataset (n = 250; no missing data). Items were excluded based on low factor loadings (<.40), substantial cross-loadings, or communalities below.20. A strong inter-item correlation structure was confirmed, and no outliers were identified. The final EFA yielded a four-factor solution comprising 29 items (Supplementary Materials), confirmed also by the Scree Plot (Supplementary material), following the procedure proposed by Cattell [65], with significant inter-factor correlations supporting the use of oblique rotation. Sampling adequacy was excellent (KMO = 0.925; Bartlett’s test: χ² = 7184, df = 406, p < .001). The four factors explained 70.0% of the total variance, with eigenvalues ranging from 6.1 to 3.6 and individual contributions of 20.9%, 18.5%, 18.2%, and 12.5%, respectively. Full factor loadings, communalities, and reliability indices are provided in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. 4-Factor final solution (EFA).

https://doi.org/10.1371/journal.pone.0342693.t003

Psychometric analysis identified four factors representing the construct of Nursing Behavioral Engagement: (1) Acknowledgment of Patient’s Uniqueness, (2) Meeting Patient’s Knowledge Expectations, (3) Fostering Patient’s Motivation, and (4) Valuing Patient’s Informal Caregivers. Internal consistency was excellent across all factors (α = .953,.890,.941, and.920, respectively). Factor 1 comprised items 19, 20, 21, 23, 25, 26, 27, and 29; Factor 2 included items 12, 14, 15, 16, 34, 40, 41, 45, and 46; Factor 3 included items 10, 11, 28, 35, 37, 38, and 39; and Factor 4 included items 3, 4, 9, 47, and 48. The scale does not yield a total score; instead, four subscale scores are calculated by summing item responses within each factor. Higher scores reflect greater levels of the corresponding construct. Table 4 presents the inter-scale correlations, which indicate strong relationships among the factors.

Reliability analysis: Item-total correlations ranged from 0.586 to 0.884, indicating strong internal consistency. Cronbach’s alpha values remained stable when individual items were removed, confirming their contribution to the scale. These results support the NuBE Scale’s reliability as a measure of nursing behaviors that foster patient engagement.

Convergent validity: To evaluate the convergent validity of the NuBE Scale, we computed bivariate Pearson correlations between the factor scores and the scores of the two subscales and total score of the Healthcare Climate Questionnaire (HCCQ) [41,42] and the Patient Health Engagement Scale (PHE-S) [45]. Additionally, correlations were examined with the first question of the SF-12 [46,47], which assesses quality of life, and with two items of the Newcastle scale measuring satisfaction with nursing care received [49,50]. Convergent validity was examined by testing theory-driven hypotheses regarding the expected associations between NuBE factors and conceptually related constructs.

The analysis aimed to test specific hypotheses regarding the relationships between the NuBE Scale and other validated instruments. In particular, it was hypothesized that the first two dimensions of the NuBE Scale, Acknowledgment of Patient’s Uniqueness and Meeting Patient’s Knowledge Expectations, would show significant correlations with the HCCQ [41,42], as both constructs relate to the quality of patient-provider interactions and the level of autonomy support provided by healthcare professionals. Similarly, the PHE-S Scale [45], which measures patient engagement as a psychological process, was expected to correlate with the third factor of the NuBE Scale, Fostering Patient’s Motivation, since both constructs emphasize the role of emotional and motivational support in enhancing patient involvement.

Furthermore, it was expected that the two ad hoc items assessing satisfaction with nursing care received would correlate with all factors of the NuBE Scale, as the scale captures key nursing behaviors that contribute to patient satisfaction. However, no specific hypothesis was formulated regarding the first item of the SF-12 Scale [46,47], which assesses general health condition, as this measure does not directly align with the dimensions of the NuBE Scale.

The results of these correlation analyses are presented in Table 4, providing further insights into the relationships between the NuBE Scale and established measures of patient engagement, satisfaction, and perceived quality of care.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Pearson correlation between NuBE Scale and HCCQ, PHE-S, SF12, and NSNS item ad hoc.

https://doi.org/10.1371/journal.pone.0342693.t004

All factors of the NuBE scale show significant and meaningful correlations with the total score of the HCCQ scale [41,42] and the two items regarding satisfaction with nursing care received [49,50] (particularly the first three factors). However, no significant correlations emerge for any factor with either the PHE-S [45] or the first item of the SF-12 [46,47]. Finally, very strong and significant correlations emerge among the four factors of the NuBE scale, ranging from.417 to.757.