Ethics Statement

This study was approved by the Ethics Committee of the Policlinico of Bari (Approval number 9881, dated 11 November 2025). All participants voluntarily participated in the study and provided written informed consent before questionnaire administration. Participation was anonymous, and all procedures were conducted in accordance with the Declaration of Helsinki and relevant national regulations.

Study design and objectives

The study was conducted in a national multicenter academic setting involving Italian universities and affiliated teaching hospitals. The primary objective of this study was to develop and validate a structured questionnaire assessing awareness, knowledge, attitudes, and practices (KAP) regarding NTDs among Italian medical students, recent graduates, and medical residents. The tool was designed to evaluate the extent of formal training on NTDs, diagnostic preparedness, clinical experience, and interest in future education or research initiatives.

Preparatory phase: Questionnaire development and first expert panel selection

The initial questionnaire was designed based on the WHO list of priority NTDs and current Italian university curricula in infectious and tropical diseases. The first version of the questionnaire was further informed by a non-systematic review of the literature of published studies and WHO reports, focusing on global priorities for NTD control, diagnostic challenges, and training gaps in medical education, until February 28th, 2025 (Fig 1). The survey comprised 43 questions divided into five sections:

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Flowchart of the questionnaire development and validation process.

https://doi.org/10.1371/journal.pntd.0014644.g001

• Demographic Data (age, gender, university, current position).
• Academic Training (exposure to NTD topics in curricula, perceived preparedness).
• Knowledge Assessment (clinical vignettes and diagnostic questions).
• Attitudes (perceptions of NTD relevance and One Health perspectives, Likert scale).
• Practice (experience with suspected cases, diagnostic requests, and extracurricular training).

Questions were designed as multiple-choice, 5-point Likert-scale, or dichotomous items. Pilot testing was conducted among a small sample (n ≈ 45) to refine clarity.

To ensure methodological rigour and content relevance, a multidisciplinary expert panel was assembled according to pre-defined criteria:

• Representation of national scientific societies: inclusion of experts from the Società Italiana di Parassitologia (SoIPa, www.soipa.it <http://www.soipa.it/>) and the Società Italiana di Medicina Tropicale e Salute Globale (SIMET, https://www.simetweb.eu <https://www.simetweb.eu/>), both members of the Italian Network for NTDs [15]
• Diversity of expertise: parasitologists, infectious disease specialists, clinicians, public health experts, and medical educators.
• Professional experience: at least half of the panel with ≥10 years of experience in tropical medicine, parasitology, or infectious disease education.
• Academic and policy perspective: participation of senior academics and contributors to national strategies on tropical and parasitic diseases.
• Early-career representation: at least two early-career researchers or residents actively engaged in NTD research or education.
• Student representation: inclusion of medical student representatives to ensure the questionnaire was relevant and accessible for its target audience.
• Gender balance.
• Geographical diversity: panelists drawn from medical schools and teaching hospitals across Northern, Central, and Southern Italy.

The final expert panel included 36 members, comprising representatives from SoIPa, SIMET, infectious disease specialists, parasitologists, medical educators, and student delegates, ensuring that the survey instrument was pedagogically robust, clinically accurate, and aligned with learner needs.

Delphi survey

The Delphi method was used to gather expert opinions independently and reach consensus through iterative online surveys. Responses were anonymized, and only study coordinators accessed individual data, ensuring a double-blind process that minimized dominance bias. After each round, aggregated feedback was shared, enabling participants to revise their responses in light of group input. This structured and controlled approach promoted convergence of opinions while preserving individual autonomy.

Each question was evaluated using a 5-point Likert scale, with different assessment criteria based on question type:

• Questions from the Demographic Data were assessed for clarity, relevance, and appropriateness for the study objectives.
• Questions from Academic Training, Knowledge Assessment, Attitudes and Practices were evaluated for clarity, relevance, appropriateness, and ability to capture the internal dimensions of the study.

Participants were encouraged to provide qualitative feedback, including suggestions for additional indicators, refinements to survey items, and general comments to enhance the survey’s robustness.

Content validity assessment

A single researcher conducted all statistical analyses to guarantee the validity of the development procedure. The content validity of the survey was assessed using two widely recognized indices: the Item Content Validity Index (I-CVI) and the Scale Content Validity Index (S-CVI/Ave) [16].

• I-CVI (Item Content Validity Index): The I-CVI was calculated for each item by dividing the number of experts who rated the item as “4” (relevant) or “5” (highly relevant) by the total number of experts. We pre-specified item-level thresholds to increase the conservativeness of content validation for a high-stakes instrument. In addition to the conventional cut-offs (I-CVI ≥ 0.78 acceptable; 0.70–0.78 revise; < 0.70 reject), we applied a more stringent banding (I-CVI ≥ 0.80 acceptable/revisions; 0.75–0.79 revise; < 0.74 reject). To ensure transparency, we report results under both schemes and provide modified kappa and 95% CIs for I-CVI.
• S-CVI/Ave (Scale Content Validity Index, Average Method): This index evaluates the overall validity of the survey by calculating the average I-CVI value across all items. An S-CVI/Ave ≥ 0.90 was considered indicative of strong content validity for the entire scale. Items with an S-CVI/Ave < 0.80 were flagged for major revision or removal, while those with an S-CVI/Ave between 0.80 and 0.89 were identified for minor revision.

The qualitative feedback supplemented the quantitative results from I-CVI and S-CVI/Ave calculations, enabling the formulation of data-informed decisions regarding the retention, revision, or exclusion of items. Items that were excluded from the final survey did not meet the I-CVI threshold in either of the Delphi sessions, despite the qualitative modifications.

To evaluate the degree of consensus among experts, the Interquartile Range (IQR) was computed for each item using the Likert scale ratings (1 –5) provided by the panel [16]. As a measure of variability in expert ratings, the IQR was calculated as the difference between the third quartile (Q3, 75th percentile) and the first quartile (Q1, 25th percentile):

• A low IQR (≤ (1) suggested that there was a high degree of agreement among experts, indicating a firm consensus on the item’s clarity, relevance, and appropriateness.
• Responses with a high IQR (> (1) exhibit a higher degree of variability, which may indicate a potential disagreement that necessitates additional revision or clarification in subsequent Delphi cycles.

By computing the IQR across expert ratings (1 = not pertinent to 5 = highly relevant), each survey item was analyzed. Items with an IQR between 0 and 1 were deemed to have achieved consensus, while those with an IQR > 1 were identified for further discussion and potential modification in the subsequent Delphi round. The IQR results were analyzed in conjunction with CVI measures to guarantee a thorough evaluation of content validity, thereby establishing a robust evaluation framework for the finalization and refinement of the survey.

Second expert panel

To strengthen content validity, a second external panel of experts was recruited, composed of specialists not involved in the initial Delphi rounds. This panel included infectious disease clinicians, parasitologists, educators, and representatives from scientific societies, providing an independent review of the revised survey. The same evaluation criteria (I-CVI, S-CVI/Ave, IQR) were applied to confirm the clarity, relevance, and robustness of the questionnaire. This step minimized potential bias from repeated exposure among the first panel and enhanced the generalizability of the instrument.

Reliability testing

A voluntary sample of 96 participants, including medical students, recent graduates, and residents from multiple Italian universities and specialties, was recruited to evaluate the psychometric reliability of the questionnaire. Participants were recruited through voluntary convenience sampling using institutional academic networks and professional contacts. Participants were selected to reflect the diversity of the survey’s target population, thereby enhancing representativeness and applicability across different stages of medical training.Given the overall sample n = 96 (and the smaller subgroup cell sizes), formal Differential Item Functioning (DIF) testing (Mantel-Haenszel, logistic DIF, or multi-group IRT) was not planned as a primary analysis due to limited power and unstable parameter estimates in small samples. Instead, we adopted a light DIF strategy to appraise item fairness across subgroups (students, residents, and recent graduates):

• Item performance by subgroup: inspection of item difficulty (p) and point-biserial discrimination separately within each subgroup.
• Reliability invariance: calculation of Cronbach’s α ± / KR-20 within subgroups to check consistency of internal reliability.
• Exploratory logistic checks (if flagged): for any item showing conspicuous subgroup discrepancies, we fitted binary logistic models (item score ~ total knowledge score + group) to probe potential bias, interpreting results cautiously and without formal multiplicity adjustments.

Together, these analyses confirmed that the instrument demonstrated high reproducibility, strong internal coherence, and robust reliability metrics, supporting its suitability for large-scale implementation in medical education and training contexts (S1 Data).

Statistical analysis

Data were analysed using descriptive statistics (mean, standard deviation, and frequencies). Knowledge scores were computed as the percentage of correct responses. Content validity was assessed using the I-CVI, S-CVI/Ave, and IQR for consensus evaluation [17]. Reliability testing included test–retest analysis (Cohen’s kappa and intraclass correlation coefficients) and internal consistency assessment with Cronbach’s α for Likert-scale items and KR-20 for dichotomous items. All analyses were conducted in R (version 4.4.1, R Foundation for Statistical Computing, Vienna, Austria).

Expert characteristics

The characteristics of the responders of both panels are reported in Table 1. Most participants (78%) had at least 10 years of experience at the time of the survey. All the inclusion criteria were met.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of the 53 experts composing both Panels.

https://doi.org/10.1371/journal.pntd.0014644.t001

Round 1 findings

The first Delphi round included 43 questions and was conducted from February 26, 2025, to March 20, 2025. Of the first invited expert panel, 28/36 (78%) completed the survey.

One reviewer analyzed the responses and assessed content validity using the S-CVI/Ave and I-CVI indices. Most questionnaire items demonstrated strong content validity and expert consensus during the first Delphi round. Four items with low content validity indices were removed after expert review because they were considered redundant, insufficiently informative, or potentially affected by recall bias. The questionnaire demonstrated excellent scale-level content validity across all evaluated domains (S-CVI/Ave range: 0.95–0.97), exceeding the conventional ≥0.90 benchmark. Detailed item-level modifications are reported in S1 Text.

The IQR analysis demonstrated a high level of consensus among experts: the mean IQR was 0.57 (median = 1.0, SD = 0.49; range: 0.0–1.25). Notably, 40.5% of items achieved an IQR of 0, indicating full agreement, while 97.6% of items had an IQR ≤ 1, confirming strong consensus across nearly all questions. Only 2.4% of items showed an IQR slightly above 1, suggesting minor variability in expert ratings.

The initial KAP questionnaire was also intended for nursing students; however, expert feedback indicated that the nursing curriculum in Italy focuses primarily on patient management rather than detailed disease-specific content. As a result, several knowledge-based items risked misrepresenting the actual scope of nursing education. To ensure the validity and fairness of the instrument, nursing students were subsequently excluded from the final survey sample.

Round 2 findings

The second Delphi round included the selected 39 questions. Of the first invited expert panel, 29/36 completed the survey, resulting in an 81% response rate. The second round was conducted from April 17, 2025, to May 12, 2025.

One reviewer analysed the responses and assessed content validity using the S-CVI/Ave and I-CVI indices: most items demonstrated strong content validity during the second Delphi round and required only minor linguistic refinements. A small number of items underwent minor revisions, while one item with low relevance scores was removed following expert consensus because it was considered insufficiently reliable for assessing curricular exposure. Detailed item-level validity results are reported in the S1 Text.

The S-CVI/Ave in Round 2 is 0.96 for Clarity (C), 0.98 for Relevance (R), 0.98 for Appropriateness (A), and 0.97 for Ability to Capture the Intended Dimension. The overall S-CVI/Ave across dimensions is 0.9736, confirming excellent content validity.

The IQR analysis from this round demonstrated near-complete consensus among experts. The mean IQR was 0.05 (median = 0.00; SD = 0.22; range: 0.00–1.00). A total of 94.7% of items achieved an IQR of 0, indicating unanimous agreement, and 100% of items scored ≤1, confirming full consensus across all questionnaire items. No item exceeded an IQR of 1.

Second panel findings

A second independent panel of experts was convened to validate the revised questionnaire developed through the initial two-round Delphi process. The panel consisted of other external 17 experts (11 women and 6 men), including 9 academics, 6 clinicians, 1 early-career researcher, and 1 student representative, ensuring multidisciplinary and learner-centred perspectives.

This second review confirmed the robustness of the revised tool, further enhancing its clarity, cultural relevance, and applicability to the target population. The second expert panel was conducted from June 30, 2025 to July 20, 2025.

The participation in the second expert panel was excellent, with 100% of invited experts submitting feedback. One reviewer analysed the responses and assessed content validity using the S-CVI/Ave and I-CVI indices: Most questionnaire items demonstrated strong content validity and required only minor linguistic refinements. A small number of items underwent minor wording revisions to improve clarity and consistency, including refinement of demographic terminology to avoid ambiguity. No items required removal during this phase, confirming the robustness of the questionnaire refinement process. Detailed item-level validity results are reported in the S1 Text.

The S-CVI/Ave for Clarity is 0.98, for Relevance 0.95, for Appropriateness 0.96, and for Ability to Capture the Intended Dimension 0.96; the overall S-CVI/Ave across dimensions is 0.96, indicating excellent content validity.

The IQR analysis further confirmed strong expert consensus. The mean IQR was 0.07 (median = 0.00; SD = 0.24; range: 0.00–1.00). A total of 91.9% of items achieved an IQR of 0, indicating unanimous agreement among experts, and 100% of items scored ≤1, confirming full consensus across all questions in this round.

The participation and feedback quality allowed the validation to be completed in a single round, given the high S-CVI scores already achieved in the initial Delphi phase.

Reliability and validity

Following two iterative Delphi rounds and an external single-round validation confirming high content validity and consensus, the finalised questionnaire was field-tested in a sample of 96 participants (mean age 26.4 years, SD 3.2; 51 students, 34 residents: 12 surgical, 18 clinical, 4 services—and 11 recent graduates; 45% South, 21% Centre, 34% North) (Tables 2, 3). The study population included Italian medical students, recent graduates, and medical residents enrolled or trained at universities and teaching hospitals across Northern, Central, and Southern Italy. The analyses below report reliability, stability, item performance, known-groups, convergent validity, and fairness checks using the light DIF strategy.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Structure of the questionnaire and content domains.

https://doi.org/10.1371/journal.pntd.0014644.t002

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Psychometric properties of analytical questionnaire domains.

https://doi.org/10.1371/journal.pntd.0014644.t003

Internal consistency

• Attitudes (5 Likert items): Cronbach’s α = 0.863, indicating good internal consistency.
• Knowledge (15 dichotomous items, scored correct/incorrect): KR-20 = 0.82, indicating good reliability.

1. Temporal stability (test–retest)
2. A subsample of 40 participants repeated the survey after 2 weeks.
3. •. Attitudes (composite of the 5 items): ICC (2,1) = 0.89; IC95% 0.83–0.93, showing good-to-excellent stability. SEM = 1.16; MDC95% = 3.22. The SEM of 1.16 reflects small absolute error, so individual changes ≥3.22 points (MDC95) can be considered real (beyond measurement error).
4. •. Practice (7 binary items in the Practice section): Cohen’s κ across items ranged 0.72–0.84; a representative κ for an NTD-related screening item was 0.79, indicating substantial agreement

Item analysis (knowledge)

All 15 knowledge items were profiled: item difficulty (p) ranged 0.42–0.78, and point-biserial discrimination ranged 0.28–0.52. No items met the pre-specified criteria for extreme difficulty (p < 0.20 or p > 0.90) or poor/negative discrimination; thus, no removals were warranted at this stage.

Known-groups validity

Residents outperformed students on total knowledge (sum of 15 items): Welch’s t = 2.50, p = 0.014, Cohen’s d = 0.56 (a moderate effect), supporting known-groups validity.

Construct validity

Exploratory factor analysis of the Attitudes items showed adequate sampling adequacy (KMO = 0.79; Bartlett’s test p < 0.001) and supported a unidimensional structure, explaining 58.6% of the variance, with all items showing salient loadings (>0.60). Confirmatory factor analysis confirmed good model fit (CFI = 0.96; TLI = 0.94; RMSEA = 0.06; SRMR = 0.04), with all standardized loadings statistically significant, supporting the construct validity of the Attitudes scale.

Convergent validity

Self-reported preparedness correlated positively with total knowledge: Spearman’s ρ = 0.35, p < 0.001, consistent with moderate convergent validity.

Fairness (light DIF)

Formal DIF testing was not performed given n = 96. Instead, we inspected item difficulty and discrimination by subgroup (students, residents, and recent graduates) for all 15 knowledge items, computed Cronbach’s α (attitudes, 5 items) and KR-20 (knowledge, 15 items) within subgroups and ran exploratory logistic checks for any flagged items. No systematic subgroup anomalies warranting item removal were identified.

The complete dataset and psychometric analyses are provided in S1 Data. The final validated questionnaire is provided in S1 Text.

Strenghts

A further strength of this study lies in the structure of the questionnaire, which combines factual items with short clinical cases. These case-based questions were included to assess not only theoretical knowledge but also the ability to recognise NTD-related presentations in practice. In addition, the Delphi process highlighted the value of involving students, early-career clinicians, and experts in the co-design and validation of educational tools, giving a meaningful role to those who are both current learners and future practitioners. The high level of consensus reached across diverse expert panels underscores the value of a participatory approach and ensures that the questionnaire reflects both pedagogical needs and clinical realities. Importantly, the composition of our panels was intentionally balanced, with representation of women and men, academic and non-academic professionals, and experts from Northern, Central, and Southern Italy, to minimise bias and ensure that the instrument incorporated a broad range of perspectives.

Limitations

This study has some limitations. The relatively small sample size used for psychometric testing, particularly for confirmatory factor analysis and subgroup comparisons, may limit the robustness of some estimates. Therefore, these findings should be interpreted cautiously. In addition, the focus on Italian medical trainees may limit the generalisability of the results to other settings. In addition, the voluntary recruitment strategy may have introduced selection bias, and the self-reported nature of KAP surveys may be influenced by recall and social desirability biases. Finally, the limited sample size restricted the possibility of performing more advanced psychometric analyses, including formal differential item functioning testing. However, the methodological framework is transferable and could be applied to other countries seeking to assess gaps in tropical medicine education.