2.1 PBL in engineering education

Problem-Based Learning (PBL) has gained significant momentum in engineering education in recent years, driven largely by the demands of Industry 4.0 and the increasing emphasis on interdisciplinary, experiential learning. In biomedical engineering, a longitudinal three-year study conducted at Georgia Tech and Emory University implemented an advanced PBL framework tailored for AI applications in biomedicine. The program involved 92 undergraduate and 156 graduate students, resulting in notable outcomes including 16 student publications and the development of computational methods addressing real-world biomedical problems [15]. Similarly, in electromechanical engineering, research introduced a PBL model centered on real-world problem-solving and critical thinking. Student feedback highlighted high levels of satisfaction, especially in areas such as problem analysis, solution design, and the integration of theoretical knowledge with industrial practice [16]. Industrial engineering programs have also embraced PBL methodologies; for instance, the “Insights 4.0” initiative combined PBL with project-based learning, leading to increased student engagement and improved academic performance. Participants reported higher final grades alongside enhanced collaboration skills and both technical and soft competencies, underscoring PBL’s effectiveness in this domain [17]. Moreover, during the COVID-19 pandemic, engineering programs adapted by integrating at-home laboratories with PBL approaches to maintain educational quality [18]. This strategy not only preserved academic performance but also elevated student motivation and self-efficacy, demonstrating PBL’s efficacy in remote and hybrid learning environments.

Collectively, these examples underscore PBL’s versatility and profound impact within engineering education, equipping students with critical thinking, practical problem-solving abilities, and the adaptability essential for navigating today’s complex, interdisciplinary challenges. As educational demands evolve, PBL continues to stand out as a robust framework for preparing future-ready engineers.

2.2 MCGDM in course evaluation

MCGDM has proven effective in addressing the complexity inherent in student course evaluation. Traditional evaluation methods often rely on single or limited criteria, which fail to capture the multidimensional nature of learning outcomes. In contrast, MCGDM provides a structured framework that integrates perspectives from multiple stakeholders, such as students, instructors, and educational experts, thereby enabling a more comprehensive and equitable assessment. For example, an empirical study at Gulf University employed the Analytic Hierarchy Process (AHP) to evaluate student contributions in cooperative learning, successfully resolving conflicts in subjective assessments and facilitating more accurate credit distribution [19]. Additionally, MCGDM methods have been applied to optimize teaching plans and grading processes in university courses, as demonstrated in studies on energy market education and chemistry performance evaluations [20].

A key strength of MCGDM lies in its capacity to simultaneously consider a broad range of criteria without reducing evaluation outcomes to oversimplified metrics, including cognitive skills, creativity, teamwork, and presentation quality. Techniques such as AHP, Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), and Simple Additive Weighting (SAW) enable the structured weighting and ranking of these diverse factors.

This approach is exemplified by Kara and Yıldırım’s study [25], which employed AHP and TOPSIS to assess high school chemistry students’ performance tasks, ensuring balanced consideration of psychomotor, cognitive, and affective domains. This approach is further illustrated by Yıldırım et al. [21], who developed an integrated intuitionistic Z-AHP and Z-TOPSIS methodology for hydrogen storage technology selection, explicitly incorporating both judgment reliability and decision-makers’ hesitancy under intuitionistic fuzzy environments. Moreover, MCGDM frameworks have demonstrated enhancements in fairness and transparency, particularly within group-based learning environments and interdisciplinary evaluations [26,27]. These methods also provide systematic alignment between teaching outcomes and program objectives, a critical consideration in outcome-based education models [28].

To further address the fuzziness and subjectivity inherent in human evaluations, researchers have integrated fuzzy logic, hesitant fuzzy sets, and probabilistic reasoning into MCGDM models. For instance, the Fuzzy Ordered Priority Approach for Multi-Criteria decision-making (FOPA-MC) model combines fuzzy logic with group decision-making for peer evaluation, effectively capturing the vagueness of subjective inputs [22]. Similarly, probabilistic hesitant fuzzy TODIM (an acronym in Portuguese for Interactive and Multi-Criteria Decision Making) and Evaluation based on Distance from Average Solution (EDAS) methods have been applied in evaluating Chinese language instruction quality, yielding promising results in mitigating evaluator hesitation and ambiguity [29].

Despite these advancements, several challenges remain. Fuzzy MCGDM models often involve high computational complexity and can be sensitive to initial assumptions such as criteria weights or decision-maker preferences. Furthermore, the need for adequate training of evaluators to use these advanced tools poses a risk to the consistency and reliability of assessment outcomes. A recent systematic review highlights the necessity for standardized frameworks and broader empirical validation across diverse disciplines [30].

In response to these challenges, researchers are exploring more robust tools for managing uncertain information. Among these, Z-number theory stands out for its ability to handle incomplete data while explicitly characterizing the reliability of information. Its unique capacity to fuse fuzzy and probabilistic information offers a flexible and accurate framework for modeling uncertainty, thereby more faithfully reflecting the complexity of real-world decision-making scenarios [23].

2.3 Z-number

In the realm of MCGDM, the complexity of decision structures and evaluation criteria often complicates the process, even for experts who are thoroughly familiar with the available alternatives. To address this challenge, the concept of the Z-number has been introduced to enhance the representation and measurement of the reliability associated with linguistic information. A Z-number is defined as , where A represents the expert’s evaluation or preference, and B denotes the degree of reliability or possibility associated with A. This dual-dimensional structure provides a novel approach to modeling both uncertainty and reliability within decision-making contexts [23].

To promote the understanding and application of Z-numbers, extensive research has been conducted, leading to the development of various extended forms, such as the Z*-number, Z-advanced number, and Z + -number, which have been applied to a range of MCGDM problems [31]. However, due to the inherent complexity of directly manipulating and solving Z-numbers in practical applications, scholars have explored transforming Z-numbers into more tractable representations, such as Triangular Fuzzy Numbers (TFNs) and trapezoidal fuzzy numbers, thereby leveraging existing fuzzy set methodologies to solve decision-making problems more efficiently [24].

Kang delved into Z-number operations and established a transformation rule to convert Z-numbers into fuzzy numbers and trapezoidal fuzzy numbers [32]. Božanić transformed Z-numbers into TFNs and applied them in camp location selection by integrating level-based weight assessment and multi-attributive ideal-real comparative analysis [33]. Garg proposed a method for converting Z-numbers into granulated Z-numbers, further expanding the application scope of Z-numbers [34].

These studies indicate that converting Z-numbers into TFNs or trapezoidal fuzzy numbers has become a widely accepted and validated approach for facilitating their practical application in decision-making. This transformation not only simplifies the computational process but also ensures the effective utilization of Z-number theory in various contexts. While Z-numbers have demonstrated remarkable capabilities in addressing uncertainty and reliability, they still fall short in fully capturing the subjectivity of Decision-Makers (DMs) in certain scenarios, such as mortise and tenon structure selection. To comprehensively address the intertwined issues of subjectivity, uncertainty, and reliability, this study integrates rough numbers with Z-numbers.

2.4 Multi-attributive border approximation area comparison (MABAC) method

Since the MABAC method was introduced by Pamučar and Ćirović, it has been applied to address MAGDM problems, such as the selection of transport and processing resources at logistics centers [35]. Its efficiency was demonstrated by comparing it with the COPRAS, TOPSIS, MOORA, and VIKOR methods. Subsequently, Pamučar improved the original MABAC technique, enhancing its applicability in more complex decision-making scenarios [36]. Zhao integrated the notion of cumulative prospect theory into the original MABAC approach and developed the IF-MABAC method, which is known as the cumulative prospect theory IF-MABAC method [37]. Liu and Zhang proposed a MABAC method for dealing with complex and uncertain decision-making situations, which considers the distance between alternatives and the Border Approximation Area (BAA) [38]. Jana developed a MCGDM technique based on a traditional MABAC model with bipolar fuzzy numbers by introducing two aggregation operators [39].

In the MABAC method, for each criterion, the Border Approximation Area (BAA) is determined by calculating the geometric mean of the corresponding elements in the weighted decision matrix. Alternatives whose criterion values fall within the upper approximation area are considered close to the ideal solution for that specific criterion, whereas those located in the lower approximation area are regarded as closer to the anti-ideal solution, as illustrated in Fig 2. After calculating the distances from the BAA across all criteria, the arithmetic mean of these distances is computed for each alternative to derive its overall performance score. Finally, alternatives are ranked based on these aggregated values, with higher arithmetic means indicating superior performance.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. The upper and the lower approximation area.

https://doi.org/10.1371/journal.pone.0349114.g002

By integrating Z-number theory into the MABAC method, this approach further strengthens the robustness of the evaluation process under conditions of uncertainty. The Z-number-based MABAC not only considers experts’ preference information but also explicitly incorporates the reliability of these judgments. This enhancement enables more transparent, interpretable, and reliable decision outcomes compared with conventional MABAC or other traditional evaluation methods [24].

3.1 Pass check phase

The evaluation process begins with the Pass Check phase, during which instructors perform an initial qualitative screening of each student’s overall engagement and integrity throughout the course. This phase serves as a gatekeeping mechanism to ensure that only those who meet the fundamental requirements of participation are eligible for further assessment. The evaluation in this stage focuses on key behavioral indicators, including class attendance, task engagement, ethical conduct, and active involvement in team-based project activities.

Students who exhibit severe deficiencies are assigned a Fail status. These deficiencies may include chronic absenteeism, failure to complete assigned project tasks, or breaches of academic integrity. Examples of misconduct include plagiarism and other violations of ethical standards. These students are directly categorized into Grade E, which reflects a critically low level of engagement incompatible with the pedagogical goals of PBL courses. Notably, the assignment of an E grade is reserved for exceptional circumstances and is exceedingly rare in practice.

To uphold the fairness and credibility of the evaluation process, students who receive a Fail status are excluded from the subsequent performance ranking. This exclusion applies not only to their own assessments but also to the peer evaluations they provided. There are two main reasons for this. First, students who fail to meet basic participation standards likely have an incomplete or biased understanding of team dynamics and peer contributions. Second, including evaluations involving these students could introduce noise and distortion into the decision matrix, thereby compromising the consistency and reliability of the group decision-making results. Their removal ensures the integrity and robustness of the evaluation process.

Given the subjectivity and complexity inherent in evaluating performance within PBL-oriented curricula, students who pass this initial screening proceed to the Score Determination phase, where their performance is systematically assessed using the proposed Z-MABAC method.

3.2 Score determination phase

Following the preliminary filtering conducted in the Pass check phase, the evaluation process advances to the score determination phase. Given the diverse and PBL courses, a comprehensive evaluation must incorporate multiple perspectives and criteria. To this end, the evaluation framework includes 15 criteria spanning both continuous and final assessment components, each reflecting a specific dimension of student performance. These criteria are assessed by a heterogeneous group of evaluators, including the Instructor (main course teacher), an Instructor Group, an Industry Expert, and Peer groups composed of student teams.

To support a more structured, consistent, and reliability-aware evaluation of student performance in the studied PBL setting, the score determination phase adopts a structured two-stage decision-making framework based on Z-number theory and fuzzy MCGDM. The entire process begins with the transformation of linguistic evaluations into TFNs, followed by a systematic decomposition into two major stages. In stage 1, criteria weights are determined using instructor group evaluations. In stage 2, the Z-MABAC method is implemented in alternative ranking. This two-stage architecture is also illustrated in Fig 5. and described in detail below.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Workflow of stage 1 and stage 2.

https://doi.org/10.1371/journal.pone.0349114.g005

3.2.1 Preparation: Z-number convert into TFNs.

Subjective assessments often involve uncertainty and inconsistent reliability. This issue is particularly evident in peer evaluations. To address this challenge, the present study adopts the Z-number formalism. In this formalism, each evaluation is expressed as a Z-number . denote the restriction of the variable, using a membership function assigns a value in this interval to each element . Then we have . denote the reliability measure, similarly we have the membership function .

Before proceeding with the evaluation stages, all qualitative assessments expressed as Z-numbers must be converted into a computationally tractable format. This applies to two sources of evaluation data. Instructor group’s Z-number assessments on criteria importance (used in Stage 1) and evaluators’ Z-number assessments on student performance across criteria (used in Stage 2).

As is shown in Table 3, a linguistic-to-fuzzy mapping table was established for both importance and reliability, ensuring consistent interpretation across expert groups. For instance, if an evaluator considers a criterion to be Very Important and expresses High Certainty about this judgment, it is represented as a Z-number with the restriction part and the reliability part , corresponding respectively to the linguistic terms Very Important (FA) and Very Certain (VC).

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Z-number linguistic scale for criteria evaluation (important & reliable).

https://doi.org/10.1371/journal.pone.0349114.t003

The weight of each expert in the aggregation process is determined based on a two-level structure:

• For each criterion, the relative weights of contributing evaluator groups are defined in accordance with the course’s instructional design.
• Within each evaluator group, all individual members are assigned equal weight, reflecting a uniform contribution assumption.

For example, consider Criterion C1, which is jointly evaluated by peer students and the course instructor, each contributing 50% to the final score. In this context, peer evaluations are conducted within student teams, and each student is assessed only by their teammates, excluding self-assessment. Suppose a team consists of five members. For any given student in the team, the remaining four peers each contribute equally to the peer evaluation component. Therefore, each peer is assigned a weight of 0.125 (i.e., 0.5 ÷ 4), and the instructor holds a weight of 0.5 in the aggregation of this criterion.

After standardizing all evaluations into Z-number representations, the assessment proceeds using the Z-MABAC method. This method facilitates aggregation, normalization, and comparison of Z-valued assessments across criteria and evaluators, while preserving both performance ratings and their associated reliability.

To make the Z-number applicable for computational processing, each Z-number can be transformed into a single representative TFN. This is achieved by computing the expected value of the fuzzy set modified by the reliability function, as proposed by Zadeh. The and mentioned before can be converted to triangular membership functions, and defuzzified value α can be calculated by the following equation.

(1)

Where is the membership function of the fuzzy restriction component, modulated by the reliability component. This transformation enables the integration of both subjective evaluations and their associated confidence levels into a unified TFN format suitable for further processing in MABAC.

Then we have

(2)

Z-number then can be converted into TFNs, shown as

(3)

This transformation adjusts the evaluated importance or performance by accounting for the evaluator’s confidence, thereby ensuring that assessments with higher uncertainty exert proportionally less influence on the final decision. To facilitate clarity and reproducibility, all relevant symbols and definitions used throughout the following stages are summarized in Table 4.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Notation and definitions.

https://doi.org/10.1371/journal.pone.0349114.t004

3.2.2 Stage 1: Criteria weight determination.

This stage determines the objective weights of evaluation criteria based on instructor group assessments. To capture both the dispersion of criteria information and the mutual independence among criteria, a hybrid weighting method combining Entropy and CRITIC is adopted. In this hybrid scheme, the Entropy component reflects the degree of dispersion in the evaluation information, while the CRITIC method is used to determine the weight of each criterion based on contrast intensity and inter-criteria conflict. Specifically, the weight of each criterion is determined by calculating the contrast intensity between that criterion and all other criteria, considering how much the criterion contrasts with the other criteria in the decision matrix [55,56]. The detailed computational steps are presented as follows.

Step 1: Each instructor provides a Z-number evaluation of the importance of every criterion. After converting these Z-numbers to TFNs, they are aggregated to form a consensus fuzzy value for each criterion:

(4)

where K is the number of experts in the instructor group.

Step 2: Each aggregated TFN is normalized across all criteria:

(5)

Step 3: Compute the entropy of each criterion by the following equations.

(6)(7)

Step 4: Compute the standard deviation and inter-criteria correlation to derive CRITIC weights by the following equations.

(8)(9)

Step 5: Final hybrid weights are combined by entropy and CRITIC weights:

where controls the fusion degree, close to 1 emphasizes the Entropy-based weights, while close to 0 emphasizes the CRITIC-based weights, and typically set . The pseudocode of this stage is shown in Fig 6.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Pseudocode of Stage 1.

https://doi.org/10.1371/journal.pone.0349114.g006

3.2.3 Stage 2: Students performance ranking via Z-MABAC method.

With the criteria weights determined, the Z-MABAC method is applied to rank student alternatives based on expert and peer evaluations. Each expert provides a Z-number . The Z-numbers are converted into TFNs according to Eqs. (1) to (3) at the preparation stage.

Step 1: Aggregate TFNs Across Experts. TFNs for each student i and criterion j using a weighted average:

(10)

This yields a unified fuzzy decision matrix .

Step 2: Normalize each element using fuzzy min–max normalization. For benefit criteria:

(11)

In this study, all 15 criteria are formulated as benefit-type, meaning that higher values indicate better performance. Therefore, normalization follows the benefit-type formula above for all criteria. Once for cost criteria, the numerator becomes .

Step 3: Multiply normalized TFNs with attribute weights , derived from the Entropy-CRITIC hybrid method:

(12)

Step 4: Compute the BAA for each criterion:

(13)

Step 5: The final MABAC score is the distance from the BAA:

(14)

Which can be calculated using the vertex method:

(15)

Here, and represent the TFNs of and respectively.

Step 6: Final Score and Ranking. Aggregate the distances across all criteria for each alternative:

(16)

Rank all students based on the descending order of . A higher score indicates stronger performance relative to the group under the weighted, uncertainty-aware multi-criteria framework. The pseudocode of this stage is shown in Fig 7.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Pseudocode of stage 2.

https://doi.org/10.1371/journal.pone.0349114.g007

3.3 The grading phase

With the exclusion of students who failed to meet the basic participation or ethical standards during the Pass Check phase, the remaining student cohort proceeds to the final Grading process. These students have demonstrated adequate engagement and are deemed eligible for ranking and grade assignments. Based on institutional records and empirical observations from prior course implementations, the distribution of grades among this qualified population generally adheres to a predefined ratio.

Let denote the proportion vector for Grades A through D, where and . In this study, the adopted distribution is . Let M be the total number of students who passed the initial screening and are eligible for ranking. Students are sorted in descending order based on their final composite performance scores obtained through the Z-MABAC method. The grade assignment is conducted based on percentile thresholds derived from the distribution vector π, with ranking intervals determined as shown in Table 5. For example, assuming M = 24, the resulting grade intervals are as follows:

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Grade assignment intervals based on percentile thresholds.

https://doi.org/10.1371/journal.pone.0349114.t005

• Grade A: Students ranked 1st to 5th
• Grade B: Students ranked 6th to 16th
• Grade C: Students ranked 17th to 23rd
• Grade D: Students ranked 24^th

In order to emphasize the technical advantages of the Z-MABAC method compared to simpler evaluation approaches, Table 6 provides a comparative analysis across several key dimensions.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Comparison of traditional and proposed assessment methods.

https://doi.org/10.1371/journal.pone.0349114.t006

This grading mechanism ensures a consistent and fair mapping between performance rankings and grade levels, grounded in institutional experience.

4.1 Background description and the pass check phase

This study was conducted in the context of the course “Design investigation and analysis”, involving 24 students divided into four peer groups of six students each. In the present study, no student was disqualified during this phase. All 24 students demonstrated sufficient participation, attendance, and ethical behavior, and therefore passed the initial screening. As a result, the performance evaluation proceeded with the full cohort, without the need to assign Grade E.

Fig 8 shows the fifteen assessment criteria, which cover key dimensions of student learning outcomes and behaviors, including teamwork, task completion, communication, attendance, problem analysis, stage report quality, individual contributions, reflection, innovation, feasibility, technical application, presentation quality, professional knowledge mastery, problem-solving skills, and communication abilities.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. The assessment criteria in the course “Design investigation and analysis”.

https://doi.org/10.1371/journal.pone.0349114.g008

The evaluation process incorporated a multi-expert structure to capture diverse perspectives: the peer groups, a core instructor team consisting of five members (T1 to T5), a primary instructor, and an industry expert. Each evaluator cluster was assigned specific roles in assessing fifteen criteria related to student performance, ensuring a comprehensive and balanced appraisal framework.

Expert weights were assigned based on the cluster roles, with peers and instructors sharing equal influence on C1, C3, C7, C8; while other criteria such as C6, C9 and C10 were evaluated solely by the instructor group or shared between the instructor team and industry expert. Criteria C6, C9, and C10 were weighed 60% by the instructor team and 40% by the industry expert, reflecting the relevance of practical industry insights. The related individual expert weights were then computed using the algorithmic approach, and the resulting values are presented in Table 7.

Download:

• PNG
  larger image
• TIFF
  original image

Table 7. Expert weight assignment.

https://doi.org/10.1371/journal.pone.0349114.t007

To address the inherent uncertainty and subjectivity in human evaluation, Z-numbers were employed to represent each expert’s judgment as a pair of fuzzy values: the restriction and reliability of the assessment. As shown in Table 4, linguistic terms for restriction ranged from “Very Inaccurate / Very Poor / Very Unimportant” to “Very Accurate / Excellent / Very Important,” modeled as TFNs, while reliability levels ranged from “Very Low / Very Uncertain” to “Almost Certain / Extremely Certain,” also expressed as TFNs.

4.2 Implementation of score determination and grading phases based on the proposed model

4.2.1 Stage 1: Criteria weight determination.

In this stage, the importance weights of the evaluation criteria were determined following the procedure outlined in Section 3.2. Specifically, the Z-number-based approach was employed to capture both the perceived importance and the associated reliability of expert assessments. Take instructor T1 as an example, who rated the importance of each criterion using linguistic terms mapped to TFNs. Reliability levels, representing the evaluator’s confidence in each assessment, were similarly expressed through linguistic-to-fuzzy mappings. The detailed linguistic evaluations and their corresponding fuzzy representations are presented in Table 8.The complete linguistic evaluation tables for criteria importance (instructors T1–T5) are provided in the Supporting Information (S2 File).

Download:

• PNG
  larger image
• TIFF
  original image

Table 8. Linguistic evaluations of criteria importance and TFN transformations (instructor T1).

https://doi.org/10.1371/journal.pone.0349114.t008

Table 8 lists the original linguistic restriction and reliability evaluations and their corresponding TFNs. The Z-number for each criterion was converted into a unified TFN according to the Eqs. (1) to (3). The resulting converted TFNs for importance assessments are shown in the “Converted TFN” column.

Following the Z-number conversion, the Entropy-CRITIC hybrid method was adopted to compute the final attribute weights. By integrating these two complementary perspectives, the final weight vector achieves greater robustness and objectivity. The resulting weights from both methods, as well as the aggregated combined weights, are summarized in Table 9 (λ = 0.5).

Download:

• PNG
  larger image
• TIFF
  original image

Table 9. Final attribute weights derived from the Entropy-CRITIC hybrid method.

https://doi.org/10.1371/journal.pone.0349114.t009

These combined weights were subsequently utilized in the Z-MABAC-based ranking stage to ensure the consistency and fairness of MCGDM under uncertainty.

4.2.2 Stage 2: Students performance ranking via Z-MABAC method.

In the 2nd stage, the Z-MABAC method was employed to calculate the final performance scores and rankings of the 24 students based on the multi-source Z-number evaluations collected in the previous phase. The raw evaluation dataset comprised a total of 1,512 entries (Table 10), covering the assessments of all students across fifteen criteria (C1–C15) by various evaluators including peers, instructors, and an industry expert. Each evaluation record included the Z-number linguistic assessments for both restriction (importance) and reliability (confidence), expressed as triangular fuzzy numbers (TFNs), alongside the corresponding expert weight. The full expert evaluation dataset based on Z-number used in this study is available in the Supporting Information (S1 File).

Download:

• PNG
  larger image
• TIFF
  original image

Table 10. Expert evaluation data based on Z-number (Partial Data).

https://doi.org/10.1371/journal.pone.0349114.t010

Following the transformation of the Z-number assessments into TFNs and the application of expert weights, a weighted normalized decision matrix was constructed, as shown in Table 11. The matrix reflects the aggregated and normalized performance of each student across the 15 criteria, incorporating both restriction and reliability through the Z-number framework. For brevity, only a subset of students’ normalized scores is presented here.

Download:

• PNG
  larger image
• TIFF
  original image

Table 11. Weighted normalized matrix of Z-number assessments (partial data).

https://doi.org/10.1371/journal.pone.0349114.t011

After constructing the weighted normalized matrix, the Z-MABAC method was applied to compute the final closeness coefficients (MABAC scores) for each student. These scores reflect the integrated performance outcomes across all criteria, considering both the reliability and importance of the evaluations provided by multiple stakeholders. Table 12 presents the final results, where students are ranked according to their MABAC scores in descending order. Scores were rounded to three decimal places for reporting clarity.

Download:

• PNG
  larger image
• TIFF
  original image

Table 12. Final Z-MABAC scores and student ranking.

https://doi.org/10.1371/journal.pone.0349114.t012

Following the ranking results obtained through the Z-MABAC method, this stage applies the grading criteria defined in Section 3.3 to categorize students into four performance bands. According to the grading policy, students falling into the top 20% are assigned Grade A, the next 45% are assigned Grade B, followed by 30% assigned Grade C, and the bottom 5% assigned Grade D. Given that the class size is M = 24, the detailed categorization based on MABAC scores is presented in Table 13.

Download:

• PNG
  larger image
• TIFF
  original image

Table 13. Grading results based on Z-MABAC ranking.

https://doi.org/10.1371/journal.pone.0349114.t013

To further summarize the dispersion of the Z-MABAC results after grading, Fig 9 visualizes the distribution of Z-MABAC scores across grade categories. The boxplot provides a compact view of the median and variability within each grade group, based on the assignments reported in Table 13. As shown in Fig 9, Grade A corresponds to consistently positive and higher Z-MABAC scores, Grade B remains mostly positive but closer to zero with moderate spread, while Grade C shifts into the negative range with a wider dispersion. Grade D contains a single student with the lowest score, which appears as a single point.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 9. Boxplot of Z-MABAC scores across grade categories.

https://doi.org/10.1371/journal.pone.0349114.g009

5.1 Sensitivity analysis

To evaluate the robustness of the proposed MABAC method under variations of key parameters, a comprehensive sensitivity analysis was conducted focusing on two dimensions: the variation of the confidence parameter λ in the Z-number framework, and incremental changes in individual attribute weights.

5.1.1 Impact of criteria weight perturbations.

The effect of amplifying each criterion’s weight individually was analyzed by recalculating the MABAC scores and ranks for the student alternatives. Spearman rank correlation coefficients between the original ranking and each perturbed ranking were computed to quantify stability. As shown in Fig 10, all Spearman coefficients remain above 0.98, indicating that the ranking order is highly stable against single-attribute weight increases.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 10. Spearman rank correlation coefficients under incremental increases of individual criterion weight.

https://doi.org/10.1371/journal.pone.0349114.g010

Correspondingly, the score variation under attribute weight increments is visualized in Fig 11, where individual student scores exhibit smooth and minor fluctuations, further confirming the robustness of MABAC with respect to moderate changes in attribute importance.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 11. Line plot of MABAC score variations for students with individual criterion weight increment.

https://doi.org/10.1371/journal.pone.0349114.g011

5.1.2. Effect of λ Variation in the Z-number Model.

The confidence parameter λ, which balances the influence of the membership and reliability functions in the Z-number, varied systematically from 0.1 to 0.9 to assess its impact on student rankings and scores. Spearman rank correlations between rankings at different λ values and the original ranking are illustrated in Fig 12. These correlations are all above 0.99, evidencing very high rank stability across the entire λ range.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 12. Spearman rank correlation coefficients under varyingλ values in the Z-number model.

https://doi.org/10.1371/journal.pone.0349114.g012

The sensitivity of individual MABAC scores to λ changes is depicted in Fig 13. While scores vary progressively with λ, no abrupt changes or rank reversals occur, demonstrating that the proposed model maintains consistent evaluations even when the confidence weighting in Z-numbers is altered.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 13. Line plot of MABAC score variations for students with differentλ values.

https://doi.org/10.1371/journal.pone.0349114.g013

The sensitivity analyses collectively indicate that the MABAC method integrated with Z-numbers exhibits strong robustness to variations in both criterion weights and the λ parameter. The consistently Spearman rank correlation coefficients mostly exceeding 0.98, reflecting that the relative ordering of alternatives is largely unaffected by these perturbations. This robustness is crucial for ensuring reliable decision-making in uncertain environments where subjective weights and confidence levels may vary.

5.2 Comparison analysis

To evaluate the consistency and discriminative capabilities of different decision-making methods, we compare eight approaches based on both Z-number and traditional TFN representations. The methods include Z-MABAC, Z-TOPSIS, Z-CODAS, Z-MARCOS, and their corresponding TFN-based versions. The Spearman rank correlation matrix is presented in Fig 14, while the ranking comparison of students across all methods is illustrated in Fig 15.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 14. The Spearman correlation across 8 methods.

https://doi.org/10.1371/journal.pone.0349114.g014

Download:

• PNG
  larger image
• TIFF
  original image

Fig 15. The ranking comparison of students across 8 methods.

https://doi.org/10.1371/journal.pone.0349114.g015

5.3 Inter-rater reliability analysis

To complement the robustness and comparison analyses presented above, an empirical inter-rater reliability (IRR) analysis was conducted based on the evaluator-level data reported in S1 and S2 Files. Specifically, S2 File provides the linguistic evaluations of criteria importance made by instructors T1–T5, together with the corresponding TFN transformations, and was therefore used to assess agreement in the criteria-weight determination stage. In contrast, S1 File contains the evaluator-level records for the case study, including criterion, student ID, evaluator identity, restriction and reliability linguistic ratings, the corresponding TFNs, and evaluator weights. From this file, the ratings of instructors T1–T5 on the six instructor-assessed criteria (C6, C9, C10, C11, C12, and C15) across the 24 students were extracted to evaluate agreement in the teacher-evaluation stage.

Because the original judgments were expressed as linguistic terms and transformed into triangular fuzzy numbers, the centroid values of the restriction TFNs were used as numeric scores for IRR estimation. A two-way random-effects intraclass correlation coefficient (ICC) with absolute agreement was adopted, and both the single-measure ICC [ICC(2,1)] and the average-measure ICC [ICC(2,k)] were reported.

First, agreement among the five instructors (T1–T5) in the criteria-weight determination stage was examined based on the 15 criteria recorded in S2 File. As summarized in Table 14, the results yielded ICC(2,1) = 0.351 and ICC(2,5) = 0.730. This indicates that, although individual-level agreement among instructors was moderate, the aggregated evaluations of the five-member instructor panel achieved acceptable reliability for group-based criteria weighting.

Download:

• PNG
  larger image
• TIFF
  original image

Table 14. Inter-rater reliability results for fixed instructor panels.

https://doi.org/10.1371/journal.pone.0349114.t014

Second, agreement among the same instructor panel was examined in the teacher-evaluation stage using the records in S1 File for the six instructor-assessed criteria (C6, C9, C10, C11, C12, and C15) across the 24 students. The pooled analysis over all student–criterion combinations produced ICC(2,1) = 0.726 and ICC(2,5) = 0.930, indicating strong agreement among instructors, particularly when their evaluations were aggregated. The overall IRR results for Stage 1 and Stage 2 are presented in Table 14.

A criterion-wise analysis further confirmed this pattern. As shown in Table 15, the average-measure ICC values were 0.903 for C6, 0.886 for C9, 0.897 for C10, 0.879 for C11, 0.959 for C12, and 0.973 for C15. These findings provide an empirical complement to the Z-number-based modeling of evaluator reliability. In other words, besides representing uncertainty and confidence through Z-numbers, the observed agreement among the fixed instructor panels also supports the internal consistency of the proposed framework within the present case study. However, these findings should still be interpreted in light of the single-course and single-institution context.

Download:

• PNG
  larger image
• TIFF
  original image

Table 15. Criterion-wise inter-rater reliability for instructor-assessed criteria.

https://doi.org/10.1371/journal.pone.0349114.t015

5.4 Limitations

The empirical validation in this study is based on a single cohort of 24 students from one practical course at one institution. Therefore, the present findings should be interpreted as case-based evidence of feasibility and internal robustness within the studied setting, rather than as definitive evidence of general applicability. In particular, the evaluator composition, course design, and grading policy adopted in this case may differ from those in other disciplines, institutions, or assessment settings. Accordingly, claims regarding fairness, transparency, and practical applicability should be understood as context-dependent and limited to the current course environment. Future research should examine the proposed framework across multiple cohorts, courses, institutions, and evaluator configurations to establish broader external validity.

In addition, although the present study now includes a formal empirical inter-rater reliability analysis for the fixed instructor panels involved in the criteria-weight determination stage and the teacher-evaluation stage, this empirical check remains limited to comparable evaluator sets with complete repeated-rating structures. Other components of the framework, such as peer evaluations and single-expert judgments, were not examined through the same ICC-based design. Therefore, the reported IRR results should be understood as a partial empirical complement to the Z-number-based reliability modeling, rather than a full agreement assessment across all evaluator types. Future research should extend empirical agreement analysis to more diverse evaluator configurations and educational settings to further strengthen the empirical basis of the framework.

A further limitation concerns the elicitation of confidence information in the Z-number assessments. In the current implementation, some reliability inputs depend on evaluators’ self-reported confidence expressed through linguistic terms. Although this design is consistent with the conceptual structure of Z-numbers, such confidence judgments may still be affected by individual response tendencies, subjective bias, or differences in evaluators’ interpretation of linguistic labels. In addition, the final grade allocation in this case study follows an institution-specific proportion-based grading policy, which may not be directly transferable to all educational contexts. Future studies may improve the framework by combining self-reported reliability with empirical calibration strategies, behavioral indicators, or alternative grading policies that are better aligned with different institutional requirements.