Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A novel MCDM approach for design concept evaluation based on interval-valued picture fuzzy sets

  • Qing Ma,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Shandong Jiaotong University, Jinan, China

  • Hongyuan Sun ,

    Roles Investigation, Supervision, Writing – review & editing

    hongyuan_000@163.com

    Affiliation Shandong Jiaotong University, Jinan, China

  • Zhe Chen,

    Roles Supervision

    Affiliation Shandong Jiaotong University, Jinan, China

  • Yuhang Tan

    Roles Supervision

    Affiliation Shandong Jiaotong University, Jinan, China

Abstract

The assessment of design concepts presents an efficient and effective strategy for businesses to strengthen their competitive edge and introduce market-worthy products. The widely accepted viewpoint acknowledges this as a intricate multi-criteria decision-making (MCDM) approach, involving a multitude of evaluative criteria and a significant amount of data that is frequently ambiguously defined and subjectively influenced. In order to tackle the problems of uncertainty and fuzziness in design concept evaluation, our research creatively combines interval-valued picture fuzzy set (IVPFS) with an MCDM process of design concept evaluation. Firstly, this study draws on the existing relevant literature and the experience of decision makers to identify some important criteria and corresponding sub-criteria and form a scientific evaluation indicator system. We then introduce the essential operational concepts of interval-valued picture fuzzy numbers (IVPFNs) and the interval-valued picture fuzzy ordered weighted interactive averaging (IVPFOWIA) operator. Thirdly, an entropy weighting method based on IVPFS is proposed in this research to calculate the weights of criteria and sub-criteria, and based on this, an integrated IVPF decision matrix is further constructed based on the presented IVPFOWIA operator. Finally, the best design concept alternative is selected by applying the extended TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) approach with IVPFS. The IVPFS combined with improved MCDM method have been proven to be superior in complex and uncertain decision-making situations through experiments and comparative assessments. The information ambiguity in the evaluation of design concept is well characterized by our augmentation based on IVPFS.

1. Introduction

Design concept is the initial stage of a product lifecycle. Although this process is normally very short in the long lifecycle of a product, about 80% of sustainability and other impacts are decided in this stage. Because decisions and requirements are made in this stage, design concept evaluation is one of the most critical steps in the product design stage. Once the design concept is selected, most impacts such as novelty, feasibility and sustainability, as well as maintenance and disposal methods, are determined [1]. Hence, design concept evaluation is a complicated process, the complexity of product concept design evaluation aries from considering multiple factors, dealing with uncertainty, making decisions based on variable criteria, and collaborating with diverse teams, often assigned by decision support tools [2]. Problems like design concept evaluation are commonly defined as an MCDM problem.

As an important research field of MCDM [3], effective related approaches have already been developed and applied for design concept evaluation [4, 5]. Some studies are based on linguistic-based information, in which experts give their preferences based on a multiple-scale index [6]. However, as the raw data are usually imprecise, incomplete or subjective [4], methods based on fuzzy sets are integrated for design concept evaluation. Rough set is an important MCDM method in design concept evaluation to transform crisp information into rough intervals, and select the optimal alternative by the decision matrix based on rough numbers [7]. Rough-TOPSIS [8, 9], Rough-VIKOR [10] and Rough-AHP and Rough-TOPSIS [11] are then applied in design concept evaluation. Other MCDM methods such as fuzzy method [12, 13], grey theory [14] and vague set [15] based on linguistic information are also implemented in specific decision-making of design schemes. However, as the experts are not always familiar with all the attributes of a product, it is very hard for experts to give an exact preference for a multiple-scale. Membership methods provide another way of dealing with MCDM problems [16]. Intuitionistic fuzzy set (IFS), as a widely applied vote-based approach, has shown its advantage for design concept evaluation [17, 18]. Experts make their decisions using agree or disagree for each attribute and each alternative. Even though the IFS has achieved ideal results, it is not sufficient to cover all the situations of experts’ decisions with only two options of agree or disagree. When an expert feels unfamiliar with an attribute, the expert has the authority to abstain or “sit on the fence”. Once these circumstances are considered, the decision information environment becomes more vague.

In order to address the issues of fuzziness and uncertainty in design concept evaluation, this study combines IVPFNs with improved TOPSIS to present a systematic framework for decision-making. The purpose of this research is as follows: (1) to present IVPFNs to handle ambiguity and subjectivity in decision-making, and enhance the objectivity of evaluation outcomes. (2) to put forward improved TOPSIS method based IVPFNs to strengthen the MCDM method and to underline the superiority of IVPFNs.The main contributions of our manuscript include:

  1. We have successfully extracted and described the fuzziness and uncertainty in the design concept evaluation process based on the IVPFS theory. More precisely, this work offers a practical technique for transforming qualitative assessment data input into IVPFNs for design idea evaluation, enabling adaptable decision-making processes.
  2. Our research develops and applies a new aggregation operator to integrate IVPFS, taking advantage of OWIA to eliminate the correlation of indicators. As is well known, no literature has examined the use of the OWIA operator and IVPFS together while evaluating design concepts.
  3. We have integrated improved TOPSIS method with IVPFS theory to evaluate some design concept schemes to determine the best one. This algorithm addresses the limitation of traditional TOPSIS that cannot reflect the dynamic changes in the evaluation index sequence, ensures the accuracy of decision results.

This is how the study’s reminder is put together: Section 2 reviews the literature. Section 3 sets out several key IVPFS concepts, clarifies some fundamental and typical IVPFN operating laws, and creates the IVPFOWIA operator. A unique framework for evaluating and choosing design concept alternatives is proposed in Section 4 based on the improved TOPSIS method with IVPFS. Section 5 provides a case study to demonstrate the scientific validity as well as the feasibility of the algorithms proposed in this study. Section 6 outlines a summary of the full manuscript and predictions for potential future applications.

2. Literature review

Based on the IVPFS, IVPFOWIA and the extended TOPSIS method, the purpose of our research is to evaluate design concept schemes. Therefore, the literature review has three sections: (1) research on IVPFS, (2) research on IVPFOWIA, and (3) research on ranking the design concept alternatives through an extended TOPSIS method under IVPFS.

2.1. Interval-valued picture fuzzy set

With the advent of the information age and the rapid development of computer technology, huge and complex fuzzy datasets have appeared in many fields of daily life [19]. To address practical uncertainties, numerous scholars have formulated classical theories that align with real-world requirements. For example, since the emergence of fuzzy sets (FS), it has quickly become a hot spot in the study of uncertainty theory, and has generated research and discussion by scholars [20]. To facilitate specific applications, various extended forms of fuzzy sets have been put forth. For example, Yager first proposed theory of Pythagorean fuzzy set to describe the uncertainty issues of experts in MCDM. Subsequently, numerous scholars conducted research on the Pythagorean fuzzy set theory. A responsible enterprise not only focuses on its own development, but also on the protection of the ecological environment. Green supplier selection contributes positively to environmental protection. Therefore, Mostafa et al.[21] conducts research on Python fuzzy sets, they apply Pythagorean fuzzy set and TOPSIS method to determined the best supplier. Morsy [22] proposed a method that incorporates Pythagorean Fuzzy Number into risk return rate, portfolio risk amount, and expected return rate. Finally, an example was used to verify the effectiveness of the proposed method. In 1986, Atanassov [23] first proposed the theory of IFS, which mainly solves the indecision phenomenon in which experts may not know enough about the decision object or other factors in the decision-making process. IFS has three parts: a membership degree, a non-membership degree and an uncertainty degree. Because of its neutrality, IFS is more widely used in daily decision-making than fuzzy set theory. As the decision-making object becomes more complex, IFS theory cannot be solved. Taking expert voting as an example, there may be four possible outcomes: support, neutrality, opposition, and abstention. Therefore, in 2013 Cuong et al.[24] proposed an extended and new fuzzy set-PFS to deal with the above situation.

However, in MCDM problems, due to the limitations of experts’ understanding of decision-making objects and the ambiguity of the decision-making environment, experts can give an interval number rather than a specific real number when making a decision. Therefore, in 2013, Cuong [24] proposed IVPFS theory, so that membership, neutrality, non-membership and abstention can be represented by interval numbers to enhance the credibility of decision-making results. In certain real-world applications, IVPFS theory, an extension of PFS, are more adept at handling and modeling inconsistent, indeterminate, and incomplete data [2527]. Therefore, it becomes very important to study the group decision problem under the IVPFS.

2.2 Interval-valued picture fuzzy ordered weighted interactive averaging operator (IVPFOWIA)

The aggregation operator is a particularly efficient technique for solving MCDM problems. To clarify concerns with decision-making, many scholars have studied aggregation operators in recent years. For example, Wei [28] put forward some aggregation operators under picture fuzzy set, then used these operators to deal with MCDM problems. In final, a typical example was used to demonstrate the effectiveness of the proposed operators. Due to the advantages of Archimedes operator and Dombey operator in providing appropriate generalization ability and flexibility in aggregating information, respectively. Saha et al. [29] concentrate on the use of these operators in the classifier’s aggregate similarity calculation. Finally, by comparing with previous operators, it is verified that the proposed operator has higher accuracy. Senapati et al. [30] introduced the Aczel-Alsina (AA) weighted geometric operator and the AA ordered weighted geometric operator for IFS, the effectiveness of the proposed operator was verified through two typical cases. Liu et al.[31] first introduced the Archimedean Heronian (AH) operator and the weighted Archimedean Heronian (WAH) operator for IFS. Then, they put in multiple attribute group decision making approach based on these operators. Similarly, verify the effectiveness of the proposed operator through its application in typical cases. Due to the stability and flexibility advantages of the Frank aggregation operator, Zeng et al. [32] proposed a Frank operator for the q-rung orthopair fuzzy set. Besides, they discuss the some properties,such as idempotency, commutativity, monotonicity and boundedness. Batool et al. [33] proposed a novel decision-making approach based on aggregation operators under Pythagorean probabilistic hesitant fuzzy number. In addition, they also conducted a detailed discussion on some properties of the proposed operator. Regarding the issue of defects in some operators, Luo and Long [34] proposed a weighted geometric operator and an ordered weighted geometric operator for a picture fuzzy set (PFS). Finally, they use the aggregation operators to deal with MCDM problems. Garg [35] put forward various fuzzy picture aggregation operations, and combined them with classic methods to solve practical problems. Nevertheless, the aforementioned aggregate operator merely takes the attributes’ independence into account; in reality, many attributes will be connected in varying degrees like complementary relationships, redundancy, preference relationships, etc. In our study, we applied the OWIA operator to IVPFS, taking advantage of OWIA to eliminate the correlation of indicators in the process of MCDM.

2.3. Improved TOPSIS method

MCDM is a decision-making approach used to evaluate and select the best option from a set of alternative solutions when multiple criteria or factors need to be considered [36]. MCDM methods help decision-makers rank and prioritize options based on these criteria, making it easier to make informed and objective decisions [3740]. This approach is widely applied in various fields such as engineering, management, environmental planning, and medical decision-making. Through MCDM, decision-makers can better balance various factors to make wiser decisions.

MCDM methods include the weighted summation method, weighted product method, TOPSIS method [41], the analytical hierarchy process (AHP) [42], the grey relation projection method (GRP) method [43], utility theory, graphical tools, quality function deployment (QFD) and fuzzy logic method [44]. In 1981, the TOPSIS technique was initially introduced as a prevalent method for MCDM. TOPSIS method is used to select the best choice among a set of schemes. The basic idea of the TOPSIS method is to assume positive and negative ideal solutions, calculate the distance between each sample and the positive and negative ideal solutions, obtain their relative closeness to the ideal solution (i.e., the closer they are to the positive ideal solution, the farther they are to the negative ideal solution), sort each solution, and ultimately determine the optimal solution. TOPSIS is widely applied in engineering, management, and decision sciences to effectively address multi-criteria decision problems. However, TOPSIS method only considers the Euclidean distance between indicators, and cannot reflect the dynamic changes of evaluation index series. There are shortcomings in decision analysis, especially for dynamic series. Grey correlation analysis can reflect the internal changes of each scheme, making up for the shortcomings of the TOPSIS method. In our research, the TOPSIS model is combined with the grey correlation method to improve TOPSIS, defining a new set of positive and negative ideal points and a formula for relative closeness. The improved TOPSIS model solves the problem of large fluctuations in data by calculating the degree of correlation between evaluation objects. When calculating Euclidean distance, it avoids the phenomenon of disordered equidistant sorting between sample points and positive and negative ideal points, and has strong stability and objectivity.

3. Basic preliminaries

In this section, we go through a number of fundamental concepts to give the necessary background information.

3.1. Picture fuzzy set

As an important extension of IFS, PFS has received increasing academic attention in recent years. Compared to FS [20] and IFS, PFS is more flexible and practical for expressing and processing fuzzy information. PFS takes into account the support, uncertainty, opposition and abstention of decision makers when evaluating alternatives in real-life scenarios, and expresses the ambiguity and uncertainty of decision makers about the decision problem they face in terms of four aspects: positive membership degree, neutral membership degree, negative membership degree and refusal membership degree [24].

The following are significant definitions and related properties of PFS.

Definition 1 [24] Let X be a non-empty set, then it is said that: (1)

Where the terms μA: A, ηA: A, vA: A∈[0,1] denote the degrees of positive, neutral, and negative membership of an element x in X, respectively. Additionally, μA, ηA and vA meet the following requirement: (2)

And is said to be refusal membership degree of an element x in X. If ξA(x) = 0, in this case A returns to the IFS.

Definition 2 [24] Suppose that θ1 = (μ1, η1, v1) and θ2 = (μ2, η2, v2) are two different PFNs, then

  1. ;
  2. ;
  3. ;
  4. .

3.2. Interval-valued picture fuzzy set

Cuong et al. (2013) proposed IVPFS theory based on PFS [24]. This theory allows for a more flexible representation of uncertainty and ambiguity in data, making it highly suitable for decision-making in complex systems [45, 46]. To formalize this concept, let X be a given domain of discourse. Let G[0,1] be the subintervals set of interval [0,1] and x≠0 be the given set. Thus, an IVPFS can be described as follows: (3)

Where . The intervals μA(x), ηA(x), νA(x) represent positive, negative and neutral membership degrees of A, and represent the lower and upper end points. Then, an interval-valued picture fuzzy set A can be written as: (4)

Where and . Refusal membership degree expressed by πA can be determined using the formula below (5).

(5)

Definition 3 For two IVPFNs A = (μA(x), ηA(x), νA(x)) and B = (μB(x), ηB(x), νB(x)). α as a scalar value α>0. The following shows the basic and significant operations of IVPFS:

The operations based on IVPFS can give the following properties:

Suppose that M, N & H are the three IVPFNs, where M = (μM(x), ηM(x), νM(x)) and N = (μN(x), ηN(x), νN(x)) and H = (μH(x), ηH(x), νH(x)) and α>0, therefore:

Definition 4 Let that be the IVPFN, Ω is the set of whole IVPFNs. ω = (ω1, ω2,⋯,ωn)T as the weight vector of them, a mapping IVPFOWIA: ΩnΩ of dimension n is an interval-valued picture fuzzy ordered weighted interactive averaging (IVPFOWIA) operator, with . Then, (6)

Let be a collection of IVPFNs, then by Eq (6) and the weighted interaction operational laws in Definition 4, the aggregated result by using the IVPFOWIA operator is also an IVPFN, and then: (7)

Where ω = (ω1, ω2,⋯,ωn)T is the weight of Ai(i = 1,2,⋯,n), such that .

Appendix A provides the proof for this theorem.

Definition 5 Let be an IVPFN, then the score function S(Ai) and the accuracy function H(Ni) of the IVPFNs can be described as: (8)

In the light of the equation of the score function S(Ai) and the accuracy function H(Ai) given in Definition 5, the size of the two IVPFNs can be compared by the following equation:

  1. Supposing that S(A1)<S(A2), so that A1<A2
  2. Supposing that S(A1) = S(A2), so that:
    1. Supposing that H(A1)<H(A2) so that A1<A2
    2. Supposing that H(A1) = <H(A2) so that A1 = A2

The rule can be drawn from the above equation: the larger the score function S(Ai) is, the larger the IVPFN is.

Definition 6 Let be two IVPFNs, so the Hamming distance of A1 and A2 is as follows: (9)

The Euclidean distance of A1 and A2 is as follows: (10)

3.3. The entropy of interval-valued picture fuzzy set

3.3.1. Shannon’s entropy.

As a prevalent technique for weight determination, Shannon’s entropy objectively assigns weights to indicators based on the amount of information they provide and the correlation between them [2, 47, 48], and it has been used in areas such as product design, environmental protection and medicine and health [48]. In order to mitigate the influence of human-induced discrepancies, Shannon’s entropy is employed to compute weights by quantifying the extent of divergence among individual index values. According to Wang and Lee [49], The specific calculation of Shannon’s entropy is as follows:

Suppose that a team of decision makers has been given the task of weighing m alternatives against n design criteria. Let D = [yij] be the decision matrix, where 1<i<m and 1<j<n.

Step 1: Standardize the decision matrix D, and eliminate differences in scale between criteria.

(11)

Step 2: Compute the information entropy matrix of criterion E = (ej) by Eq (12).Where ej denotes jth criterion’s entropy value.

(12)

In Eq (12): k is a constant, we can use formula to calculate it.

Step 3: Determine the weight () of the criteria element, which is calculated as follows: (13)

In Eq (13), .

3.3.2. The entropy of IVPFS.

Due to the properties of the information environment, Shannon entropy has various extensions. For example, Burillo and Bustince [50] introduced entropy of IFS, and Chen [2] employed entropy of rough set (RS) in design concept evaluation. Joshi and Kumar [51] defined the entropy of interval-valued intuitionistic hesitant fuzzy set (IVIHFS), by using the entropy of IVIHFS to calculate the weights of criteria. Ye [52] introduced a fuzzy cross entropy of the interval-valued intuitionistic fuzzy sets (IVIFS) to determine the weights of schemes. Qiyas et al. [53] proposed the entropy of generalized interval-valued picture fuzzy linguistic set (GIVPFLS), which is defined in detail in Definition 7. Based on the literature review on entropy weight mentioned above, we can see that many scholars have provided entropy measures for RS, IFS, IVIHFS, IVIFS, and GIVPFLS. In this manuscript, we utilize the entropy weighting method proposed by Qiyas et al. In MCDM, IVPFS exhibits a notable advantage in addressing decision-maker hesitancy and uncertainty. A key feature of IVPFS is its incorporation of abstention intervals, which proves invaluable in handling decision-maker hesitancy [54]. When decision-makers encounter uncertainty regarding certain elements or attributes, or when they cannot definitively ascertain the membership and non-membership degrees of elements, they can employ abstention intervals to signify their hesitancy. These abstention intervals allow decision-makers to express the extent of their uncertainty within a specified interval range, without the necessity of making explicit decisions. This flexibility is particularly pertinent for complex real-world decision problems where decision-makers regularly confront uncertainty and conflicting information. The entropy of IVPFS is a useful method, especially in circumstances when decision-makers are unable to convey their preferences about criteria during the early phases of the decision-making process or are unsure of how to do so. This approach accounts for the degree of uncertainty pertaining to the alternatives offered by decision-makers and the divergence among various aspects within the IVPFS, permitting the straightforward computation of the relative relevance of the criteria.

Definition 7 [53] let be a separate element in an IVPFLS . For this, entropy is defined as.

According to the Definition 7, we derive the calculation formula for the entropy method of IVPFS.

(14)

Subsequently, the subsequent formula is employed for the computation of attribute weights.

(15)

for all j = 1,2,⋯,n.

4. Proposed methodology

In this section we provide a novel framework for choosing leisure yachts design alternatives that is based on IVPFS and the improved TOPSIS technique. The procedure phases of the IVPFS-Improved TOPSIS method are depicted in Fig 1. There are three phases: (1) prepare phase, (2) construct the collective IVPF decision matrix, and (3) sort and select the best design scheme. Phase 1 involves establishing the design concept’s assessment index system by making use of the body of existing research and the expertise of decision makers. In phase 2, the weights of criterion layer as well as indicator layer can be calculated through the entropy of IVPFS. With the help of IVPFOWIA, the collective IVPF decision matrix can be constructed. In phase 3, the TOPSIS technique is improved in the IVPFS to compute the relative closeness of design concept schemes to positive and negative ideal solutions. The final ranking of different design concept solutions can be obtained.

thumbnail
Fig 1. Flowchart of the IVPFS-MCDM for design concept evaluation.

Phase I is the preparation stage, which mainly includes expert interviews, literature collection, determination of alternative solutions, and evaluation index system. Phase II mainly includes the determination of criteria weights and the construction of a comprehensive evaluation matrix. Phase III mainly involves determining the optimal design alternative.

https://doi.org/10.1371/journal.pone.0294596.g001

4.1 Prepare phase

It is vital to determine each of the assessment criteria and sub-criteria during the design concept formulation and selection process [55]. The evaluation indicators for proposing design concepts can be determined from the perspectives of product economic benefits, technology, and environmental protection. The design concepts evaluation index system has been constructed from four aspects: aesthetic evaluation, technical evaluation, economic evaluation, and environmental evaluation. We have collected and summarized literature on design concept evaluation in recent years, and found that certain indicators have appeared multiple times in the design concept evaluation indicator system. In order to identify the most appropriate indicators for the evaluation of design concepts, a panel of decision-makers from relevant fields such as industrial design and design concepts was invited to this study. By communicating with the expert group, analyzing and determining the evaluation indicators of design concept, a design concept evaluation indicator system is constructed, as shown in Fig 2. For each standard layer and its sub-standard layers, literature sources are provided in Table 1.

thumbnail
Fig 2. Design concept evaluation index system.

The design evaluation index system is mainly divided into goal layer and criteria layer. The criteria layer includes first level criteria and second level sub-criteria layer.

https://doi.org/10.1371/journal.pone.0294596.g002

thumbnail
Table 1. Design concept evaluation index system.

https://doi.org/10.1371/journal.pone.0294596.t001

4.2 Determine comprehensive evaluation decision matrix

According to the index system in Table 1, for such decision problems as design concept evaluation, the indicators are portrayed by qualitative methods, which have uncertainty and imprecision. In this regard, there are generally two types of measurement methods: a quantitative measurement method, where each indicator is artificially assigned a deterministic value within a specified range of values; and a qualitative level measurement method, where each indicator is assigned a certain level within a specified range of levels. Therefore, the latter method is often used to avoid extreme evaluation events and reflect the fuzzy characteristics of fuzzy indicators, such as using A, B, C, and D levels. Of course, there are also extreme events, some are assigned A and some are assigned D. However, the dispersion of such ratings is obviously much smaller for different ratings given by multiple decision-makers. If we can adopt a scientific approach to normalize the statistics and apply them scientifically without losing the information contained in them, the latter must be better than the former. To this end, this research proposes a new index measurement method based on the connotation of IVPFNs and its generation mechanism.

For the MCDM problem of design concept evaluation, we assume D = {D1, D2,⋯,Dk} is the set of k DMs, C = {C1, C2,⋯,Cn} is the set of n design criteria, A = {A1, A2,…,Am} is the set of m design schemes. The weights of design criteria are presented by w = (w1, w2,⋯,wj), where . The next sections discuss the specifics of the established design concept alternative evaluation model based on these assumptions.

Step 1: Construct interval-valued picture fuzzy decision matrices belonging to the criteria.

Based on the views of all decision makers, under the lth sub-criteria with respect to criterion Cj, IVPF evaluation values of alternatives

(16)

The specific calculation steps are as follows:

First, for sub-criteria with respect to criteria Cj, each decision-maker evaluates n design concept alternatives. The evaluation opinions are provided by the questionnaire survey in Table 2 and Fig 3.

thumbnail
Fig 3. IVPFN composition graph.

This figure includes four evaluation options (High、Medium、Low and Difficult to judge). Each evaluation option has two attitudes, namely affirmative and hesitate.

https://doi.org/10.1371/journal.pone.0294596.g003

thumbnail
Table 2. Sub-criteria evaluation questionnaire.

https://doi.org/10.1371/journal.pone.0294596.t002

Second, for data statistics, for the second level evaluation indicator with respect to criteria Cj, calculate the evaluation opinions of all decision-makers on n alternatives. can be calculated by Eq (17).

(17)

Where P represents the number of all evaluation experts. denotes the number of evaluation experts who hold a high (affirmative) attitude towards the design concept alternative Ai relative to the sub-criterion denotes the number of evaluation experts who hold a high (hesitate) attitude towards the design concept alternative Ai relative to the sub-criterion denotes the number of evaluation experts who hold a medium (affirmative) attitude towards the design concept alternative Ai relative to the sub-criterion denotes the number of evaluation experts who hold a medium (hesitate) attitude towards the design concept alternative Ai relative to the sub-criterion denotes the number of evaluation experts who hold a low (affirmative) attitude towards the design concept alternative Ai relative to the sub-criterion denotes the number of evaluation experts who hold a low (hesitate) attitude towards the design concept alternative Ai relative to the sub-criterion .

By using Eq (17), building the IVPF evaluation matrix corresponding to sub-criteria belonging to the criterion Cj.

(18)

Step 2: Construct comprehensive IVPF decision matrix.

the weight of the sub-criteria can be computed using Eqs (14)–(15). Through applying Eq (19), compute the aggregated value for design concept scheme Ai relative to criterion Cj by employing IVPFOWIA.

(19)

Finally, the comprehensive IVPF decision matrix can be determined by using the IVPFOWIA operator given in Eq (20): (20)

4.3 Determine and choose the best design concept alternative(s)

In this phase, we employ the improved TOPSIS technique based on IVPFS to rank the best design concept scheme. Following are the detailed steps:

Step 1: Standardize the decision-making assessment matrix. To counteract the impact of varying criteria scales on the decision-making procedure, it is essential to render the initial matrix dimensionless.Therefore, in this step, we transform the initialized judgement matrix into a normalized decision matrix .

(21)

Where i = 1,2,⋯m, j = 1,2⋯,n.

Step 2: Utilizing the standardized judgment matrix, ascertain the IVPF-positive ideal solution (R+) and the IVPF-negative ideal solution (R): (22) (23)

Where: (24) (25)

Step 3: Compute the grey correlation degree between alternatives and positive (negative) ideal points, respectively.

  1. i. Compute the grey correlation coefficients and between alternatives and the IVPF-PIS (NIS) , respectively:
(26)

Where ρ is called the resolution coefficient. When ρ = 0, there is no surrounding environment. Likewise, when ρ = 1, nothing changes in the surrounding environment. Usually, is the distance between and which can be obtained by Eq (9).

  1. ii. Compute the grey correlation degree and between alternatives and IVPF-PIS (NIS).
(27)(28)

Step 4: Compute the Euclidean distance between alternatives and IVPF-PIS (NIS).

(29)

Normalize and separately, and merge the normalized , and combinations to obtain the closeness of and between alternatives and the IVPF-PIS (NIS) .

(30)

Step 5: Sort and select the best design concept scheme.

Compute the relative closeness Di of design concept schemes: (31)

The design concept alternatives are ranked and preferred according to their relative closeness Di, with higher values of Di indicating higher design effectiveness of the alternative. Conversely, the lower the value.

5. Empirical example

In this study, the improved TOPSIS method of IVPFS proposed in section 4 is used to select the optimal design scheme from multiple leisure yachts. In this research work, we needed to redevelop a leisure yacht for a yachting company in Weihai, China. All decision-makers are longstanding experts in the field of leisure yachts design, and have extensive design experience. The decision-makers form an evaluation and selection group. There are 30 decision-makers D = {D1, D2,…,D30}, and five design concept alternatives A = {A1, A2, A3, A4, A5}. According to the main procedures of the proposed model, the specific example steps of design concept evaluation are as follows.

5.1. Construct the collective IVPF decision matrix

According to the design concept evaluation index system in Table 1, taking leisure yachts design A1 as an example. This study invited 30 experts to participate in the evaluation. There are 7 experts who have a high (affirmative) attitude towards the leisure yachts alternative A1 relative to the sub-criterion , 5 experts hold a high (hesitate) attitude towards the leisure yachts alternative A1 relative to to the sub-criterion , 5 experts hold a medium (affirmative) attitude towards the leisure yachts alternative A1 relative to the sub-criterion , 3 experts hold a medium (hesitate) attitude towards the leisure yachts alternative A1 relative to the sub-criterion , 4 experts hold a low (affirmative) attitude towards the leisure yachts alternative A1 relative to the sub-criterion , 5 experts hold a low (hesitate) attitude towards the leisure yachts alternative A1 relative to the sub-criterion , and then, taking P = 30 in Eq (17), IVPF evaluation value can be calculated for leisure yachts alternative A1 relative to sub-criteria :

Therefore . In this study, we can also calculate the leisure yachts alternative A1 relative to the sub-criteria relative to criterion C2. The above calculation process is based on A1 as an example And then, the IVPF evaluation value for the leisure yachts alternatives A2A5 relative to the sub-criteria , which are shown in Table 3, is determined using the same calculation procedure. Finally, we can also compute the IVPF evaluation matrix relative to the sub-criteria belonging to criteria (C1, C3, C4) using the same procedure.

thumbnail
Table 3. Judge matrices belonging to the sub-criteria of Technology (C2).

https://doi.org/10.1371/journal.pone.0294596.t003

The entropy of IVPFS can be used to computed the local weights of sub-criteria, as shown by Eqs (14)–(15):

By applying the IVPFOWIA, we can obtain the collective decision matrix as shown in Table 4.

thumbnail
Table 4. The comprehensive IVPF decision matrix.

https://doi.org/10.1371/journal.pone.0294596.t004

With the help of Eqs (14)–(15), we can determine the entropy weights of IVPFS of C = {C1, C2, C3, C4} is w = (0.245,0.244,0.251,0.260)T.

5.2. Determine and choose the best design concept alternative(s)

Step 1: Due to the four criteria are all benefits (not costs), according to Eq (21), therefore the normalized evaluation decision matrix is consistent with Table 5.

thumbnail
Table 5. The standardized IVPF decision matrix.

https://doi.org/10.1371/journal.pone.0294596.t005

Step 2: The IVPF-PIS and IVPF-NIS of the standardized decision matrix are calculated through Eqs (22)–(23).

Step 3: Determine the grey correlation degree between alternatives and positive (negative) ideal points, respectively.

Determine the grey correlation coefficients and of each alternative with the positive (negative) ideal point sets and by using Eq (26), and then calculate the grey correlation degree and of each alternative with the positive (negative) ideal point sets and based on Eqs (27)–(28). The results obtained are shown in Table 6.

Step 4: Compute the Euclidean distance between alternatives and IVPF-PIS (NIS) by Eq (29). The results obtained are shown in Table 7.

Step 5: Select the best design concept scheme.

Compute the relative closeness Di of each design concept alternative through Eqs (30)–(31). The specific parameters and alternatives are listed in Table 8.

thumbnail
Table 8. The order of the five design concept alternatives.

https://doi.org/10.1371/journal.pone.0294596.t008

5.3. Sensitivity analysis

In this section, we conduct the sensitivity analysis of the resolution coefficient ρ in the proposed IVPF-Improved TOPSIS method. When ρ = 0.5, the ranking of the five design concept solutions is A3>A1>A2>A5>A4. Fig 4 shows the results of the ranking of the alternatives for different values of the resolution coefficients ρ. According to Fig 4, it can be seen that in addition to ρ = Beyond 0.1, A3 is always the best of the five design concept alternatives. Sensitivity analysis shows the stability and reliability of the model proposed in this study.

thumbnail
Fig 4. Results of sensitivity analysis.

This figure presents the sensitivity analysis results for IVPF-improved TOPSIS method at different ρ values (ρ = 0.1, 0.3, 0.5, 0.7, 0.9). When ρ = 0.1, due to ρ The value of is relatively small, which may make the model less sensitive to changes in parameters. Therefore, even slight changes in parameters may result in different sorting results. This is because the smaller ρ The value reduces the sensitivity threshold to changes in weights and evaluation values.

https://doi.org/10.1371/journal.pone.0294596.g004

5.4. Comparative analysis

In order to evaluate and verify the effectiveness of the proposed algorithm in this study, comparison studies are done in addition to the case study using interval-valued intuitionistic fuzzy (IVIF)-Extended TOPSIS method and IVPF-TOPSIS method. Fig 5 compares several models, while Table 9 and Fig 6 present the findings of a comparison of various methodologies.

thumbnail
Fig 5. Comparison of different models.

This figure illustrates the comparison between various models and their performance in the evaluation process.

https://doi.org/10.1371/journal.pone.0294596.g005

thumbnail
Fig 6. Relative closeness of different MCDM methods.

This figure depicts the relative closeness and comparative analysis of various methods used in the study.

https://doi.org/10.1371/journal.pone.0294596.g006

thumbnail
Table 9. The results of comparisons between different methods.

https://doi.org/10.1371/journal.pone.0294596.t009

From Fig 6, we can see that the proposed model and the other two models obtain the same optimal solution. The differences are reflected in the entire process of design optimization or some data processing steps. The details are as follows:

  1. IVIF improved TOPSIS method. The difference between this method and our model is that the fuzzy environment is different. The IVPFS proposed in this research considers IVIFS at the same time. In addition, it also takes into account the neutrality of expert evaluation information, which is closer to the real evaluation process.
  2. IVPF-TOPSIS method. The difference between this and our proposed method is that this research improves the TOPSIS method. The improved TOPSIS model solves the problem of large data fluctuations by calculating the degree of correlation between evaluation objects. From Table 9, it can also be seen that the difference in relative closeness between the five design alternatives obtained by the method proposed in this research is smaller, and the difference in relative closeness values between the five design alternatives obtained using IVPF-TOPSIS is significant. Thus, the method proposed in this manuscript has strong sensitivity.

The Spearman correlation coefficients among the results of the three methods are shown in Fig 7. It can be clearly seen from the Fig 7 that the correlation between the IVIF-Extended TOPSIS and IVPF-TOPSIS methods is not very high (Spearman correlation coefficient is less than 0.8), because the IVIF-Extended TOPSIS method did not consider neutrality, and the IVPF-TOPSIS method did not consider the shortcomings of TOPSIS method in its evaluation. The Fig 7 also shows that the correlation between the IVPF-TOPSIS method and the proposed IVPF-improved TOPSIS is very high, up to 0.900. However, the sorting order between these two methods is not exactly the same, indicating the effectiveness of improved TOPSIS method based on IVPFS.

thumbnail
Fig 7. Spearman correlation coefficients between the results of the different methods.

This figure presents the Spearman correlation coefficients that measure the relationships between the results obtained from various evaluation methods.

https://doi.org/10.1371/journal.pone.0294596.g007

6. Conclusion

Design concept evaluation is an essential step in the product development process. The objective of this manuscript is to introduce an innovative approach for evaluation design concepts that takes into account multiple criteria while addressing the challenges of uncertainty and imprecise information. The main contributions of this manuscript can be summarized as follows:

  • This study effectively addressed fuzziness and uncertainty in the design concept evaluation using IVPFS theory. Specifically, this work provided a practical method to convert qualitative assessment data into IVPFNs for adaptable decision-making in design evaluation.
  • This study considered the advantages of the OWIA operator and combines IVPFS and OWIA operators to form the IVPFOWIA operator, which is used to aggregate expert decision information into a comprehensive decision matrix.
  • This study combined the improved TOPSIS method with IVPFS theory to evaluate various design concepts and determine the best one. This approach overcomes the limitation of traditional TOPSIS, which could not account for dynamic changes in the evaluation index sequence, thus ensuring the accuracy of decision outcomes.

The results of the alternatives ranking and the comparative analysis validate the accuracy and scientific validity of the IVPFS-improved TOPSIS model presented in this study. As shown in this manuscript, the provided design concept evaluation and choose algorithms can also be used to deal with other uncertain decision-making problems.

In this manuscript, there are still some limitations that need to be addressed in subsequent research: (i) The methods of objectively calculating weights often overlook important information related to subjective factors, which may result in decision results not being sufficient to reflect the actual situation, as they cannot capture the influence of human subjective experience and knowledge. (ii) In this study, improved TOPSIS method cannot eliminate the nonlinear correlation between indexes.

In the future research, we will analyze and build design concept evaluation index system from perspective of producers, designers and users. we will also explore other operators to combine with IVPFS, enabling the model to adapt to different decision-making scenarios and requirements. In addition, we intend to improve more decision making method, such as ANP and MABAC methods, in order to combine with IVPFS for ranking alternatives.

Appendix A

Proof. By applying mathematical induction to positive integer n, we demonstrate that Eq (7) holds.

When n = 1, we have:

Thus, Eq (7) holds for n = 1.

Assuming that the Eq (7) also holds when n = k, then:

Then, when n = k+1, by inductive assumption and Definition 4, we have

The conclusion that Eq (7) is true for positive integer n = k+1 follows from this. After that, it is true for all positive integers n.

References

  1. 1. Qi J, Hu J, Peng Y. Modified rough VIKOR based design concept evaluation method compatible with objective design and subjective preference factors. Applied Soft Computing. 2021;107. WOS:000663737200006.
  2. 2. Chen Z, Zhong P, Liu M, Sun H, Shang K. A novel hybrid approach for product concept evaluation based on rough numbers, shannon entropy and TOPSIS-PSI. Journal of Intelligent & Fuzzy Systems 2021;40:12087–99. WOS:000667508800118.
  3. 3. Bozorg-Haddad O, Zolghadr-Asli B, Loaiciga HA. A handbook on multi-attribute decision-making methods: John Wiley & Sons; 2021.
  4. 4. Shidpour H, Da Cunha C, Bernard A. Group multi-criteria design concept evaluation using combined rough set theory and fuzzy set theory. Expert Systems with Applications. 2016;64:633–44. WOS:000383810800050.
  5. 5. Qi J, Hu J, Huang H, Peng Y. New customer-oriented design concept evaluation by using improved Z-number-based multi-criteria decision-making method. Advanced Engineering Informatics. 2022;53. WOS:000825367100003.
  6. 6. Chan L-K, Wu M-L. A systematic approach to quality function deployment with a full illustrative example. Omega. 2005;33(2):119–39.
  7. 7. Zhai L-Y, Khoo L-P, Zhong Z-W. Design concept evaluation in product development using rough sets and grey relation analysis. Expert Systems with Applications. 2009;36(3):7072–9. WOS:000263817100161.
  8. 8. Zhu G-N, Hu J, Ren H. A fuzzy rough number-based AHP-TOPSIS for design concept evaluation under uncertain environments. Applied Soft Computing. 2020;91. WOS:000535477500013.
  9. 9. Chen Z, Zhong P, Liu M, Ma Q, Si G. An integrated expert weight determination method for design concept evaluation. Scientific Reports. 2022;12(1):6358. pmid:35428829
  10. 10. Tiwari V, Jain PK, Tandon P. Product design concept evaluation using rough sets and VIKOR method. Advanced Engineering Informatics. 2016;30(1):16–25. WOS:000370094400002.
  11. 11. Song W, Ming X, Wu Z. An integrated rough number-based approach to design concept evaluation under subjective environments. Journal of Engineering Design. 2013;24(5):320–41.
  12. 12. Zhang JM, Wei XP, Wang J, Ieee, editors. Evaluating design concepts by ranking fuzzy numbers. International Conference on Machine Learning and Cybernetics; 2003 2003 Nov 02–05; Xian, PEOPLES R CHINA2003.
  13. 13. Carnahan J, Thurston D, Liu T. Fuzzing ratings for multiattribute design decision-making. Journal of Mechanical Design. 1994.
  14. 14. Akay D, Kulak O, editors. Evaluation of product design concepts using grey-fuzzy information axiom. IEEE International Conference on Grey Systems and Intelligent Services; 2007 2007 Nov 18–20; Nanjing, PEOPLES R CHINA2007.
  15. 15. Geng X, Chu X, Zhang Z. A new integrated design concept evaluation approach based on vague sets. Expert Systems with Applications. 2010;37(9):6629–38. WOS:000278424600059.
  16. 16. Xu Z, Zhao N. Information fusion for intuitionistic fuzzy decision making: an overview. Information Fusion. 2016;28:10–23.
  17. 17. Hayat K, Ali MI, Alcantud JCR, Cao B-Y, Tariq KU. Best concept selection in design process: An application of generalized intuitionistic fuzzy soft sets. Journal of Intelligent & Fuzzy Systems. 2018;35(5):5707–20. WOS:000451343900073.
  18. 18. Wang C-H. An intuitionistic fuzzy set–based hybrid approach to the innovative design evaluation mode for green products. Advances in Mechanical Engineering. 2016;8(4):1687814016642715.
  19. 19. Haktanir E, Kahraman C. A novel picture fuzzy CRITIC & REGIME methodology: Wearable health technology application. Engineering Applications Of Artificial Intelligence. 2022;113. WOS:000830168800011.
  20. 20. Zadeh LA. Fuzzy sets. Information & Control. 1965;8(3):338–53.
  21. 21. Hajiaghaei-Keshteli M, Cenk Z, Erdebilli B, Özdemir YS, Gholian-Jouybari F. PYTHAGOREAN FUZZY TOPSIS METHOD FOR GREEN SUPPLIER SELECTION IN THE FOOD INDUSTRY. Expert Systems with Applications. 2023;224. WOS:000982581000001.
  22. 22. El-Morsy S. Stock Portfolio Optimization Using Pythagorean Fuzzy Numbers. Journal of Operational and Strategic Analytics. 2023;1(1):8–13.
  23. 23. Atanassov KT. Intuitionistic fuzzy sets. Fuzzy Sets & Systems. 1986;20(1):87–96.
  24. 24. Cuong BC, Kreinovich V, Ieee, editors. Picture Fuzzy Sets—a new concept for computational intelligence problems. Third World Congress on Information and Communication Technologies (WICT); 2013 2013 Dec 15–18; Hanoi, VIETNAM2013.
  25. 25. Cao G, Shen LX. A novel parameter similarity measure between interval-valued picture fuzzy sets with its application in pattern recognition. 2023;- 44(6):- 10239.
  26. 26. Kahraman C, Onar SC, Oztaysi B. Cloud Service Provider Selection Using Interval-Valued Picture Fuzzy TOPSIS. 2022; 504: 507.
  27. 27. Kahraman C, Oztaysi B, Onar S. A Novel Interval Valued Picture Fuzzy TOPSIS Method: Application on Supplier Selection. 2022; 39 (5–6): 635.
  28. 28. Wei G. Picture fuzzy aggregation operators and their application to multiple attribute decision making. Journal of Intelligent & Fuzzy Systems 2017;33(2):713–24. WOS:000406149900004.
  29. 29. Saha A, Reddy J, Kumar R. A Fuzzy Similarity Based Classification with Archimedean-Dombi Aggregation Operator. Journal of Intelligent Management Decision. 2022;1(2):118–27.
  30. 30. Senapati T, Chen GY, Mesiar R, Yager RR. Intuitionistic fuzzy geometric aggregation operators in the framework of Aczel-Alsina triangular norms and their application to multiple attribute decision making. Expert Systems with Applications. 2023;212. WOS:000870058400003.
  31. 31. Liu P, Chen SM. Group Decision Making Based on Heronian Aggregation Operators of Intuitionistic Fuzzy Numbers. IEEE Transactions on Cybernetics. 2017;47(9):2514–30. pmid:28029636
  32. 32. Zeng SZ, Hu YJ, Llopis-Albert C. Stakeholder-inclusive multi-criteria development of smart cities. Journal Of Business Research. 2023;154. WOS:000863222800001.
  33. 33. Batool B, Abdullah S, Ashraf S, Ahmad M. Pythagorean probabilistic hesitant fuzzy aggregation operators and their application in decision-making. Kybernetes. 2022;51(4):1626–52. WOS:000661923000001.
  34. 34. Luo MX, Long HF. Picture Fuzzy Geometric Aggregation Operators Based on a Trapezoidal Fuzzy Number and Its Application. Symmetry-Basel. 2021;13(1). WOS:000611837900001.
  35. 35. Garg H. Some Picture Fuzzy Aggregation Operators and Their Applications to Multicriteria Decision-Making. Arabian Journal for Science and Engineering. 2017;42(12):5275–90. WOS:000414788400022.
  36. 36. Zoghi M, Rostami G, Khoshand A, Motalleb F. Material selection in design for deconstruction using Kano model, fuzzy-AHP and TOPSIS methodology. Waste Management and Research. 2022;40(4):410–9. pmid:33928814
  37. 37. Baydas M, Elma OE, Pamucar D. Exploring the specific capacity of different multi criteria decision making approaches under uncertainty using data from financial markets. Expert Systems with Applications. 2022;197. WOS:000792298400004.
  38. 38. Jiang F, Ma L, Broyd T, Chen WY, Luo HB. Digital twin enabled sustainable urban road planning. Sustainable Cities And Society. 2022;78. WOS:000780336800002.
  39. 39. Jin CX, Mi JS, Li FC, Liang MS. A novel probabilistic hesitant fuzzy rough set based multi-criteria decision-making method. Information Sciences. 2022;608:489–516. WOS:000864028900006.
  40. 40. Wang YB, Jia XL, Zhang LX. Evaluation of the survival of Yangtze finless porpoise under probabilistic hesitant fuzzy environment. International Journal Of Intelligent Systems. 2022;37(10):7665–84. WOS:000788628800001.
  41. 41. Hwang CL, Yoon K, Hwang CL, Yoon K. Multiple Attribute Decision Making. Lecture Notes in Economics & Mathematical Systems. 1981;404(4):287–8.
  42. 42. Saaty TL. What is the Analytic Hierarchy Process? Mathematical Modelling. 1988;48:3–13.
  43. 43. Ju Y, Ju D, Santibanez Gonzalez EDR, Giannakis M, Wang A. Study of site selection of electric vehicle charging station based on extended GRP method under picture fuzzy environment. Computers and Industrial Engineering. 2019;135:1271–85. WOS:000482244100094.
  44. 44. Sivaloganathan AMKaS. Development of a methodology for using function analysis in flexible design strategies.
  45. 45. Masmali I, Hassan R, Shuaib U, Razaq A, Razzaque A, Alhamzi G. Stock Reordering Decision Making under Interval Valued Picture Fuzzy Knowledge. 2023; 15 (4).
  46. 46. Gocer F. A Novel Interval Value Extension of Picture Fuzzy Sets Into Group Decision Making: An Approach to Support Supply Chain Sustainability in Catastrophic Disruptions. Ieee Access. 2021;9:117080–96. WOS:000690419100001.
  47. 47. Shumaiza , Akram M, Al-Kenani AN, Alcantud JCR. Group Decision-Making Based on the VIKOR Method with Trapezoidal Bipolar Fuzzy Information. Symmetry. 2019;11(10):1313.
  48. 48. Tiwari V, Jain PK, Tandon P. An integrated Shannon entropy and TOPSIS for product design concept evaluation based on bijective soft set. Journal of Intelligent Manufacturing. 2017.
  49. 49. Wang T-C, Lee H-D. Developing a fuzzy TOPSIS approach based on subjective weights and objective weights. Expert Systems with Applications. 2009;36(5):8980–5.
  50. 50. Burillo P, Bustince H. Entropy on intuitionistic fuzzy sets and on interval-valued fuzzy sets. Fuzzy sets and systems. 1996;78(3):305–16.
  51. 51. Joshi DK, Kumar S. Entropy of interval-valued intuitionistic hesitant fuzzy set and its application to group decision making problems. Granular Computing. 2018.
  52. 52. Ye J. Fuzzy cross entropy of interval-valued intuitionistic fuzzy sets and its optimal decision-making method based on the weights of alternatives. Expert Systems with Applications. 2011;38(5):6179–83. WOS:000287419900171.
  53. 53. Qiyas M, Abdullah S, Al-Otaibi YD, Aslam M. Generalized interval-valued picture fuzzy linguistic induced hybrid operator and TOPSIS method for linguistic group decision-making. Soft Computing. 2021;25(7):5037–54. WOS:000619703000002.
  54. 54. Mahmood T, Waqas HM, Ali Z, Ullah K, Pamucar D. Frank aggregation operators and analytic hierarchy process based on interval-valued picture fuzzy sets and their applications. International Journal Of Intelligent Systems. 2021;36(12):7925–62. WOS:000693320300001.
  55. 55. Sun HY, Ma Q, Chen Z, Si GY. A Novel Decision-Making Approach for Product Design Evaluation Using Improved TOPSIS and GRP Method Under Picture Fuzzy Set. International Journal Of Fuzzy Systems. 2023;25(4):1689–706. WOS:000936440200001.
  56. 56. Wenyan S, Zixuan N, Pai Z. Design concept evaluation of smart product-service systems considering sustainability: An integrated method. Computers & Industrial Engineering. 2021;159.
  57. 57. Turan FM, Badrul O. The Integration of HOQ and Fuzzy-AHP for Design Concept Evaluation. Applied Mechanics and Materials. 2013;315(1):25–9.
  58. 58. Yang XA, Deng Q, Sun GL, Wang BB. Research on Evaluation and Decision-Making Model of Product Design Scheme Based on TFN-AHP-GRA. Advanced Materials Research. 2011;201–203:1170–6.
  59. 59. Yu SH, DengKai FangWang, editors. Study of information axiom in evaluation method of product design scheme. I Desi & Con Design; 2008.
  60. 60. Kim KJ, Lim CH, Heo JY, Lee DH, Park K. An Evaluation Scheme for Product-Service System Models with a Lifecycle Consideration from Customer’s Perspective. Springer Singapore. 2013.
  61. 61. Kocak SA, Alptekin GI, Bener AB, editors. Evaluation of Software Product Quality Attributes and Environmental Attributes using ANP Decision Framework. International Workshop on Requirements Engineering for Sustainable Systems; 2014.
  62. 62. Wang D, Yu H, Wu J, Meng Q, Lin Q. Integrating fuzzy based QFD and AHP for the design and implementation of a hand training device. J Intell Fuzzy Syst. 2019;36(4):3317–31.