## Figures

## Abstract

Design concept evaluation is a huge challenge in the R&D stage of new product development. The information in the assessments often depends on the decision-makers’ individual preferences. However, sometimes the decision-makers cannot give precise and complete information because it is very difficult for them to be familiar with all the criteria. In this situation, an incomplete information decision-making matrix is established. In this paper, decision-making methods based on incomplete information are compared in the literature review. Incomplete information determination method using trust mechanism is proved as a proper way to solve this problem, and the missing information are computed based on the alternatives However, in design concept evaluation, experts commonly provide their preferences using linguistic words according to the different attributes. Hence, we propose a three-step Multiple Attributes Group Decision-making (MAGDM) method where the missing value are determined by attributes. In step one, a data repairing method is proposed based on trust theory. After that, in step two, a comprehensive weight determination method combining AHP and entropy is proposed to obtain the weight of index attributes. Finally, the Rough-TOPSIS method is applied in the design scheme ranking step. In the case study, the proposed method is implemented in a tourism product design process to show its effectiveness.

**Citation: **Chen Z, Pan Z, Ma Q, Hou T, Zhao P (2022) An MAGDM method for design concept evaluation based on incomplete information. PLoS ONE 17(11):
e0277964.
https://doi.org/10.1371/journal.pone.0277964

**Editor: **Ronnason Chinram, Prince of Songkla University, THAILAND

**Received: **September 18, 2022; **Accepted: **November 8, 2022; **Published: ** November 23, 2022

**Copyright: ** © 2022 Chen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Data Availability: **All relevant data are within the paper and its Supporting information files.

**Funding: **This work was supported by Major Art Project of China National Social Science Fund (21ZD25) awarded to Zhao Pan; The Culture and Tourism Research Project of Shandong Province, China (21WL(H)64) awarded to Zhe Chen; and Humanities and Social Sciences Foundation of the Ministry of Education, China (22YJCZH142) awarded to Zhe Chen. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Competing interests: ** The authors have declared that no competing interests exist.

## 1. Introduction

Design concept evaluation is important in the early stage of new product development (NPD) [1]. Once the design scheme is determined, over 70% of the investment in product design and manufacture is determined [2]. Remodeling the design scheme at a later stage may have a huge adverse impact on the quality, cost, and timeliness of the product [3]. A common method for R&D at the early stage of NPD is to prepare multiple design schemes, which are then judged by the decision-makers (DMs) according to some evaluation attributes, and the decision-making organizer needs to select an optimal alternative according to the information obtained from the decision-makers’ preferences and the design parameters. Later, some improved and modified methods based on the mentioned theories are proposed for specific situations. Problems with several alternatives, which have multiple attributes like this, are usually defined as multiple attribute decision-making (MADM) problems. Hence, design concept evaluation is a critical application of MADM.

How to deal with an imprecise and uncertain environment is a huge challenge in recent studies. To make assessments precise and objective, utility theory, the Analytic Hierarchy Process method (AHP), the graphical method, the Quality Function Deployment method (QFD), and fuzzy logic are applied in design concept evaluation [4]. Due to the uncertainty caused by the subjectivity of the user preferences and for other reasons, fuzzy logic has been integrated in MADM problems for years [5]. Fuzzy logic was first introduced by Zadeh [6], and soon showed its superiority. Its extensions, including interval fuzzy set [7], intuitionistic fuzzy set [8], rough set [9], triangular fuzzy set [10, 11] and trapezoidal fuzzy set [12], are widely applied in related fields including design concept evaluations.

However, most fuzzy logics do not have complete decision-making models, which means the assessment organizers cannot determine the rankings of the alternatives only by the fuzzy set. Instead, fuzzy set normally has a role in building a fuzzy environment to address the uncertainty of the decision-making models. Hence, fuzzy set integrated methods are applied in design concept evaluation and other MADM problems.

The Technique of Order Preference by Similarity to Ideal Solution (TOPSIS), proposed by Hwang and Yoon, is one of the most popular MADM methods, and is widely used in several fields including design concept evaluation [13, 14]. To decrease the subjectivity caused by individuals, the decision organizer recruits more experts in the decision-making process [15], and the MADM methods evaluated by a group of experts are defined as Multiple Attributes Group Decision-making (MAGDM). MAGDM, which integrates the fuzzy set and TOPSIS method, has shown its advantages for design concept evaluation [5]. However, the decision matrix may be incomplete because some experts may not give their opinions quickly based on their backgrounds and experiences [16]. Once the expert group increases in size and the criteria and products become more complicated, more incomplete decision matrices will occur. In previous studies, methods have been implemented to deal with incomplete decision matrices [17, 18]. The trust mechanism is an effective way of determining the missing values. Xu [19] computed the missing data by combining direct and indirect trust. However, in design concept evaluation, experts commonly provide their preferences using linguistic words according to the different attributes, and it is preferred that the missing information is determined by the existing information for the same attribute.

To solve this problem, a novel trust mechanism and Rough-TOPSIS method is proposed in this paper to deal with the uncertain and incomplete information. We first propose a method to determine the missing values relying on the existing information for the corresponding attribute. Meanwhile, we also combine the rough set and the TOPSIS method. The decision matrix is transferred into rough numbers, and a distance based TOPSIS method is applied in the alternative ranking phase. Then a real-life case study is applied for design concept evaluation, and the properties of the improved closeness are analyzed.

The remainder of the paper is structured as follows: Section 2 reviews relevant decision-making models, Section 3 proposes a novel design concept evaluation method based on incomplete information, Section 4 provides an example and some comparisons to demonstrate how the framework can be applied, and Section 5 concludes with some important findings.

## 2. Literature review

This section includes two parts. The related MAGDM studies in design concept evaluation are reviewed in section 2.1, then methods to handle incomplete information are reviewed in section 2.2.

### 2.1 Related MAGDM methods for design concept evaluation

In the R&D of new products, some design concept evaluation methods are proposed as quick decision-making approaches, such as SWOT analysis [20], house of quality [21], Pugh chart [22], and screening matrix [23]. However, because products in modern society are complex, and numerous competing decision factors need to be considered in the development process, MAGDM approaches are implemented in design concept evaluation [15]. There are two main MAGDM approaches. In some MAGDM problems, the decision-makers are asked to give their preferences, and the alternatives are ranked by comparing the preferences without the criteria [24]. This is the alternative-based approach. However, in design concept evaluation, the ranking of the alternatives is relevant to the decision factors (criteria). Hence, it is appropriate to rank the alternatives based on the criteria.

Evaluating the alternatives by distance measurement is one of the most popular theories in MAGDM problems. In distance based ranking methods, the ideal solution is firstly determined by a mathematical model, and the distances between the alternatives and the ideal solution are then calculated. At the end, the alternatives are ranked by the distances. The most common distance based MAGDM methods are the TOPSIS method and the VIKOR method [25, 26].

Enterprises facing tough market competition are paying more attention to user experiences. Under the guidance of user-centered design thinking, expert user groups have participated in the decision-making of solutions, where the participation of the customers largely improves the usability of the products. Meanwhile, the subjectivity of the user preferences results in uncertainty of decision-making information, especially at the design concept evaluation stage. Methods such as the fuzzy set [27] and grey degree theory [28] have been used in MAGDM decision-making. The two methods rely on additional parameters to describe the boundaries of fuzzy intervals, which may increase the subjectivity and complexity of decision-making [29].

The rough set theory, based on the rough number (RN), has shown its advantage in product design decisions. Compared with the other fuzzy methods, the rough set no longer depends on any external information in the data processing process [30]. In other words, this method can only rely on evaluation indicators and objective data for evaluation. Therefore, it can objectively process a large number of subjective data items. In addition, rough sets also show excellent performance in representing the fuzziness of information. In the information processing, the fuzzy numbers are simply expanded the same distance towards both sides. However, the rough set method establishes the interval via a series of rigorous equations. Thus, the rough set method is similar to human cognition behaviors. The evaluation of the design concept is based on human cognition behaviors, the rough set method is suitable for design concept evaluation.

The method describes the uncertainty of decision-making information in the form of rough intervals, transferred from the obtained crisp numbers according to their relationships. Zhai [31] first introduced the concept of rough set of product design evaluation, and proposed a design scheme evaluation method based on the rough set and the gray level. It is difficult to evaluate effectively because of a lack of an evaluation system relying only on the rough set theory. The combination of traditional evaluation system and rough set can achieve better results. In practice, many multiple rough sets based on methods have been successively applied in different product decision-making processes including Rough-TOPSIS [29], Rough-VIKOR [32] and Rough-AHP & Rough-TOPSIS [33]. The TOPSIS based method is considered the commonly used method. Thus, we combine the rough set and TOPSIS method in our research. In design concept evaluations, the experts often give their preferences by linguistic information, which is then transferred into crisp numbers according to the multiple level scale table [34]. In the rough set based on methods, the crisp numbers are then converted into rough numbers.

### 2.2 Handling method for incomplete information in MAGDM

Although we have complete MAGDM methods for design concept evaluation, it is still necessary to develop strategies to deal with an incomplete decision matrix. When the experts make their preferences, it is difficult for them to give accurate judgments on all the criteria due to their lack of knowledge on some criteria. To avoid bias in the raw data, the decision-maker can refuse to give judgment on questions they are not familiar with. Hence, a decision matrix with missing information occurs.

At present, there are some methods for dealing with incomplete information. One way is to determine the optimal alternative without the missing information being completed, where the alternatives are ranked with an incomplete decision matrix. Zhang et al. proposed a two-sided matching decision making (TSMDM) method based on multi-particle hesitant fuzzy linguistic term sets (HFLTS) to simulate this situation, aiming at the problem of TSMDM with high uncertainty, and the matching object may have some hesitancy [35]. The characteristics of the proposed method were verified by taking the matching between the supply and demand of green building technology as an example. Li et al. established the optimization model, and for consensus reaching processes (CRPs) in group decision making (GDM) with incomplete hesitant fuzzy linguistic preference relations (IHFLPRs), proposed some feedback adjustment rules based on local adjustment strategies and an iterative consensus algorithm for GDM with IHFLPRs to ensure the non-randomness and logic of decision-makers. Zhang et al. [36] proposed a new MAGDM method based on multi granulation probabilistic models, Multi-Objective Optimization by Ratio Analysis plus the full multiplicative form and the technique of precise order preference (TPOP) method in incomplete q-rung orthopair fuzzy (q-ROF) information systems was systematically investigated, and the validity of the presented methodology was demonstrated by two experimental studies. Nafei et al. [37] first proposed a modified score function for ranking single-valued neutron sophic numbers. Then, they suggested a TOPSIS method based on the proposed function for decision-making under group recommendation. They provided powerful and practical tools for dealing with uncertainty in decision-making problems. Wang et al. [38] proposed a method based on interval intuitionistic fuzzy numbers. They applied a linear method to fuse each individual intuitionistic fuzzy number (IVIFN), and formed the fused IVIFN of each scheme. The alternatives were then compared and sorted. Wei [39] established an incomplete information processing method by using the traditional grey relational analysis theory (GRA) combined with IVIFS. Liu et al. [40] further proposed an intuitionistic fuzzy multi-attribute decision-making method based on evidence reasoning, converting intuitionistic fuzzy numbers into level of confidence. Finally, the optimal scheme is selected based on the total evaluation value. Hua et al. [17] proposed a DS-AHP method for MAGDM problems with incomplete information. Initially, all possible focal elements directly from the incomplete decision matrix are identified, and then the basic probability assignment of each focal element is determined. Afterwards, the belief intervals of the decision alternatives are evaluated, and preference relations among the alternatives can be computed, and the alternatives are ranked according to the preference relations. Another way of dealing with incomplete information is to estimate the missing values of the incomplete matrix. Hong et al. [41] established a new learning algorithm to repair the missing information based on the rough set theory. Xu et al. [42] integrated the continuous interval argument operator and the continuous interval average operator to transform the incomplete uncertain judgment matrix into a complementary judgment matrix, and proposed a calculation method for the correlation matrix.

Recently, studies of incomplete matrices conducted from the perspective of evaluation experts themselves and their relationships proposed feasible methods for incomplete matrices based on the trust mechanism. Capuano et al. [18] established an incomplete data processing method through expert preference and the trust mechanism, and estimated the missing values through a social influence network model. Xu et al. [19] adopted an incomplete matrix repair method based on the trust mechanism for large-scale and incomplete preference information. Based on the trust mechanism, the direct trust degree and the recommendation trust was calculated, and the incomplete decision matrix was repaired. In other words, the incomplete information can be determined by comparing the trust degree with other decision-makers, and comprehensively calculating the information and trust degree of other decision-makers. It is a practical method to repair missing values, and can meet the needs of common MAGDM methods. The missing values of an expert depend on the information provided by the expert’s most trusted decision-maker. Nevertheless, the trust degree may change among various criteria, meaning that because expert A trusts expert B in terms of a specific attribute does not mean A continues to trust B on another attribute. Hence, it is an appropriate solution to determine the missing data by comparing the trust degree according to the specific attribute.

According to the trust mechanism, the expert confidence can be set as an interval [0, 1], where 0 means without any trust at all, and 1 means with complete trust. Therefore, the trust degree of any expert *x* to expert *y* can be represented by a numerical value in [0, 1]. The overall trust mainly includes direct trust and indirect trust [43], where the direct trust degree represents the mutual trust degree between experts, and the indirect trust degree represents the third party’s trust degree in the expert group. Although the expert groups for design concept evaluation normally consist of professional designers, manufacturers, and experienced customers [15], the selection of decision-making group members has a certain degree of randomness. They are likely to belong to multiple different fields except for their experiences in the specific product. Generally, there is not any direct interaction between decision-making members, and they lack relatively stable trust relationships. Accordingly, in this study, it is believed that the overall trust degree directly depends on the indirect trust degree.

Mostly, the missing values are computed by the existing information. The evaluation mechanism and measurement may be different based on the attributes, but the measurement standards for a specific attribute among the alternatives can be considered to be consistent. Hence, the missing data can be determined based on the attribute.

## 3. Preliminaries

This paper adopts the TOPSIS method and combines rough number theory to evaluate concept design. Although the rough number method has advantages in dealing with uncertain information in the early stage of new product development, it is difficult to form an effective evaluation because it only relies on rough number theory and lacks a relevant evaluation system. Using the TOPSIS combined with rough number theory for evaluation can achieve better results.

The specific process of information treatment is shown as follows [31]:

Let *U* be the universe with the objects according to the crisp numbers. Let object *R* be a set with *n* classes, expressed as *R* = {*C*_{1}, *C*_{2}, *C*_{3}, …, *C*_{n}}, where *C*_{1} < *C*_{2} < *C*_{3} < ⋯ < *C*_{n}. Assume *C*_{i} is an interval, and *C*_{i} ∈ *R*, 1 ≤ *i* ≤ *n*. The lower and the upper limit can be denoted as *C*_{li} and *C*_{ui}, respectively. Then we have *C*_{i} = [*C*_{li}, *C*_{ui}], where *C*_{li} < *C*_{ui}.[34] ∀ *Y ∈ U*, the lower and the upper approximation can be defined as the equations below.

Where *Apr* (*C*_{i}) and illustrate the lower and the upper approximation, respectively. The boundary region of *C*_{i} is:
(2)

Let *Lim* (*C*_{i}) and be the upper and the lower limit of *C*_{i}, then
(3)

Where *M*_{L} / *M*_{U} is the number of elements in . The *RN* (*C*_{i}) can be expressed as:
(4)

The interval of the *RN*(*C*_{i}) is computed as:
(5)

In a rough set, the *RBnd* (*C*_{i}) shows the vagueness of the class. The operations of RN are similar to vector operations of intervals, which are shown in Eqs (6)–(9), where *k* represents a constant, *and RN*_{1} and *RN*_{2} represent two RNs.

## 4. The proposed method based on incomplete information

This study proposes a decision-making method based on the trust mechanism and the rough-TOPSIS method. A four-phase procedure is proposed in our study as shown in Fig 1. In phase 1, the preparations are made for the assessment, and the incomplete information is obtained from the decision-makers according to the criteria. Then a missing data determination method that integrates other decision-makers and trust degrees is proposed in phase 2. Afterwards, in phase 3 the criteria weights are determined by an integrated method combined with AHP and Shannon entropy. The alternatives are then ranked by a Rough-TOPSIS method.

### 4.1 Obtain the raw data

In the decision preparation phase, the design schemes are prepared as the alternatives, and the criteria need to be chosen and the expert group needs to be arranged. Subsequently, the experts are asked to give their preferences of the alternatives according to the criteria.

Assume *p* decision-makers (*DM*_{1} ~ *DM*_{p}) are assigned to provide their preferences on *m* design schemes (Alternative 1~ *m*) according to *n* attributes (A_{1}~A_{n}). The preferences are in the form of a seven-level scale linguistic information shown as Table 1, and the linguistic information is then transferred into crisp numbers.

When the decision-maker *DM*_{t} (1 ≤ *t* ≤ *p*) is not available to make an accurate judgment on attribute *j* of scheme *i*, the decision-maker can refuse to give the corresponding judgment, then is recorded as "-", where represents the judgment made by the *DM*_{t} based on scheme *i* and attribute *j*. Related examples are shown in Table 2. To determine the missing values from the raw data and complete all items marked with "-", it is necessary to establish a numerical method to compute the missing information on the basis of existing values.

### 4.2 Compute the missing values

Before the assessment, the missing data in the decision matrix need to be calculated. In the proposed method, we use a model by trust mechanism, and missing data are determined by the indirect trust degree. The acquisitions of indirect trust generally come from the evaluation of third-party institutions, or by some calculation methods. In design concept evaluation, the similarity based on method is applied, and the mean of preference values is calculated through the willingness of experts to reach consensus as soon as possible in decision-making problems [44]. In this study, the indirect trust degree of the subjects is judged by the quantitative relationship between each decision data item and the mean value of the preferences. The calculation method is as follows:

The decision matrix *D* of all experts’ judgements for a specific attribute *A*_{j} is shown in Table 3.

Assume *τ* experts “*g*, …, *g*′” in the expert group cannot make accurate judgments on an attribute due to some reasons, where 1 ≤ *g* ≤ *g*′ ≤ *p*, and *τ* ≪ *p*. As is shown in Table 3, each missing information item in the groups to is marked as "-". Hence, the missing values need to be determined according to information provided by other decision-makers.

Let be the expert decisions after removing ‘’, where *i* ∈ {1, 2, …, *m*}.

Then the mean value of the preference vector avg_{ij} for *Arr*_{i} can be expressed by the equation below:
(10)

Where *t* means the sequence of the experts, and *t* ∈ {1, 2, …, *p* − *τ*}. According to Torra and Xu [45, 46], the similarity of expert *t* and avg_{ij} can be calculated by Eq (11).

Where sim(α, β) means the similarity of α and β. Let be the acceptability of expert *t*, then
(12)

Considering *m* alternatives of attribute *j* comprehensively, the average acceptability of expert can be computed as:
(13)

The larger the value of is, the easier it is to accept the judgment of expert *t* for attribute *j*. Assume expert *h* has the largest value of , the values of can be replaced by , thus:
(14)

Similarly, the other missing data can be determined in the same way, and then we have a complete decision matrix. The decision matrix obtained by expert *t* is shown as:
(15)

### 4.3 Determine the criteria weight

Criteria weight determination is a vital phase in design concept evaluation, and an integrated weight determination model is established here. In our study, a combination of the subjective weight determination method and the objective method is applied to reserve the decision-makers’ opinion as well as the objective of the assessment [47]. The details of the criteria weight determination are shown, including the subjective weight determination, the objective weight determination and the criteria weight integration.

#### 4.3.1 Calculate the subjective weight based on AHP.

The AHP method was first proposed by Saaty [48], and it has become the most important method of the subjective decision-making approach. The core of the method is quantitative analysis by comparing experts’ preferences pairwise. The main steps are as follows:

All the experts need to draw corresponding pairwise conclusions on the relationship between the criteria, using the nine-level scale [48] as shown in Table 4, and transform the experts’ judgment language into the evaluation scale value. We define *u* and *v* as two attributes, and 1 ≤ *u* < *v* ≤ *n*.

Let *b*_{uv} be the comparison result of *u* versus *v*, and *b*_{uv} is generally obtained from the experts. Hence, a judgment matrix B is established by comparing the relative importance of the evaluation indicators in pairs as follows:
(16)
Where *b*_{uv} = (*b*_{uv})^{−1}.

The column vector normalization of matrix *B* can be calculated by Eq (17).

Where *c*_{uv} is the element of the normalized decision matrix. The row summation of the normalized judgment matrix *C*_{u} can be computed by Eq (18).

Thus, the weight of the *u*^{th} row of the normalized decision matrix is calculated as:
(19)

Compute the largest eigenvalue of the decision matrix *λ*_{max} using the equation as follows:
(20)

Where *B* is the judgment matrix and *W* represents the eigenvector . The consistency index of *B* is computed as:
(21)

Hence, the consistency ratio is computed as the following equation: (22)

Where *R*_{I} is the random consistency index, and the values of the 1–9 order matrix *R*_{I} are shown in Table 5.

When *C*_{R} < 0.1, the consistency requirement is met; otherwise, the judgment matrix should be adjusted.

#### 4.3.2 Calculate the objective weight determination based on entropy.

The entropy weight method is an important weighting method. In this study, the entropy weight method is used for objective weighting, and the specific weighting method is as follows.

The average value of the experts should be calculated, and the modified decision matrix can be expressed as *D*′, calculated by the equation below:
(23)

According to the entropy weight method, it is necessary to calculate the proportion of the *j*^{th} attribute of the *i*^{th} alternative:
(24)

The entropy attribute *j* can be calculated by the following equation:
(25)

Where *k* is a constant, and usually *k* = (ln*m*)^{−1}. Then the objective weight of judging attribute *j* should satisfy , and . Then
(26)

#### 4.3.3 Integrate criteria weight.

The subjective weight mainly depends on the personal subjective tendency of the experts, while the objective weight depends on the numerical characteristics of the assessment data. The integration methods for evaluating data mainly include additive integration and multiplication integration. The integration method is as follows: (27)

Where *μ* ∈ [0, 1] is the coefficient showing the important rate of subjective criteria weight. When *μ* > 0.5, the subjective weight outweighs the objective weight, and vice versa.

### 4.4 Rank the alternatives by Rough-TOPSIS

From Eq (15), we define the judgments made on attribute *j* of scheme *i* as:
(28)

According to rough set theory [29], for any , we have: (29) and the upper and lower limit of can be expressed as: (30)

Where and represent the lower and the upper limit of . Let be the arithmetic mean in the corresponding set. So far, for alternative *i* and attribute *j*, the rough set of *a*_{ij} can be expressed as a rough number , where the lower and the upper limits can be calculated by Eq (31):
(31)

In this way, the rough decision matrix *X* can be written as:
(32)

Next, the decision matrix needs to be normalized. According to Chen [49], the normalization processing by the vector method will not change the characteristics of each attribute, which is an ideal solution in TOPSIS. Therefore, the vector method is applied here in this study, and the specific method is shown in Eq (33).

(33)Then the normalized decision matrix can be expressed as . The weighted decision matrix is , where * and .*

Subsequently, the Positive Ideal Solution (PIS) *v*_{P} (*j*) and Negative Ideal Solution (NIS) *v*_{N} (*j*) can be determined as:
(34)
(35)

Where *BA* and *CA* represent the benefit attribute and the cost attribute, respectively.

Next, let the interval be the distance between the PIS and .

(36)Similarly, the distance from to NIS can be expressed as , shown in Eq (37).

(37)The distance and represent the comprehensive distances from the alternatives to PIS and NIS, respectively. The calculations are shown as Eqs (38) and (39).

(38)(39)According to Song [50], the optimistic coefficient α ∈ [0, 1] can be used to express the optimism for the decision, and the exact distance from each scheme to PIS/NIS can be calculated by Eqs (40) and (41). When α is close to 1, it means that the decision-maker is optimistic about the decision result, and when α is close to 0, it means that the decision-maker is not optimistic about the decision result. In general, when α = 0.5, the exact distance is the middle value of the interval.

(40)(41)Then the closeness coefficient *CI* of each scheme is:
(42)

The larger the close distance is, the better the solution is and the closer it is to the ideal solution; otherwise, the worse the solution is. The schemes can be ranked through the value of *CIs*.

## 5. Case study

### 5.1 A real-life example: Designing a tourism product

This paper takes the design concept of the ‘Yimeng pancake’ as an example. The pancake is a distinctive food in the area around the Yimeng mountain in China. Recently, a pre-packaged pancake has been developed as a cultural tourism product in this area.

The decision preparation is made in phase 1. Based on the market research, three design concepts of the product are provided by the development team as the alternatives, marked as Alternative 1 to Alternative 3. Fig 1 shows the process of product decision-making.

As shown in Table 6, there are a total of 9 criteria determined from user experience and cultural transmission. A group of 15 designers and 15 customers are invited and assigned to give their preferences to the three design schemes according to the criteria. The linguistic information is then converted into crisp numbers according to Table 1. In this phase, some decision-makers cannot make clear judgments for all criteria, resulting in the incompleteness of the decision matrix. Therefore, the decision matrix needs to be repaired and the missing data items need to be determined.

Here we take decision attribute *A5* (cost and quality control) as an example for data patching. As is shown in Table 7, the data for decision-maker 15 is missing.

After removing the missing values of the decision- maker 15, the decision matrix *D*_{A5} is formed. Then, we take the decision attribute *A5* (cost and quality control) as an example. According to Eqs (10)–(13), the average acceptability of decision-making experts can be determined and the detailed calculation process is as follows:

First, the mean value of the preference vector avg_{iA5} (i = 1,2,3) for *Arr*_{i} can be expressed by Eq (10):
(43)

Then avg_{1A5} = 5.48, avg_{2A5} = 4.379, avg_{3A5} = 4.103

Second, the similarity of experts and avg_{iA5} (i = 1,2,3) can be calculated by Eq (11).

Next, let be the acceptability of expert *t*, then the acceptability of experts can be calculated as follows:
(45)

Finally, considering three alternatives of attribute *A*_{5} comprehensively, the average acceptability of experts can be computed as:
(46)

For example, according to the above method, we can calculate the average acceptability of expert 5 of index attribute *A5*. The other detailed calculated values of the average acceptability of decision-making experts of index attribute *A5* are shown in Table 8.

As shown in Table 8, , which is the maximum value, indicating that the data of decision-maker 14 is most easily accepted by decision-maker 15. Therefore, for the indicator A5, the missing data of decision-maker 15 can be supplemented with reference to decision-maker 14. The decision values for the three alternatives are (5, 5, 4), respectively.

The process of calculating other missing values is the same as above.

In the same way, the entire decision matrix can be completed to form a complete preference matrix *D*′.

Next, determine the criteria weight. The subjective and objective weights of each attribute are obtained by the AHP method and the entropy weight method respectively. When *μ* = 0.5, the weight value of each attribute is shown in Table 9, and the integrated criteria weights *w* are determined.

Then according to Eqs (28)–(32) in step 4, the rough decision matrix *X* can be calculated as shown in Table 10.

We take the decision attribute *A5* (cost and quality control) as an example. The detailed calculation process by Eqs (28)–(32) is as follows:

First, from Eq (15), we define the judgments made on attribute *A*_{5} of scheme 1 as:
(47)

Second, according to rough set theory [29], for , we have: (48) and the upper and lower limit of can be expressed as: (49)

Next, for alternative 1 and attribute *A*_{5}, the rough set of *a*_{1A5} can be expressed as a rough number , where the lower and the upper limits can be calculated by Eq (31):
(50)

The process of calculating the other values is the same as above, and we can see the other values in Table 10. Finally, the rough decision matrix *X* can be determined.

After normalization, the decision matrix can calculate the distance between each scheme and PIS and NIS and the closeness coefficient *CI* according to Eqs (33)–(42), as shown in Table 11.

We take the decision attribute *A5* (cost and quality control) in scheme 1 as an example, and the detailed calculation process by Eqs (33)–(42) is as follows:

First, the Positive Ideal Solution (PIS) *v*_{P} (*A*_{5}) and Negative Ideal Solution (NIS) *v*_{N} (*A*_{5}) can be determined as:
(51)
(52)

Where *BA* and *CA* represent the benefit attribute and the cost attribute, respectively.

Next, let the interval be the distance between the PIS and .

(53)The distance and represent the comprehensive distances from the alternatives to PIS and NIS, respectively. The calculations are shown as Eqs (38) and (39).

(54)(55)According to Song [50], the optimistic coefficient α ∈ [0, 1] can be used to express the optimism for the decision, and the exact distance from each scheme to PIS/NIS can be calculated by Eqs (40) and (41). When α is close to 1, it means that the decision-maker is optimistic about the decision result, and when α is close to 0, it means that the decision-maker is not optimistic about the decision result. In general, when α = 0.5, the exact distance is the middle value of the interval.

(56)(57)Then the closeness coefficient *CI* of scheme 1 is:
(58)

We can calculate the closeness coefficient *CI* of the remaining two schemes according to the above method. The larger the close distance is, the better the solution is and the closer it is to the ideal solution; otherwise, the worse the solution is. The schemes can be ranked through the value of *CIs*.

The process of calculating other values is the same as above. We can see the relative variables and CIs in Table 11.

Let α = 0.5, and it can be seen from the *CIs* that scheme 1 is the best alternative.

### 5.2 Comparison and sensitivity analysis

To reveal the effectiveness of the proposed method, a comparison is conducted. As is shown in Table 12, although all three methods use the trust mechanism for the missing value determination phase, diversities exist among the methods. There are two advantages for design concept evaluation compared with the other methods:

- The missing values are determined by preferences provided by the experts, and no more information is needed. Compared with the other methods, the proposed method does not need additional information. Considering that the experts are not familiar with each other, it is not possible to obtain the additional information shown in Table 12. Moreover, it is much more objective without additional information in this process. Hence, of the three methods, the proposed method is simple and easy to carry out.
- The proposed method is based on the criteria, while the others focus on the alternatives. The design concept evaluation is a complicated decision including multiple inconsistent criteria. The criteria are balanced in this process. Thus, multiple criteria are essential for design concept evaluation, and the superiority of the proposed method is obvious.

In the design process of real cultural and creative products, the assignment of subjective and objective weights can affect the evaluation results within a certain range. In the proposed method, the weight coefficient *μ* and the optimistic coefficient *α* are the factors that may influence the result. When the value of the coefficient *μ* is 0.1, 0.3, 0.5, 0.7, and 0.9, respectively, the closeness coefficient *CI* of the three schemes is shown in Fig 2. Similarly, when the α equals 0.1 to 0.9, the closeness coefficient *CIs* are shown in Fig 3. Some conclusions are revealed in the figures.

- When the value of
*μ*changes, with the increase of*μ*

It can be seen from Fig 2 that when the value range of*μ*changes, as the value of the evaluation coefficient*μ*increases, the gap between the three schemes gradually narrows; however, the order of the three schemes has not changed from beginning to end. Scheme A1 is better than the other two alternatives, and its order is A1 > A2 > A3. - When the value of
*α*changes, with the increase of*α*

Fig 3 shows that as the value range of modifications and the value of the evaluation optimistic coefficient*α*increases, the gap between the three schemes gradually narrows; however, the order of the three schemes remains constant from beginning to finish. Scheme A1 is superior to the other two options, and its ranking is A1>A2>A3.

## 6. Conclusions

In the design decision-making process of product design options, the source of information is often related to the decision-makers’ personal preferences. Decision-makers are often affected by their own knowledge background, decision-making experience, etc., and sometimes cannot evaluate all decision-making attributes, which will lead to incomplete information in the decision-making matrix. To address the lack of evaluation preference matrix data caused by subjective and objective reasons in product design evaluation, this study proposes a decision-making method for design scheme selection. As an extension of the widely used common decision-making method TOPSIS, this method can be applied to some incomplete information multi-attribute decision-making. The method is applied to the early stage of product design, which can improve the quality and efficiency of design and development. In real life, there are situations where considerable information is missing in the raw data, and *τ* ≪ *p* is not satisfied in Eq (10). In this situation, the proposed method is not accurate and precise because of a lack of raw information. Therefore, how to deal with a large number of missing data values is a challenge in future studies. There are some research gaps in multi-criteria sorting of group decision-making problems (MCS-GDM problems for short). For example, different MCS-GDM problems may have different needs in practical applications. Some MCS-GDM problems may be urgent and need to determine the ranking of alternatives in a short time, while for some nonurgent MCS-GDM problems, experts may want to preserve more initial opinions during the consensus reaching process (CRP). Recognizing these gaps of MCS-GDM problems, and to fill these research gaps, another study proposed an improved Rough-TOPSIS method, which attempts to reduce the imprecision for design concept evaluation in two ways. They applied this method to a numerical example for green building rating and detailed simulation experiments were presented to justify the proposed algorithms [51]. According to social network large-group decision making (SN-LGDM) problems, they aim to investigate the bidirectional interaction-based hybrid consensus strategies of subgroups, and design a minimum adjustment feedback model to help the interacting subgroups pursue greater consensus. In addition, the personalized individual semantic (PIS) model is used to deal with the ambiguity of different semantics of individuals’ expressed opinions [52].

In the future, the following topics can be investigated. First, with the rapid development of information and technology, the interaction of experts can become more and more frequent, and the experts’ comments will be influenced by each other. As a result, it is necessary to introduce new methods to deal with MAGDM problems by considering the consensus reaching process. Second, in some situations, some experts may falsify the final ranking by dishonestly giving their evaluation results. Thus, it is meaningful to study strategic falsifying issues in MAGDM problems. Finally, it will be interesting to further study MAGDM in cases where a large number of experts are involved in problems. Another future research direction is to implement proposal techniques that can be extended to decision problems within the framework of interval valued intuitionistic fuzzy sets as well as interval valued picture fuzzy environments.

## References

- 1. Nikander JB, Liikkanen LA, Laakso M. The preference effect in design concept evaluation. Design studies. 2014;35(5):473–99.
- 2. Geng X, Chu X, Zhang Z. A new integrated design concept evaluation approach based on vague sets. Expert Systems with Applications. 2010;37(9):6629–38.
- 3. Asiedu Y, Gu P. Product life cycle cost analysis: state of the art review. International journal of production research. 1998;36(4):883–908.
- 4. Ayağ Z. An integrated approach to evaluating conceptual design alternatives in a new product development environment. International journal of production research. 2005;43(4):687–713.
- 5. Chen Z, Zhong P, Liu M, Ma Q, Si G. A novel integrated MADM method for design concept evaluation. Scientific Reports. 2022;12(1):15885. pmid:36151244
- 6. Akram M, Kahraman C, Zahid K. Group decision-making based on complex spherical fuzzy VIKOR approach. Knowledge-Based Systems. 2021:106793.
- 7. Deschrijver G. Arithmetic operators in interval-valued fuzzy set theory. Information Sciences. 2007;177(14):2906–24.
- 8. Xiao F. A distance measure for intuitionistic fuzzy sets and its application to pattern classification problems. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2019;51(6):3980–92.
- 9. Zhang P, Li T, Wang G, Luo C, Chen H, Zhang J, et al. Multi-source information fusion based on rough set theory: A review. Information Fusion. 2021;68:85–117.
- 10. Jiang T, Hu D, Yu X. Enhanced IT2FCM algorithm using object-based triangular fuzzy set modeling for remote-sensing clustering. Computers & Geosciences. 2018;118:14–26.
- 11. Kumar R, Dhiman G. A comparative study of fuzzy optimization through fuzzy number. International Journal of Modern Research. 2021;1(1):1–14.
- 12. Mendel JM. On computing the similarity of trapezoidal fuzzy sets using an Automated Area Method. Information Sciences. 2022;589:716–37.
- 13. Dutta B, Dao SD, Martinez L, Goh M. An evolutionary strategic weight manipulation approach for multi-attribute decision making: TOPSIS method. International Journal of Approximate Reasoning. 2021;129:64–83.
- 14.
Hwang C-L, Yoon K. Methods for Multiple Attribute Decision Making. In: Hwang C-L, Yoon K, editors. Multiple Attribute Decision Making: Methods and Applications A State-of-the-Art Survey. Berlin, Heidelberg: Springer Berlin Heidelberg; 1981. p. 58–191.
- 15. Chen Z, Zhong P, Liu M, Ma Q, Si G. An integrated expert weight determination method for design concept evaluation. Scientific Reports. 2022;12(1):6358. pmid:35428829
- 16. Zhang Z, Chen S-M, Wang C. Group decision making with incomplete intuitionistic multiplicative preference relations. Information Sciences. 2020;516:560–71.
- 17. Hua Z, Gong B, Xu X. A DS–AHP approach for multi-attribute decision making problem with incomplete information. Expert systems with applications. 2008;34(3):2221–7.
- 18. Capuano N, Chiclana F, Fujita H, Herrera-Viedma E, Loia V. Fuzzy group decision making with incomplete information guided by social influence. IEEE Transactions on Fuzzy Systems. 2017;26(3):1704–18.
- 19. Xu X-H, Wang B, Zhou Y-J. Method for large group decision making with incomplete decision preference information based on trust mechanism (in Chinese). Control and Decision. 2016;31(004):577–85.
- 20. Shinno H, Yoshioka H, Marpaung S, Hachiga S. Quantitative SWOT analysis on global competitiveness of machine tool industry. Journal of engineering design. 2006;17(03):251–8.
- 21. Shvetsova OA, Park SC, Lee JH. Application of quality function deployment for product design concept selection. Applied Sciences-Basel. 2021;11(6):2681.
- 22.
Lønmo L, Muller G, editors. 7.1. 2 Concept Selection‐Applying Pugh Matrices in the Subsea Processing Domain. INCOSE International Symposium; 2014: Wiley Online Library.
- 23.
Ulrich K, Eppinger S. Product design and manufacturing. The McGraw-Hill Inc; 2000.
- 24. Wang Y-M, Fan Z-P. Fuzzy preference relations: Aggregation and weight determination. Computers & Industrial Engineering. 2007;53(1):163–72.
- 25. Lin MC, Wang CC, Chen MS, Chang CA. Using AHP and TOPSIS approaches in customer-driven product design process. Computers in Industry. 2008;59(1):17–31.
- 26. Sayadi MK, Heydari M, Shahanaghi K. Extension of VIKOR method for decision making problem with interval numbers. Applied Mathematical Modelling. 2009;33(5):2257–62.
- 27. Shidpour H, Da Cunha C, Bernard A. Group multi-criteria design concept evaluation using combined rough set theory and fuzzy set theory. Expert Systems with Applications. 2016;64:633–44.
- 28. Liu S, Yang Y, Xie N, Forrest J. New progress of grey system theory in the new millennium. Grey Systems: Theory and Application. 2016.
- 29. Chen Z, Zhong P, Liu M, Sun H, Shang K. A novel hybrid approach for product concept evaluation based on rough numbers, shannon entropy and TOPSIS-PSI. Journal of Intelligent & Fuzzy Systems. 2021;40(6):12087–99.
- 30. Tiwari V, Jain PK, Tandon P. Product design concept evaluation using rough sets and VIKOR method. Advanced Engineering Informatics. 2016;30(1):16–25.
- 31. Zhai L-Y, Khoo L-P, Zhong Z-W. A rough set enhanced fuzzy approach to quality function deployment. International Journal of Advanced Manufacturing Technology. 2008;37(5–6):613–24.
- 32. Qi J, Hu J, Peng Y-H. Integrated rough VIKOR for customer-involved design concept evaluation combining with customers’ preferences and designers’ perceptions. Advanced Engineering Informatics. 2020;46.
- 33. Zhu G, Hu J, Ren H. A fuzzy rough number-based AHP-TOPSIS for design concept evaluation under uncertain environments. Applied Soft Computing. 2020;91:106228.
- 34. Chan L-K, Wu M-L. A systematic approach to quality function deployment with a full illustrative example. Omega. 2005;33(2):119–39.
- 35. Zhen Z, Junliang G, Yuan G, Wenyu Y. Two-sided matching decision making with multi-granular hesitant fuzzy linguistic term sets and incomplete criteria weight information. Expert Systems with Applications. 2020(prepublish).
- 36. Zhuolin L, Zhen Z, Wenyu Y. Consensus reaching with consistency control in group decision making with incomplete hesitant fuzzy linguistic preference relations. Computers & Industrial Engineering. 2022;170.
- 37. AmirHossein N, Amir J, Hadi N, Wenjun Y. Optimized score function and its application in group multiattribute decision making based on fuzzy neutrosophic sets. International Journal of Intelligent Systems. 2021;36(12).
- 38. Wang Z, Li KW, Wang W. An approach to multiattribute decision making with interval-valued intuitionistic fuzzy assessments and incomplete weights. Information Sciences. 2009;179(17):3026–40.
- 39. Wei G-W. GRA method for multiple attribute decision making with incomplete weight information in intuitionistic fuzzy setting. Knowledge-Based Systems. 2010;23(3):243–7.
- 40. Liu W-Q W Y, Lan Y-X. Intuitionistic fuzzy multi-attribute decision-making method based on evidential reasoning under incomplete information (in Chinese). Statistics and Decision. 2018;34(15):42–5.
- 41. Hong T-P, Tseng L-H, Wang S-L. Learning rules from incomplete training examples by rough sets. Expert Systems with Applications. 2002;22(4):285–93.
- 42. Xu Z. Group decision making method based on different types of incomplete judgment matrices (in Chinese). Control and Decision. 2006;01:28–33.
- 43. Ferrin DL, Dirks KT, Shah PP. Direct and indirect effects of third-party relationships on interpersonal trust. Journal of Applied Psychology. 2006;91(4):870. pmid:16834511
- 44. Koksalmis E, Kabak Ö. Deriving decision makers’ weights in group decision making: An overview of objective methods. Information Fusion. 2019;49:146–60.
- 45.
Torra V, Narukawa Y. Modeling decisions: information fusion and aggregation operators: Springer Science & Business Media; 2007.
- 46. Xu Z. An overview of methods for determining OWA weights. International journal of intelligent systems. 2005;20(8):843–65.
- 47. Xu D, Lv L, Ren X, Ren J, Dong L. Route selection for low-carbon ammonia production: A sustainability prioritization framework based-on the combined weights and projection ranking by similarity to referencing vector method. Journal of Cleaner Production. 2018;193.
- 48.
Saaty TL. What is the analytic hierarchy process? Mathematical models for decision support: Springer; 1988. p. 109–21.
- 49. Chen P. Effects of normalization on the entropy-based TOPSIS method. Expert Systems with Applications. 2019;136:33–41.
- 50. Song W, Ming X, Wu Z. An integrated rough number-based approach to design concept evaluation under subjective environments. Journal of Engineering Design. 2013;24(5):320–41.
- 51. Zhang Z, Li Z. Consensus-based TOPSIS-Sort-B for multi-criteria sorting in the context of group decision-making. Annals of Operations Research. 2022:1–28.
- 52. Gai T, Cao M, Chiclana F, Zhang Z, Dong Y, Herrera-Viedma E, et al. Consensus-trust driven bidirectional feedback mechanism for improving consensus in social network large-group decision making. Group Decision and Negotiation. 2022:1–30.