Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A novel financial risk assessment model for companies based on heterogeneous information and aggregated historical data

  • Dan-Ping Li,

    Roles Data curation, Software, Writing – original draft

    Affiliation School of Business, Hunan University of Science and Technology, Xiangtan, China

  • Si-Jie Cheng,

    Roles Data curation, Investigation, Methodology, Resources, Software, Validation, Writing – review & editing

    Affiliation School of Geosciences and Info-Physics, Central South University, Changsha, China

  • Peng-Fei Cheng ,

    Roles Conceptualization, Funding acquisition, Investigation, Project administration, Supervision, Writing – original draft, Writing – review & editing

    1180033@hnust.edu.cn

    Affiliations School of Business, Hunan University of Science and Technology, Xiangtan, China, Hunan Engineering Research Center for Intelligent Decision Making and Big Data on Industrial Development, Xiangtan, China

  • Jian-Qiang Wang,

    Roles Funding acquisition, Investigation, Methodology, Project administration

    Affiliations Hunan Engineering Research Center for Intelligent Decision Making and Big Data on Industrial Development, Xiangtan, China, School of Business, Central South University, Changsha, China

  • Hong-Yu Zhang

    Roles Formal analysis, Investigation, Validation

    Affiliations Hunan Engineering Research Center for Intelligent Decision Making and Big Data on Industrial Development, Xiangtan, China, School of Business, Central South University, Changsha, China

Abstract

The financial risk not only affects the development of the company itself, but also affects the economic development of the whole society; therefore, the financial risk assessment of company is an important part. At present, numerous methods of financial risk assessment have been researched by scholars. However, most of the extant methods neither integrated fuzzy sets with quantitative analysis, nor took into account the historical data of the past few years. To settle these defects, this paper proposes a novel financial risk assessment model for companies based on heterogeneous multiple-criteria decision-making (MCDM) and historical data. Subjective and objective indexes are comprehensively taken into consideration in the financial risk assessment index system of the model, which combines fuzzy theory with quantitative data analysis. Moreover, the assessment information obtained from historical financial information of company, credit rating agency and decision makers, including crisp numbers, triangular fuzzy numbers and neutrosophic numbers. Furthermore, the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method is used to determine the ranking order of companies according to their financial risk. Finally, an empirical study of financial risk assessment for companies is conducted, and the results of comparative analysis and sensitivity analysis suggest that the proposed model can effectively and reliably obtain the company with the lowest financial risk.

Introduction

Financial risk involves a combination of different methods, models and approaches to reduce the likelihood of a threat and the extent of losses [1]. Financial analysis can help to companies to detect financial risks in advance, take appropriate actions to minimize the losses, and support better decision-making [2, 3]. An accurate understanding and a well assessment of financial risk would have lots of positive consequences such as reduction of insolvency, reduction of bankruptcy rate, reduction of financial hardship. Therefore, the establishment of financial risk assessment model, the early diagnosis of the financial crisis and take appropriate measures to maintain health and safety and sustainable development of enterprises, it is very important [4]. Consequently, it’s necessary to study and develop an appropriate approach to assess financial risk of companies.

In the field of financial risk assessment, although some remarkable achievements have been made, there are still three shortcomings. First, the quantitative and qualitative analysis which adopts the combination of fuzzy theory and data analysis have not been used in financial risk assessment. Second, the historical data over several years have not been considered. Third, the information is used partially in financial risk assessment. The financial risk can be assessed by historical financial information of company, credit rating agency and decision makers. Hence, fuzziness and accuracy are existed simultaneously in the assessment information. The existing approaches only considered the data information of company, which may lead to information loss [5]. Therefore, in order to overcome these shortcomings, a novel financial risk assessment model for companies needs to be studied. To sum up, the motivations of this article are as follows:

  1. The assessment of financial risk involves quantitative and qualitative indexes. Some scholars have employed objective financial indexes to assess financial risk quantitatively [69]. Subjective indexes such as controlling system of financial risk have not been utilized in extant study. Thus, the subjective indexes combined with objective indexes are employed in the index system of the proposed financial risk assessment model.
  2. With respect to the partial use of information in assessment [10], it is appropriate to apply historical financial information and fuzzy theory to describe assessment information about financial risk for companies. The assessment information from historical financial information of company mainly involves crisp numbers. Wang et al. [11] presented that triangular fuzzy numbers can reflect the uncertainty of objective things and the fuzziness of human thought. Thus, it can be used to improve the objectivity and accuracy of the description of credit rating. Zhang et al. [12] presented that a neutrosophic set is an effective tool for reflecting the fuzziness in text evaluation because the evaluation information from decision makers is text information that represents sentiment values, and every sentiment value has not only a certain degree of truth, but also a falsity degree and an indeterminacy degree [13]. Thus, it needs to transform sentiment values into neutrosophic numbers with positive, medium, and passive values. For example, when asked to assess whether controlling system of financial risk would be “good”, from the sentiment value of a decision maker, we may deduce that the membership degree of truth is 0.8, the membership degree of indeterminacy is 0.1, and the membership degree of falsity is 0.1. Therefore, assessment information, including crisp numbers, triangular fuzzy numbers and neutrosophic numbers, needs to be taken into account in the financial risk assessment model.
  3. In order to deal with the ranking order of companies according to their financial risk based on historical data and heterogeneous MCDM, a systematic approach need to be employed in the proposed model. Shih et al. [14] pointed out that TOPSIS is a practical and useful technique for the ranking and selection of a number of externally determined alternatives through distance measures, and it has been applied in multiple-criteria decision-making (MCDM) [15]. Lourenzutti et al. [16] and Li et al. [17] proposed heterogeneous TOPSIS for multi-criteria decision making method. Therefore, the TOPSIS method is used to obtain the ranking order of companies in the financial risk assessment model.

In this paper, a novel financial risk assessment model is developed to help managers assess company’s financial risk by utilizing TOPSIS according to above discussion, which is based on MCDM and heterogeneous information including qualitative data and non-qualitative data. The contributions of this paper are concluded as three aspects. The first one is the establishment of an improved financial risk index system with comprehensive consideration of subjective and objective indexes, which combines fuzzy theory with quantitative data analysis. The second is the consideration of the impact of historical financial position on current financial risk analysis by aggregating historical data over several years which assesses financial risk accurately. The third is the application of heterogeneous information obtained from historical financial information, credit rating and decision makers, including crisp numbers, triangular fuzzy numbers and neutrosophic numbers. The final contribution is that the TOPSIS method for heterogeneous multi-criteria decision-making is employed to get the ranking order of companies based on their financial risk, which can help manager to assess financial risk.

The rest of the paper is organized as follows. In Section 2, previous researches about financial risk assessment are introduced briefly. A brief introduction about research methodology including TOPSIS method, exponential smoothing method, neutrosophic number and triangular fuzzy number is presented in Section 3. Subsequently, a novel financial risk assessment model is developed based on heterogeneous MCDM in Section 4. In Section 5, an empirical study is presented concretely, and the effectiveness of the proposed method is verified by a comparative analysis. Finally, Section 6 summarizes the paper and proposes some directions for future research.

Literature review

Among the studies of financial risk of listing companies, numerous scholars have made great contributions. Some scholars evaluated financial risk by utilizing quantitative analysis. For example, the utility functions that classify the considered alternatives into predefined risk classes were developed in the study by Doumpos and Zopounidis [6]. It proposed the multi-group hierarchical discrimination method (M.H.DIS) that classified countries into four groups like c1, c2, c3, c4 from good to bad. Lee et al. [7] evaluated financial positions of shipping companies using entropy and grey relation analysis. Stochastic frontier analysis (SFA) was illustrated in the study proposed by Wang et al. [18], which calculated efficiency estimation of risk indicators with determined influence factors of risk assessment indicators computed by panel frontier model. Financial ratios of capital structure risk, liquidity risk and insolvency risk studied by balance sheet, statement of income, expenses and cash flow of dozens of businesses were described to assess financial risk in Kociu et al. [8]. Furthermore, many fuzzy theory and MCDM method have been applied in business [19] and risk assessment. Sabokbar et al. [20], Mardani et al. [21] and Ribeiro et al. [22] utilized fuzzy set to assess risk. Kochanek and Tynan [23] adopted linguistic label to present uncertainty of risk. Chang et al. [24] adopted the Fuzzy Analytic Network Process (FANP) method to assess ERP implementation risks. Gonçalves et al. [25] employed the Interactive Multiple Criteria Decision Making (TODIM) approach to analyze credit risk. Kou et al. [26] presented an MCDM to analyze financial risk. Shaverdi et al. [27] ranked companies by using fuzzy AHP and fuzzy TOPSIS comparatively to evaluate financial performance. In conclusion, in the extant researches, fuzzy theory and quantitative analysis have not been employed simultaneously to assess financial risk.

With respect to financial risk index system, numerous assessment indexes have been researched. Cui et al. [28] established a financial evaluation index system of Chinese listing companies with four financial risk criteria and corresponding objective numerical indexes, including financing risk, investment risk, income distribution risk and cash flow at risk. The example for country risk assessment conducted by Doumpos and Zopounidis [6] used twelve economic indicators like import and export volume growth and GNP growth as risk evaluation indexes. Jurczyk et al. [29] quantified systemic risks by numerous stock indexes, and without non-numerical indexes. Wang and Liu [30] evaluated the real estate investment risk by qualitatively analyzing financing risk and investment site risk, etc. Gonçalves et al. [25] used fuzzy theory to analyze credit risk. In view of the above-mentioned review, a financial risk index system including quantitative and qualitative index needs to be researched.

As studied in numerous risk evaluation researches, existing approaches used partial information. Wang and Liu [30] utilized crisp numbers to evaluate financing risk qualitatively. Kiliçman and Sivalingam [31] used triangular fuzzy numbers to represent return rates, etc. Kochanek and Tynan [23] adopted linguistic label to present uncertainty of risk. And every linguistic value has not only a certain degree of truth, but also a falsity degree and an indeterminacy degree; it needs to transform sentiment values into neutrosophic numbers with positive, medium, and passive values. None of the previous risk evaluation research considered these information types mentioned above simultaneously. Thus, in this paper, the evaluation information obtained from historical financial information, credit rating agency and decision makers includes crisp numbers, triangular fuzzy numbers and neutrosophic numbers, which need to be considered in the assessment progress.

In current methods of subjective weight, numerous scholars have made notable contributions. Wang et al. [32] used criteria priorities to compute criteria weights. Zhao et al. [33] calculated weight by using a probabilistic method. Mangla et al. [34] used fuzzy analytic hierarchy process (AHP) to get subjective weight or provided by decision-makers. Rezaei [35, 36] utilized the (best-worst method) BWM method to calculate subjective weight with lesser comparison times and information loss compared with AHP. Tian et al. [37] presented that the BWM method can require fewer pairwise comparisons than does fuzzy AHP but obtain more highly reliable weights. Therefore, the subjective weight of financial criteria in the proposed model is calculated by BWM.

In conclusion, previous researches about financial risk evaluation should be modified in future study. In order to settle these issues based on the above discussion, we (1) establish a novel financial risk index system combining fuzzy theory with quantitative analysis, (2) consider various types of information including crisp numbers, triangular fuzzy numbers and neutrosophic numbers, (3) utilize BWM method to calculate the subjective weight of financial risk criteria, (4) utilize TOPSIS method to manage heterogeneous information and obtain the ranking order of companies according to their financial risk.

Research methodology

In this section, the specific and processes of TOPSIS method, the concepts, definitions and algorithms of neutrosophic set and triangular fuzzy number are introduced.

TOPSIS method

The TOPSIS method, an MCDM method, was proposed by Hwang and Yoon in 1981 [38]. It provides the best alternative which is as close as possible to the best solution. Dozens of scholars have applied TOPSIS to solve simple or complex problems in different areas [39], e.g., weapon selection, alternative evaluation and risk assessment [4042]. The procedure of the TOPSIS method can be described as shown in the following steps [16]:

  1. Step 1. Define and normalize the decision matrix R = (rij).
  2. Step 2. Aggregate the weights to the decision matrix by making vij = wjrij.
  3. Step 3. Define the positive ideal solution (PIS), , and the negative ideal solution (NIS), , for each criterion. Usually, and for benefit criteria, and and for cost criteria.
  4. Step 4. Calculate the separation measures for each alternative. (1) (2)
  5. Step 5. Calculate the closeness coefficients to the ideal solution for each alternative. (3)
  6. Step 6. Rank the alternatives according to CCi. The bigger CCi is, the better alternative Ai will be.

Exponential smoothing method

First, the trend changes of time series can be distinguished based on data visualization by using professional software. Then, the procedure of the method is given as follows: (4) Let is weighting coefficient which represents the weight of latest data (the bigger is, the more important the new data is). St+1 is the (t+1)th prediction value, Xt is the t phase data. The most important aspect of exponential smoothing is the solution of and original value, as follows.

If the time series is stable comparatively, the selection of is a lower value as (0.1–0.3); on the contrary, it may bigger like (0.6–0.8). Different value is selected subjectively, count mean absolute error (MAE) with different like formula (4), the value is best which minimize the error. For si is predicted value, xi is true value, the solution is depicted as formula (5).

(5)

Let S0 is the original value; formula (6) can describe the definite way of S0. When t < 20, in general, S0 is the mean value of three years’ true data initially, in this article we elicit two stages.

(6)

Neutrosophic set theory

Definition 1. [43] Let X be a space of points (objects), with a generic element in X denoted by x. Then an NS A in X is characterized by three membership functions, including a truth-membership function TA(x), indeterminacy-membership function IA(x), and falsity-membership function FA(x), and is defined as‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬‬ A =x, TA(x), IA(x), FA(x) ˃ | xX}, where TA(x), IA(x), and FA(x) are real standard or nonstandard subsets of ]-0, 1+[, i.e. TA(x): X →]-0, 1+[, IA(x): X →]-0, 1+[, FA(x): X →]-0, 1+[. The sum of TA(x), IA(x), and FA(x) is unrestricted, and -0TA(x)+IA(x)+ FA(x) ≤ 3+.

Definition 2. [44] Let X be a universal space of points (objects), with a generic element of X denoted by x. A single-valued neutrosophic set (SVNS) is characterized by a truth membership , an indeterminacy-membership function and a falsity-membership function with , , for all xX. The sum of three membership functions of a SVNS , the relation for all xX holds good.

Definition 3. (Euclidean distance) [45] Let and be two SVNSs for xiX (i = 1, 2, …, n). Then the Euclidean distance between two SVNSs and can be defined as follows: (7)

Definition 4. According to the study by Majumdar and Samanta [45], single-valued neutrosophic set A = {˂x, TA(x), IA(x), FA(x)˃|| xX}, an entropy on neutrosophic set A is computed as formula (8).

(8)

Definition 5. The entropy weight of neutrosophic set in the study proposed by Tan et al. [46] is as follows: (9)

Definition 6. The single valued neutrosophic weighted averaging (SVNWA) aggregation operator proposed by Ye’s study [47] is as follows: (10) where Ψ = (Ψ1,Ψ2,…,Ψn)T is the weight vector of Ai.

Definition 7. Suppose that S = {si|i = −t,…,t} is a limited and ordered discrete label set [48]. In this system, we let t = 3, si represents a possible linguistic term. The specific label set could be: S = {s-3 = very bad, s-2 = bad, s-1 = slightly bad, s0 = ok, s1 = slightly good, s2 = good, s3 = very good}. The semantic values proposed in Table 1 are calculated throughout sentiment analysis by using the software of ‘The R Project for Statistical Computing’. According to different sentiment word, we allocate linguistic variable si to positive, neutral and passive value as a neutrosophic number Ai = <Ti, Ii, Fi>, i = 1, …, n, (positive value is T value, neutral value is I, passive value is F). The T-value is the average value of positive values, while the I-value is 1 if s0 exists or 0 if s0 not exists, the F-value is mean of the absolute passive values. For example, a set S = {s−1, s0, s1, s2}, the neutrosophic value is , i.e. .

Triangular fuzzy number

Definition 8. A fuzzy set is a triangular fuzzy number (TrFN) [49], denoted by a = (a1, a2, a3), if it is defined on the real line with membership function given by: (11) where a1 < a2 < a3, a1 and a2 stand for the lower and upper values of the support of , respectively, and a2 for the modal value.

Definition 9. Let and be two TrFN. A distance measure between and is given in the study by Dağdeviren et al. [40]: (12)

Definition 10. A triangular fuzzy number is denoted by , an interval value is denoted by l = [a,b]. Laarhoven and Pedrycz [50] indicated a1 and a3 is the upper limit value and lower limit value of triangular fuzzy number, and a2 is (a1 + a3)/2 when the TrFN is symmetrical. The lower value a1 and the upper value a3 are determined by standardized a and b respectively based on interval value [a, b]. Based on the definition of triangular fuzzy number, the value of a2 whose membership is 1 is the median value like in the situation of the shape of its membership function is an isosceles triangle. However, in the situation of irregular triangle, the value of a2 whose membership is 1 can be computed by following formula (13): (13) where λ represents attitude of evaluator. Value of λ = 1 indicates supportive attitude, λ = 0.5 indicates neutral attitude and λ = 0 indicates opposing attitude. As the evaluation of credit rating is determined by credit rating agency’s comprehensive assessment, we calculate a2 with λ = 0.5 i.e. , the membership shape of this TrFN is an isosceles triangle. Then, the triangular fuzzy number can be determined by standardized interval value, for example, [70,80] can be transformed into triangular fuzzy number (0.7,0.75,0.8).

Financial risk assessment model

The proposed financial risk assessment model consists five parts, as depicted in S1 Fig. The first part is the establishment of financial risk index system, the important criteria as well as sub-criteria are determined from literature reviews and experts. The second part is the calculation of subjective criteria weight by using BWM method. Moreover, the evaluation information from historical financial information of companies, credit rating agency and decision makers can be obtained. And the evaluation matrix is established including crisp numbers, triangular fuzzy numbers and neutrosophic numbers. In addition, with the computation of objective entropy weight of financial index, the comprehensive index weight can be calculated by multiplying entropy weight and subjective criteria weight. Finally, the ranking order of companies according to financial risk is derived utilizing TOPSIS method based on heterogeneous MCDM. The specific details of this novel model will be described in the rest of this section.

The establishment of the financial risk index system

The assessment of financial risk involves financial condition of company, credit rating and evaluation of decision maker, and it is very complicated. Based on the discussion in the literature review, financial risk can be mainly evaluated from four criteria, which are financing risk, investment risk, income distribution risk and cash flow at risk, denoted as Ai(i = 1, 2, 3, 4) respectively. Moreover, every criterion can be divided into multiple sub-criteria (i.e. financial indexes). Based on the definition of financial risk and previous studies analyzed in the literature review, fuzzy information is taken into account in the financial risk assessment model. Generally, the credit rating indicates the capacity of financing; the contractual capacity of partners represents the fund risk; and the management system of financial risk indicates the risk management ability of company. Therefore, the credit rating index is added to financing risk criterion, and the contractual capacity of partner index and the controlling system of financial risk index are added to investment risk criterion. Hence, an improved risk index system is established as shown in S2 Fig. Because of the complexity of financial condition and the uncertainty of information, the assessment values of financial risk indexes can be divided into multiple types. Therefore, heterogeneous information including crisp numbers, triangular fuzzy numbers and neutrosophic numbers exists in this proposed financial risk index system. The financial risk index system including the definition of the financial risk indexes is established in Table 2.

The estimation of criteria weights with BWM

According to the discussion in the literature review, a more efficient method (i.e. BWM method) is used to calculate the subjective weight in this section. The detailed steps of BWM to compute the weights of the four financial risk criteria are summarized as follows [35].

  1. Step 1. Determine a set of decision criteria.
    In this step, we consider the criteria {c1, c2, …, cn} that should be used to arrive at a decision.
  2. Step 2. Determine the best (e.g. most desirable, most important) and the worst (e.g. least desirable, least important) criterion.
  3. Step 3. Determine the preference of the best criterion over all the other criteria, using a number between 1 and 9. The resulting best-to-others vector would be: AB = (aB1, aB2, …, aBn) where aBj indicates the preference of the best criterion B over criterion j. It is clear that aBB = 1.
  4. Step 4. Determine the preference of all the criteria over the worst criterion, using a number between 1 and 9. The resulting others-to-worst vector would be: AW = (a1W, a2W, …, anW) where ajW indicates the preference of the criterion j over the worst criterion W. It is clear that aWW = 1.
  5. Step 5. Find the optimal weights .

The optimal weight for the criteria is the one where, for each pair of wB / wj and wj / wW, we have wB / wj = aBj and wj / wW = ajW. To satisfy these conditions for all j, we should find a solution where the maximum absolute differences and for all j is minimized. Considering the non-negativity and condition for the weights, the following problem is resulted: (14)

The formula (14) is equivalent to the following formula: (15) Solving problem (15), the optimal weights and ξ* are obtained.

Then calculate the consistency ratio, using ξ*and the corresponding consistency index, as follows: (16)

Table 3 shows the maximum values of ξ (consistency index) for different values of aBW [35].

If consistency ratio ≤ 0.1, it implies a very good consistency which is acceptable. Otherwise we can revise aBj and ajW to make the solution (more) consistent.

BWM introduced above is limited to derive unique optimum weight vector when the number of criteria is more than three. It might leads to multiple optimal solutions. The improved method present in [36] is used to obtain optimal weights with n criteria. If we use {|wBaBjwj|, |wjajwww|} instead of , the problem can be solved as follows.

(17)

The formula (17) can be transferred to the following linear programming problem: (18) Formula (18) is a linear problem, which can calculate the only optimal weights . Hence we can compute the weight vector (w1, w2, w3, w4) of financing risk A1, investment risk A2, income distribution risk A3 and cash flow at risk A4.

The evaluation matrix of financial risk

The financial risk evaluation of company involves qualitative and quantitative indicators such as financial status, credit rating, decision makers, etc. Different decision makers may make different assessment based on their distinct knowledge and different judgment standards. Therefore, in this section, the evaluation information determined by historical financial information, credit rating and decision makers is heterogeneous, including crisp numbers, interval numbers and linguistic labels. Specifically, the crisp numbers are the evaluation values of asset liability ratio, current ratio, quick ratio, number of times interest earned, main business cost ratio, operating expense ratio, main business revenue growth rate, total asset growth rate, net assets yield, net profit growth rate, shareholder’s equity growth rate, equity ratio, retention ratio, cash debt coverage ratio, cash ratio and security surplus cash multiples; the interval numbers are the evaluations of credit rating; the linguistic labels are the evaluation values of contractual capacity of partner and controlling system of financial risk. Because of the uncertainty information, the interval numbers provided by credit rating agency can be transformed into triangular fuzzy numbers, and the linguistic labels obtained by decision makers can be transformed into neutrosophic numbers. Therefore we can get the evaluation matrix R = (rij) with crisp numbers, triangular fuzzy numbers and neutrosophic numbers.

  1. Step 1. Aggregate financial data over several years.

The evaluation values of financial risk indexes such as asset liability ratio, current ratio, quick ratio, number of times interest earned, main business cost ratio, operating expense ratio, main business revenue growth rate, total asset growth rate, net assets yield, net profit growth rate, shareholder’s equity growth rate, equity ratio, retention ratio, cash debt coverage ratio, cash ratio and security surplus cash multiples are obtained by financial data over the years of company. In consideration of that the financial risk of a company influenced by historical financial condition, the historical data and current data which reflect development trend should be taken into account. In this section, according to the method introduced in Section 3.2, we aggregate financial data over the years by using exponential smoothing method through EViews software to get scientific and reasonable evaluation of financial risk. Then, the evaluation matrix of R = (rij) based on a11, a12, a13, a14, a21, a22, a23, a24, a25, a26, a31, a32, a33, a41, a42, a43 is computed.

  1. Step 2. Obtain the depiction of credit rating.

The evaluation information of credit rating is determined by credit rating agency. Because of the uncertainty and fuzziness of credit rating, it should be transformed into triangular fuzzy numbers. According to the method mentioned in Section 3.4, the relative descriptions and function of crediting rating can be conducted as seen in Table 4 and S3 Fig. Therefore, the evaluation matrix of R = (rij) based on a15 is calculated.

  1. Step 3. Evaluate contractual capacity of partner and financial risk control system.

The fundamental thesis of neutrosophy presented in the study by Rivieccio [13] is that every idea has not only a certain degree of truth, as is generally assumed in many-valued logic contexts, but also a falsity degree and an indeterminacy degree that have to be considered independently from each other. As mentioned in [51] and [52], they can deal with consistent, hesitant, and inconsistent information at the same time, and benefit the management of the evaluation information mentioned in [53]. The evaluation values of contractual capacity of partner and financial risk control system are linguistic values determined by decision makers, so it must first be transformed into neutrosophic numbers with positive, medium and passive values [54]. In this section, we transform the linguistic evaluation information of contractual capacity of partner and financial risk control system according to the symmetric linguistic evaluation scale into neutrosophic numbers with truth, indeterminacy and falsity. Then, we can aggregate the neutrosophic numbers using single valued neutrosophic weighted averaging (SVNWA) aggregation operator described by formula (10). Therefore, the evaluation matrix of R = (rij) based on a27 and a28 is calculated eventually, where Ψ = (Ψ1, Ψ2, …, Ψp)T is the weight vector of decision makers corresponding to these indexes.

The calculation of index weight

In this section, the principle of the combination between subjectivity and objectivity is applied in the calculation of the index weight. First, we compute the entropy weights of financial risk indexes. Then, we can get comprehensive index weight which combines entropy weight of index with subjective weight of financial risk criteria.

  1. Step 1. Determine the entropy weight of data index.

Normalizing the evaluation matrix in formula (19), and further normalization matrix R = (rij)p*q is (p evaluation indicators, q evaluated objects). The normalization formula is as follows: (19)

The entropy weight can be defined as: (20) Hence, the entropy weight E11, E12, E13, E14, E21, E22, E23, E24, E25, E26, E31, E32, E33, E41, E42 and E43 can be calculated.

  1. Step 2. Compute the entropy weight of credit rating index.

To normalize the TrFN, we use the following formula: (21) Hence, the entropy weight E15 can be calculated by using formula (22).

(22)
  1. Step 3. Compute the entropy weight of contractual capacity of partner and controlling system of financial risk.

The following formula is used to normalize the neutrosophic numbers: (23)

The formula (8) and formula (9) in Section 3.3 is used to calculate entropy weight based on evaluation matrix. Hence, the entropy weight E27 and E28 can be computed.

  1. Step 4. Calculate financial risk index weight.

According to the explanation of BWM in Section 4.2, the optimal weight vector of financial risk criteria is . The synthetic weight of financial risk index is calculated as the following way: (24)

The ranking order of companies according to financial risk by using TOPSIS

Suppose there are n alternatives xj (j = 1, …, n), thus the sets of alternatives (i.e. companies) can be denoted by X = {x1, x2, …, xn}. The TOPSIS method based on heterogeneous MCDM is used to solve the ranking order of companies according to financial risk. Because of the existence of heterogeneous evaluation information of financial risk, the criteria set A = (A1, A2, A3, A4) can be divided into three subsets Oi (i = 1,2,3), where Oi are sets of criteria whose values are crisp numbers, triangular fuzzy numbers and neutrosophic numbers. The procedure of this method is summarized [17] as follows:

  1. Step 1. Normalize evaluation matrix R.

The normalized evaluation value has already been solved at the time of entropy weight of risk index calculating. So we can get the normalized evaluation matrix R = (rij) directly.

  1. Step 2. Define the positive ideal solution (PIS) and the negative ideal solution (NIS) for each index.

Let y+ represents the PIS, y- represents the NIS, where (25) Here, or ; or ; or .

Similarly, the NIS y- is as follows: (26) Here, or ; or ; or .

  1. Step 3. Compute the separation measures between each company and the PIS as well as the NIS.

The distance between the normalized values of the company xj (j = 1, 2, …, n) and PIS x+ on all index aio1 is defined as follows. (27) where and are the normalized value vectors of the company xj and the PIS x+ on all indexes in O1, respectively.

The distance between the normalized values of the company xj (j = 1, 2, …, n) and PIS x+ on all indexes aio2 is described using formula (12) as follows. (28) where and are the normalized value vectors of the company xj and the PIS x+ on all indexes in O2, respectively.

The distance between the normalized values of the company xj (j = 1, 2, …, n) and PIS x+ on all indexes aio3 is described using formula (7) as follows. (29) where and are the normalized value vectors of the company xj and the PIS x+ on all indexes in O3, respectively.

According to formula (27), (28), (29), the distance between a company xj (j = 1, 2, …, n) and the PIS x+ is defined as follows: (30)

In the same way, the distance between the normalized values of the company xj (j = 1, 2, …, n) and NIS x- on all indexes aio1 is defined as follows: (31) where is the normalized value vector of the NIS x- on all indices in O1.

The distance between the normalized values of the company xj (j = 1, 2, …, n) and NIS x- on all indices aio2 is expressed as follows: (32) Where is the normalized value vector of the NIS x- on all indices in O2.

The distance between the normalized values of the company xj (j = 1, 2, …, n) and NIS x- on all indexes aio3 is introduced as follows. (33) where is the normalized value vector of the NIS x- on all indices in O3.

Using the formula (31), (32), (33), the distance between a company xj (j = 1, 2, …, n) and the NIS x- is defined as follows: (34)

  1. Step 4. Calculate relative closeness degree of companies to the PIS.
(35)
  1. Step 5. Rank the companies according to τi.

The bigger τi is, the better company xj will be.

Empirical study

Background and data collection

At present, biological medicine is one of the most important emerging industries in China. And the financial risk assessment is conducive to the risk control and healthy development of pharmaceutical companies. In this section, we selected three companies from Chinese A-share pharmaceutical manufacturing listed companies randomly and conducted an empirical study in order to verify the effectiveness of the proposed model. The stock codes of the three companies are 600196, 600664 and 600085, denoted by A, B, C respectively.

The original financial data and related information can be collected from the website http://www.qianzhan.com or credit rating agency, and the subjective information can be obtained from supervisors of company through questionnaire surveys. In the study, supervisors from the three companies were invited to participate in a questionnaire to obtain linguistic assessment. The raw data on the operation and technology of the three companies from 2010 to 2016, the original evaluation information of credit rating, contractual capacity of partner and financial risk control system were collected, as supporting information; see S1 Raw Data.

According to the financial risk assessment model proposed in Section 4, we compute financial risk criteria weight and get the evaluation matrix through historical financial information of company, credit rating agency and decision makers. Then, the synthetic weight of financial risk index is computed by multiplying criteria weight with the entropy weight of risk index. Finally, the ranking order of the three companies according to their financial risk can be calculated using TOPSIS method based on heterogeneous MCDM. In addition, the effectiveness and reliability of the proposed financial risk assessment model are verified by comparative analysis and sensitivity analysis.

Financial risk criteria weight

According to the BWM method introduced in Section 4.2, we calculate the subjective weight of financial risk criteria. Among the financial risk criteria denoted by A1 to A4, listed in Table 2, financing risk (A1) is the most important criterion, and income distribution risk (A3) is the least important criterion, it is determined by experts. The pairwise comparison vector of the most important and least important criteria can be described in Tables 5 and 6. Table 5 indicates that the preference values of the most important criterion (A1) over criterion (A2), criterion (A3) and criterion (A4) are 3, 8 and 6 respectively. The preference values of the criterion (A1), (A2) and (A4) over the least important criterion (A3) are 8, 7 and 5, respectively, which are shown in Table 6. Therefore, the weight vector of criteria is computed.

thumbnail
Table 5. The pairwise comparison vector of the most important criterion.

https://doi.org/10.1371/journal.pone.0208166.t005

thumbnail
Table 6. The pairwise comparison vector of the least important criterion.

https://doi.org/10.1371/journal.pone.0208166.t006

From formula (17) and (18) illustrated in Rezaei’s study [36], we can get w1* = 0.3809, w2* = 0.3334, w3* = 0.0476, w4* = 0.2381, and ξL* = 0. Based on the proposed method, ξL* indicates consistency index directly without extra computation. As ξL* = 0, we can obtain complete consistency. Thus, the subjective criteria weight is w* = (0.3809, 0.3334, 0.0476, 0.2381).

Evaluation matrix

According to the evaluation method proposed in Section 4.3, the evaluation matrix determined by historical financial information of company, credit rating agency and decision makers is obtained, and the information is heterogeneous, including crisp numbers, interval numbers and linguistic labels.

Firstly, the evaluation matrix of numerical index is calculated by aggregating the historical financial data of 2010–2016, according to the exponential smoothing forecasting method introduced in Section 4.3. And the final evaluation values of financial risk on quantitative indices are listed in Table 7. The data visualization is depicted as shown in S4S7 Figs.

thumbnail
Table 7. Evaluation values of financial risk on quantitative indices.

https://doi.org/10.1371/journal.pone.0208166.t007

Secondly, we compute the evaluation value of credit rating.

According to the evaluation method of credit rating introduced in Section 4.3, we translate the evaluation information of credit rating into triangular fuzzy numbers transformed by membership function, as shown in Table 8.

Thirdly, we get the assessment value of contractual capacity of partner and financial risk control system.

Considering the deficiency of practical information and the efficiency of neutrosophic set mentioned in [55], we transform decision makers’ linguistic labels into neutrosophic numbers by using the method introduced in Section 4.3, and aggregate the neutrosophic numbers using SVNWA operator described in formula (10). The description of evaluation values is shown in Table 9.

thumbnail
Table 9. Evaluation values of financial risk by decision makers.

https://doi.org/10.1371/journal.pone.0208166.t009

Therefore, the evaluation matrix R = (rij) can be determined directly, as shown in Tables 7, 8 and 9.

Weight of financial risk index

The weight of the indexes can be calculated by using the entropy weight method introduced in Section 4.4. According to the normalization method, the normalized evaluation matrix is described in Table 10.

The entropy weight is computed by formula (20), formula (22) and formula (9), the result is obtained as E1 = (E11, E12, E13, E14, E15) = (0.196258983, 0.193583809, 0.265492452, 0.138674702, 0.20599005); E2 = (E21, E22, E23, E24, E25, E26, E27, E28) = (0.085170054, 0.088203688, 0.085201275, 0.086354441, 0.227198836, 0.08753941, 0.177338538, 0.162993756); E3 = (E31, E32, E33) = (0.296810103, 0.261347451, 0.441842446); E4 = (E41, E42, E43) = (0.525714023, 0.240714367, 0.233571609). The eventual weights of financial risk indexes are calculated by synthesizing subjective and objective weights, which is calculated by formula (24). The result is described as E1 = (E11, E12, E13, E14, E15) = (0.074755047, 0.073736073, 0.101126075, 0.052821194, 0.078461612); E2 = (E21, E22, E23, E24, E25, E26, E27, E28) = (0.028395696, 0.02940711, 0.028406105, 0.028790571, 0.075748092, 0.029185639, 0.059124669, 0.054342118); E3 = (E31, E32, E33) = (0.014128161, 0.012440139, 0.0210317); E4 = (E41, E42, E43) = (0.125172509, 0.057314091, 0.0556134). Thus, according to the description in Section 4.4, the weight of the attributes and indexes are obtained, as in Table 11.

Ordering result of TOPSIS method

Based on the method illustrated in Section 4.5, the degrees of similarity with respect to the PIS are calculated for the three companies x1, x2, x3 as follows:

It is easy to conclude the following ranking order of the three companies: x3x1x2. Therefore, the company with the lowest financial risk is x3, i.e. 600085.

Comparison analysis and discussion

As described in Section 4, the proposed model can be used to assess the financial risk of company considering historical financial information, credit rating, the conditions of partners and risk control, and heterogeneous information. To validate that the proposed model can effectively and reliably identify which company has the lowest financial risk, a comparative analysis is made with heterogeneous TODIM (an acronym in Portuguese of interactive and multi-criteria decision making) [56] and heterogeneous VIKOR (VlseKriterijumska Optimizacija I Kompromisno Resenje) [57] in the empirical study. Table 12 shows the ranking order of the three companies as obtained using these methods. Based on Table 12, the ranking order calculated by the proposed hybrid assessment model are the same as those computed by heterogeneous TODIM method and heterogeneous VIKOR method, so the effectiveness of the model is proved. Compared with other assessment methods of financial risk for companies, the advantages of the proposed model in the paper can be generalized as the following:

  1. The proposed model considers both subjective and objective indexes in the financial risk index system, and combines fuzzy theory with quantitative data analysis. Thus effectively ensuring that the financial risk assessment for companies can be more in line with reality.
  2. The evaluation information is evaluated from historical financial data of the company, credit rating agency and decision-makers, including crisp numbers, triangular fuzzy numbers and neutrosophic numbers. So that the financial risk assessment model is more accurate and reliable.
  3. In the proposed model, TOPSIS method is used to determine the ranking order of financial risk of the companies, which is more flexible and simple in solving MGCDM problem [16]. Therefore, the proposed financial risk assessment model for companies can obtain the best company with the least financial risk reliably.

Sensitivity analysis

In order to monitor the robustness of the financial risk assessment model for companies, the sensitivity analysis is conducted according to the change of the weight coefficient and the evaluator’s attitude λ. The corresponding ranking order of the three companies can be obtained when the value of is changed, which are listed in Table 13. And the influence on the proposed financial risk assessment model with different values of listed in Table 13 can be figured in S8 Fig.

According to Table 13 and S8 Fig, it is clear that the ranking order calculated are the same as in the above experimental example, when the value of is changed from 0 to 1. This means that the ranking order is insensitive to the changes of parameter . That is to say, despite the assessment process involving different values of the weighting coefficient , the final ranking order is consistent.

When the evaluator’s attitude λ is changed, the evaluation values of credit rating are transformed into triangular fuzzy number, the influence of values λ on the proposed model is shown in Table 14 and S9 Fig. Obviously, the changes of the evaluator’s attitude λ do not influence the ranking order of the three companies, and the results are the same as that of the above experimental example.

According to the visualized results shown in S8 and S9 Figs, the final ranking order is consistent in the experimental example of the sensitivity analysis. In other words, although the different selection of weighting coefficient and evaluator’s attitude λ, x3 is the best company with the least financial risk. The two sensitivity analysis results indicate that the ranking order of the proposed model is insensitive to the values of and λ in the example. Therefore, to a certain degree, the robustness of the proposed model is verified.

Conclusion and future research

In this paper, a multi-level fuzzy comprehensive financial risk assessment model for companies has been developed. In order to assess the company’s financial risk accurately, subjective and objective indexes have been utilized simultaneously in the financial risk index system, which combines fuzzy theory with quantitative data analysis. Moreover, heterogeneous information obtained from historical financial information of company, credit rating agency and the decision makers’ estimation, such as crisp numbers, triangular fuzzy numbers and neutrosophic numbers, has been employed to decrease the information loss. In addition, TOPSIS based on heterogeneous MCDM has been employed to obtain the ranking order of companies according to their financial risk.

The proposed model has been used in empirically study to assess the financial risks of the listed pharmaceutical manufacturing companies of Chinese A-share. Moreover, the comparison results with the two other methods show that the proposed model is effective and reliable. In addition, the sensitivity analysis has been carried out and the results verify the robustness of the proposed model.

In summary, this paper not only contributes to the development of theory, but also contributes to practical application. First, the proposed model uses both quantitative historical data analysis and fuzzy theory; it is helpful to enrich the contents of risk research. Second, the proposed model will optimize the financial risk assessment method for companies. Third, the proposed model can be applied to provide rational support for decision makers in the process of financial risk management.

There are several possible directions of further research. First, more information types of financial risk assessment could be considered in the proposed model in order to adapt to the dynamic financial environment in future research. Second, the proposed model can obtain the ranking order of companies according to financial risk by investigating MULTIMOORA (multi–objective optimization by ratio analysis plus the full multiplicative form) method because of its simple computation. Finally, the proposed model can be adopted for risk assessment for some other fields in future study.

Supporting information

S1 Fig. Summary of the process of the proposed model.

https://doi.org/10.1371/journal.pone.0208166.s001

(TIF)

S3 Fig. The membership function of credit rating.

https://doi.org/10.1371/journal.pone.0208166.s003

(TIF)

S8 Fig. The result of the sensitivity analysis with different .

https://doi.org/10.1371/journal.pone.0208166.s008

(TIF)

S9 Fig. The radar plot displaying the result of the sensitivity analysis.

https://doi.org/10.1371/journal.pone.0208166.s009

(TIFF)

Acknowledgments

This work was supported by a Key Project of Hunan Social Science Achievement Evaluation Committee (XSP2016040508, XSP18ZD1002), the human philosophy social science fund project (15JD21).

References

  1. 1. matillaevich HON. Financial risk management at the enterprise: methods and models. Working Papers. 2016;4(4):67–9.
  2. 2. Ergu D, Kou G, Shi Y, Shi Y. Analytic network process in risk assessment and decision analysis. Computers & Operations Research. 2014;42(2):58–74.
  3. 3. Rousseeuw PJ. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational & Applied Mathematics. 1987;20(20):53–65.
  4. 4. Fu JJ, editor Establishment and research on the model of the company’s financial risk warning based on principal component analysis and logistic regression. International Conference on Education Technology, Management and Humanities Science; 2015: Atlantis Press
  5. 5. Ji P, Zhang H-Y, Wang J-Q. Fuzzy decision-making framework for treatment selection based on the combined QUALIFLEX–TODIM method. International Journal of Systems Science. 2017;48(14):3072–86.
  6. 6. Doumpos M, Zopounidis C. Assessing financial risks using a multicriteria sorting procedure: the case of country risk assessment. Omega. 2001;29(1):97–109.
  7. 7. Lee PT-W, Lin C-W, Shin S-H. A comparative study on financial positions of shipping companies in Taiwan and Korea using entropy and grey relation analysis. Expert Systems with Applications. 2012;39(5):5649–57.
  8. 8. Kociu L, Mano R, Hysi A. Financial risk assessment of albanian SMEs with the help of financial ratio (a case study-SME-s in Gjirokasra region). European Scientific Journal. 2015;11:309–21.
  9. 9. Shaverdi M, Heshmati MR, Ramezani I. Application of fuzzy AHP approach for financial performance evaluation of Iranian petrochemical sector Procedia Computer Science. 2014;31:995–1004.
  10. 10. Li D-P, He J-Q, Cheng P-F, Wang J-Q, Zhang H-Y. A Novel Selection Model of Surgical Treatments for Early Gastric Cancer Patients Based on Heterogeneous Multicriteria Group Decision-Making. Symmetry. 2018;10.
  11. 11. Wang J, Ding D, Liu O, Li M. A synthetic method for knowledge management performance evaluation based on triangular fuzzy number and group support systems. Applied Soft Computing. 2016;39:11–20.
  12. 12. Zhang H-Y, Ji P, Wang J-Q, Chen X-H. A novel decision support model for satisfactory restaurants utilizing social information: A case study of TripAdvisor.com. Tourism Management. 2017;59:281–97.
  13. 13. Rivieccio U. Neutrosophic logics: prospects and problems. Fuzzy Sets and Systems. 2008;159(14):1860–8.
  14. 14. Shih H-S, Shyur H-J, Lee ES. An extension of TOPSIS for group decision making. Mathematical & Computer Modelling. 2007;45(7):801–13.
  15. 15. Afshar A, Mariño MA, Saadatpour M, Afshar A. Fuzzy TOPSIS multi-criteria decision analysis applied to Karun Reservoirs System. Water Resources Management. 2011;25(2):545–63.
  16. 16. Lourenzutti R, Krohling RA. A generalized TOPSIS method for group decision making with heterogeneous information in a dynamic environment. Information Sciences. 2016;330:1–18.
  17. 17. Li D-F, Huang Z-G, Chen G-H. A systematic approach to heterogeneous multiattribute group decision making. Computers & Industrial Engineering. 2010;59(4):561–72.
  18. 18. Wang GWY, Woo S-H, Mileski J. The relative efficiency and financial risk assessment of shipping companies. Maritime Policy & Management. 2014;41(7):651–66.
  19. 19. Mardani A, Jusoh A, Zavadskas EK. Fuzzy multiple criteria decision-making techniques and applications—Two decades review from 1994 to 2014. Expert Systems with Applications. 2015;42(8):4126–48.
  20. 20. Sabokbar HF, Ayashi A, Hosseini A, Banaitis A, Banaitienė N, Ayashi R. Risk assessment in tourism system using a fuzzy set and dominance-based rough set. Technological & Economic Development of Economy. 2016;22(4):554–73.
  21. 21. Mardani A, Nilashi M, Zavadskas EK, Awang SR, Zare H, Jamal NM. Decision making methods based on fuzzy aggregation operators: three decades review from 1986 to 2017. International Journal of Information Technology & Decision Making. 2018.
  22. 22. Ribeiro MIF, Ferreira FAF, Jalali MS, Meidutėkavaliauskienė L. A fuzzy knowledge-based framework for risk assessment of residential real estate investments. Technological & Economic Development of Economy. 2017;23(1):140–56.
  23. 23. Kochanek K, Tynan S. The environmental risk assessment for decision support system for water management in the vicinity of open cast mines (DS WMVOC). Ukio Technologinis Ir Ekonominis Vystymas. 2010;16(3):414–31.
  24. 24. Chang B, Kuo C, Wu CH, Tzeng GH. Using Fuzzy Analytic Network Process to assess the risks in enterprise resource planning system implementation. Applied Soft Computing Journal. 2015;28:196–207.
  25. 25. Gonçalves TSH, Ferreira FAF, Jalali MS, Meidutėkavaliauskienė I. An idiosyncratic decision support system for credit risk analysis of small and medium-sized enterprises. Technological & Economic Development of Economy. 2016;22(4):1–19.
  26. 26. Kou G, Peng Y, Wang G. Evaluation of clustering algorithms for financial risk analysis using MCDM methods. Information Sciences. 2014;275(11):1–12.
  27. 27. Shaverdi M, Ramezani I, Tahmasebi R, Rostamy AAA. Combining fuzzy AHP and fuzzy TOPSIS with financial ratios to design a novel performance evaluation model. International Journal of Fuzzy Systems. 2016;18(2):248–62.
  28. 28. Cui XH, He ZQ. Application of the fuzzy ahp model based on a new scale method in the financial risk assessment of the listing corporation. Chemical Engineering Transactions. 2015;46:1231–6.
  29. 29. Jurczyk J, Eckrot A, Morgenstern I. Quantifying systemic risk by solutions of the mean-variance risk model. PLoS One. 2016;11(6):e0158444. pmid:27351482.
  30. 30. Wang M, Liu P. An extended VIKOR method for investment risk assessment of real estate based on the uncertain linguistic variables. Advances in Information Sciences and Service Sciences. 2011;3(7):35–43.
  31. 31. Kiliçman A, Sivalingam J. Portfolio optimization of equity mutual funds—malaysian case study. Advances in Fuzzy Systems. 2010;2010.
  32. 32. Wang J, Wang J-Q, Tian Z-P, Zhao D-Y. A multihesitant fuzzy linguistic multicriteria decision-making approach for logistics outsourcing with incomplete weight information. International Transactions in Operational Research. 2018;25:831–56.
  33. 33. Zhao X-B, Hwang B-G, Gao Y. A fuzzy synthetic evaluation approach for risk assessment: a case of Singapore’s green projects. Journal of Cleaner Production. 2016;115:203–13.
  34. 34. Mangla SK, Kumar P, Barua MK. Risk analysis in green supply chain using fuzzy AHP approach: a case study. Resources, Conservation and Recycling. 2015;104:375–90.
  35. 35. Rezaei J. Best-worst multi-criteria decision-making method. Omega. 2015;53:49–57.
  36. 36. Rezaei J. Best-worst multi-criteria decision-making method: some properties and a linear model. Omega. 2016;64:126–30.
  37. 37. Tian Z-P, Wang J-Q, Zhang H-Y. An integrated approach for failure mode and effects analysis based on fuzzy best-worst, relative entropy and VIKOR methods. Applied Soft Computing. 2018.
  38. 38. Hwang CL, Yoon KP. Multiple attributes decision making: methods and applications. 1st Edition ed. New York: Springer-Verlag; 1981.
  39. 39. Zavadskas EK, Mardani A, Turskis Z, Jusoh A, Nor KM. Development of TOPSIS method to solve complicated decision-making problems: An overview on developments from 2000 to 2015. International Journal of Information Technology & Decision Making. 2016;15(03):645–82.
  40. 40. Dağdeviren M, Yavuz S, Kılınç N. Weapon selection using the AHP and TOPSIS methods under fuzzy environment. Expert Systems with Applications. 2009;36(4):8143–51.
  41. 41. Ahmadi MH, Ahmadi MA. Multi objective optimization of performance of three-heat-source irreversible refrigerators based algorithm NSGAII. Renewable & Sustainable Energy Reviews. 2016;60:784–94.
  42. 42. Taylan O, Bafail AO, Abdulaal RMS, Kabli MR. Construction projects selection and risk assessment by fuzzy AHP and fuzzy TOPSIS methodologies. Applied Soft Computing Journal. 2014;17(4):105–16.
  43. 43. Wang H-B, Smarandache F, Zhang Y-Q, Sunderraman R. Interval neutrosophic sets and logic: theory and applications in computing. Computer Science. 2005;65:vi 87.
  44. 44. Wang H-B, Zhang Y-Q, Sunderraman R, Smarandache F. Single valued neutrosophic sets. Fuzzy Sets, Rough Sets and Multivalued Operations and Applications. 2011;3:31–9.
  45. 45. Majumdar P, Samanta SK. On similarity and entropy of neutrosophic sets. Journal of Intelligent & Fuzzy Systems. 2014;26:1245–52.
  46. 46. Tan R, Zhang W, Chen L. Study on emergency group decision making method based on VIKOR with single valued neutrosophic sets. Journal of Safety Science and Technology. 2017;13:79–84.
  47. 47. Ye J. A multicriteria decision-making method using aggregation operators for simplified neutrosophic sets. Journal of Intelligent & Fuzzy Systems. 2014;26:2459–66.
  48. 48. Wang J-Q, Yang Y, Li L. Multi-criteria decision-making method based on single-valued neutrosophic linguistic Maclaurin symmetric mean operators. Neural Computing and Applications. 2018;30(5):1529–47.
  49. 49. Procházka A, Uhlíř J, Rayner PWJ, Kingsbury NG. System Identification. Signal Analysis and Prediction. Boston, MA: Birkhäuser Boston; 1998. p. 163–73.
  50. 50. Laarhoven PJMv, Pedrycz W. A fuzzy extension of Saaty’s priority theory: Elsevier North-Holland, Inc.; 1983. 229–41 p.
  51. 51. Luo S-Z, Cheng P-F, Wang J-Q, Huang Y-J. Selecting project delivery systems based on simplified neutrosophic linguistic preference relations. Symmetry. 2017;9(8):151.
  52. 52. Tian Z-P, Wang J-Q, Zhang H-Y. Hybrid single-valued neutrosophic MCGDM with QFD for market segment evaluation and selection. Journal of Intelligent & Fuzzy Systems. 2017;34:177–87.
  53. 53. Wang J-Q, Zhang X, Zhang H-Y. Hotel recommendation approach based on the online consumer reviews using interval neutrosophic linguistic numbers. Journal of Intelligent & Fuzzy Systems. 2018;34:381–94.
  54. 54. Liang RX, Wang JQ, Zhang HY. Evaluation of e-commerce websites: an integrated approach under a single-valued trapezoidal neutrosophic environment. Knowledge-Based Systems. 2017;135:44–59.
  55. 55. Peng H-G, Zhang H-Y, Wang J-Q. Probability multi-valued neutrosophic sets and its application in multi-criteria group decision-making problems. Neural Computing and Applications. 2018;30(2):563–83.
  56. 56. Lourenzutti R, Krohling RA. TODIM based method to process heterogeneous information. Procedia Computer Science. 2015;55:318–27.
  57. 57. Wu Y, Liu L, Gao J, Chu H, Xu C. An extended VIKOR-based approach for pumped hydro energy storage plant site selection with heterogeneous information. Information. 2017;8(3). https://doi.org/10.3390/info8030106.