https://orcid.org/0000-0002-6047-9434
Figures
Abstract
The performance evaluation of local self-government entities is very difficult, as their primary goal is not to make a profit, but to provide services to their residents that will contribute to an increase in their quality of life. In this context, it is necessary to evaluate their activity from the point of view of several available criteria, for which it is possible to find relevant and recognized sources. The presented research works with five criteria, identified by the Institute for Economic and Social Reforms, and aims to quantify the agreement of the results of the assessment of the financial health of territorial self-government entities in 2020 using the TOPSIS technique with a gradually decreasing number of criteria. For this purpose, a total of 26 combinations of criteria are created, with the number of 5, 4, 3 and 2 used criteria, the importance of which is determined based on their absolute variability using the standard deviation method. The results obtained in this way are interpreted using a wide range of mathematical and statistical methods including the Kolmogorov-Smirnov test, Levene test, Jaccard index and others. As a result, the multi-criteria evaluation of territorial self-government subjects (in our case, district cities) proved to be highly applicable. However, the result itself is largely determined by the structure and number of entry criteria. Based on the performed analyses, we can see that significant differences result from their reduction. Each such reduction has an impact on the overall results, but it is possible to find combinations that defy this conclusion.
Citation: Vavrek R (2024) How much is too much: A case study of local self-government units in Slovakia using absolute variability to determine the importance of financial criteria in MCDM analysis. PLoS ONE 19(10): e0311842. https://doi.org/10.1371/journal.pone.0311842
Editor: Kealeboga Kaizer Moreri, University of Botswana, BOTSWANA
Received: July 3, 2024; Accepted: September 25, 2024; Published: October 15, 2024
Copyright: © 2024 Roman Vavrek. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data are fully available at https://www.hospodarenieobci.sk/.
Funding: This work was supported by the Student Grant Competition Project at VSB – Technical University of Ostrava, project number SP2023/064. There was no additional external funding received for this study.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Evaluating the performance of local self-government entities is very complex, which is a consequence of the fact that municipalities are not primarily supposed to make a profit, but rather to provide services to their residents that will contribute to an increase in their quality of life. The results of such municipal activities are largely intangible and often not easily observable or measurable [1, 2]. Measuring the success, or performance, of their activities is therefore especially demanding.
Pursuant to Act no. 221/1996 Coll. on the Territorial and Administrative Organization of the Slovak Republic, its territory is divided into 8 regions and 79 districts; see the following map/figure (Fig 1).
Out of 79 districts, only 69 have district towns, the exception being 5 districts in the Bratislava self-governing region (Bratislava I district, Bratislava II district, Bratislava III district, Bratislava IV district, Bratislava V district) and 5 districts located in the Košice self-governing region (Košice I district, Košice II district, Košice III district, Košice IV district, Košice-okolie district). Apart from these 69 district towns, the Institute for Economic and Social Reforms also evaluates the financial health of the municipality of the capital Bratislava and Košice, i.e. the total number of evaluated subjects within the submitted research is 71.
Currently, in the conditions of municipalities, single-criteria methods are used sporadically. Among those used, we can include cost-benefit analysis (CBA), cost-effectiveness analysis (CEA), return on investment method (RI), payback method (PM), net present value method (NPV) and internal rate of return method (IRR).
Some authors [3] consider performance in the public sector and thus also in local government to be a multidimensional concept. Subjects moving in this environment should be evaluated based on several criteria. However, different applications are offered: in the form of a financial and property analysis assessment (FAMA) with 22 criteria [4]; in the form of a global competitiveness index with 12 pillars and a total of 102 criteria [5]; in the form environmental sustainability with 25 criteria [6]; in the form of fiscal rules and 13 criteria [7]; in the form of an assessment of financial autonomy of rural municipalities with 9 criteria [8]; and in the form of 10 European common indicators (ECI) [9]. In Slovak conditions, the issue was also addressed: in the evaluation of selected municipal services using 15 criteria [10]; in evaluating the use of the property of local governments using 6 criteria [11]; in evaluating the efficiency of municipal property used for business activity using 12 criteria [12]; in evaluating the management of municipalities using 8 criteria [13, 14]; and in the form of the financial health of municipalities and higher territorial unit (HTU) consisting of 5 indicators of financial stability [15].
All these approaches evaluate a selected group of subjects using a different number of criteria, starting with 5 and ending with a group with over 100 criteria. If the number of criteria is reduced, information is lost. What loss is acceptable? When working with a number of 100 criteria [5], a reduction of one criterion does not automatically imply a significant change in the overall results of the multi-criteria analysis. As the number of criteria decreases, their relative importance increases (1/n, when n = 100, 99, 98, …, 5, 4, 3, 2). For the purposes of this research, we have chosen the previously used financial health of municipalities and HTU [16] that works with 5 criteria, namely:
- Total debt,
- Debt service,
- Current account balance,
- Liabilities after the due date,
- Liabilities at least 60 days past due.
By reducing the number of criteria by one, 20% of the information is lost (1 in 5). The loss of one criterion increases the importance of the remaining criteria. Our interest here is to what extent the loss of one criterion affects the overall results. If the change is too large, it is not possible to reduce the criteria at all. If the change is too small, the use of the criterion is questionable. But what is the optimal number of criteria? Therefore, the research question of the present research is as follows:
Is there a minimum number of monitored criteria that does not result in a significant loss of information about the real state of the assessed entities?
The aim of the presented research is to quantify the agreement of the results of the assessment of the financial health of the entities of the territorial self-government in 2020 using the TOPSIS technique with a gradually decreasing number of criteria.
To fulfil the set goal, the presented article is divided into sections as follows. The first section is the Introduction. The second section is devoted to an introduction of the TOPSIS technique as the main methodological tool of the presented research. The third section describes a set of five evaluated criteria using selected moment characteristics. Its content also includes the identification of the structure of 26 combinations of criteria with a decreasing number, which are subsequently the subject of further evaluation. In a separate sub-section, attention is also directed to determining the weight of the criteria, which directly affects the result of the multi-criteria analysis. The content of this section also includes the identification of the research sample and the mathematical and statistical methods used to evaluate the obtained results. These are described and interpreted within the fourth section, which is processed using 5, 4, 3 and 2 entry criteria. Summarizing the knowledge gained is the content of the fifth and last section, which is the Conclusion.
Literature review
Need of comparing MCDM methods and the importance of selection for subsequent analysis for the first time was first explored in 1968 [17]. It is possible to find a number of studies that addressing this issue [18, 19]. One of them proposed a methodological approach to selecting appropriate MCDM method for a particular decision-making situation [20]. Subsequently, they were other studies comparing the appropriateness of using different methods have been published [21, 22]. Most of these approaches are based on restrictive assumptions regarding the structure of preferences of the decision maker or are developed to address problems of special structure, e.g., many (qualitative) criteria and few alternatives [23, 24] or few quantitative criteria and many alternatives [25, 26]. In general, it is very difficult to decide which technique is the most appropriate [27, 28] and depends on the specific conditions of the decision project.
The TOPSIS technique (Technique for Order of Preference by Similarity to Ideal Solution) is one of the basic methods of multi-criteria decision-making, and its primary use is in solving different types of decision-making problems. This method is one of the most widely used [14], with the SAW, AHP, or VIKOR methods as possible alternatives. A theoretical comparison is already processed [29]. Results of practical applications [30] show that TOPSIS produces rankings which are extremely similar to the ones resulting from SAW. Compared to the VIKOR method, TOPSIS method introduces two “reference” points, but it does not consider the relative importance of the distances from these points [31]. In the comparison between the VIKOR and TOPSIS methods, 84.1% (data in uniform distribution) and 82.3% (data in normal distribution) of the ranking results were the same, and in the comparison between the proposed method and VIKOR, 83.7% (data in uniform distribution) and 83.7% (data in normal distribution) of the ranking results were the same [32]. The TOPSIS approach is less predictive than the AHP model [33]. Table 1 offers a simple comparison with other MCDM methods.
An overview of its applications was captured by many authors [34, 35], who noted an annually increasing number of articles in which the use of the TOPSIS technique could be found. The choice of the TOPSIS technique for the purposes of our research was based on its previous successful use in solving decision-making problems of a similar nature. You can find its applications in environmental science, where this technique was used for the evaluation of electricity policy and progress in renewable energy technologies usage [36–38]. It was used also in transport sector for energy-environment efficiency of European transport sectors evaluation and efficiency of transport companies in the Czech Republic [39, 40]. The comparison of local government units [41] and culture units [42] were other examples of successful application of this approach. We are able to find many other areas of its successful applications that support our decision to select this method for our research, see [43–49].
The principle of the TOPSIS technique could be illustrated using the above figure [34], see Fig 2. Each white ball represents one specific alternative, i.e. one evaluated subject. The grey ball represents the Negative Ideal Solution (NIS) alternative, i.e. a real or hypothetical alternative (subject) with the worst values of the individual criteria. The black ball represents the Positive Ideal Solution (PIS) alternative, i.e. a real or hypothetical alternative (subject) with the best values of the individual criteria. The alternative (one of the white balls) that is farthest from the grey ball (NIS) and closest to the black ball (PIS) is the best rated.
This method is a suitable decision-making tool based on incomplete data [50]. The range of selected data is not decisive for its use, i.e. it is possible to use data of any range [51, 52]. The strengths include straightforwardness and simplicity of calculation [53, 54], the ability to work with all types of criteria [52], and complexity [55]. The main disadvantage of the TOPSIS method is considered to be the absence of the possibility of assigning weights to the monitored criteria and the absence of consistent control by the decision-maker [51]. For this reason, this method is dependent on the process of finding the relative importance of different criteria, which is also the area of interest of the presented research. Many authors [56–58] deal with the calculations of the TOPSIS technique itself in more detail.
In general, it can be stated that the result of the application of the multi-criteria decision-making (MCDM) method is directly determined by the person making the decision (by the decision-maker). The process of determining the importance of the evaluated criteria is very important [59], as weights affect the final arrangement, and an improper determination of the weights can lead to a change in the arrangement, including what approach or method is chosen for determining the importance of individual criteria. One classification of these methods identifies two groups of methods [60]. Other one works with three groups [58], while the third one [61] uses four groups of weighting approaches that represent an extension of the previous classifications. The groups in question are as follows:
- subjective methods,
- expert methods,
- objective methods,
- integrated methods.
Subjective methods reflect the personality of the decision-maker and his/her individual preferences (the weight of the indicator is determined on the basis of subjective opinion). Expert evaluation is carried out by a smaller number of experts in the given field, while an application from the past is offered [62–64]. The use of the opinion of an expert group using a pairwise comparison of criteria (e.g. using Fuller’s triangle) can be found in several researches [65–67]. The third group, i.e. a group of objective methods, assigns weight to individual criteria based on a predetermined mathematical model unique to each method. Therefore, the decision-maker does not have a direct influence on determining the importance of the criteria. She/He chooses, according to personal preference, the attributes of the data used, e.g. according to the variability or relationships between criteria. In this group it is possible to find methods such as the mean weight method [68–70], standard deviation method [41, 71], Mahalanobis-Taguchi system method [72], λ bi-capacity model [73], coefficient of variance method [74] and others [75, 76]. The last group presents integrated methods, which represent a combination of the already described approaches. Other approaches are presented in actual research applied in different areas, e.g., AHP according to experts’ views [77], a linear regression model [78], KEMIRA-M procedure [79], MOWSCER method [80], N-CRITIC method [81], BWM method [82], etc.
Materials and methods
The financial health of a district city calculated by the third side [15] provides information about sustainable management and whether or not municipal management is causing problems for the respective city. This rating is “normed”, representing a single number between 0 (worst) and 6 (best). More detailed information on the calculation of individual indicators can be found on the website.
Our intention is not to challenge this calculation–quite the contrary. In the previous section, different approaches to the evaluation of the management of the basic entities of territorial self-government were presented, which worked with a different number of entry criteria. Compilation of a relevant set of evaluated criteria is a significant factor influencing the final evaluation in any evaluation process. Our research works with the entry criteria of this financial health [13], which is calculated as a weighted average of the scores achieved by each of its five components:
- K1 ‐ Total debt,
- K2 ‐ Debt service,
- K3 ‐ Current account balance,
- K4 ‐ Liabilities after the due date,
- K5 ‐ Liabilities at least 60 days past due.
All criteria represent a relative variable expressed in percentages, i.e. from this point of view, it is a homogeneous group, the parameters of which can then be easily compared with each other.
Our methodology can be described using the following steps (phases):
- Literature review I. ‐ based on the comprehensive literature review, the TOPSIS technique is selected as a main research tool,
- Literature review II. ‐ based on the comprehensive literature review, the SD method is selected as tool for determining the importance of entry criteria (in association with the TOPSIS technique),
- Data preparation ‐ 5 criteria are selected and processed for the needs of use using the above methods,
- Analysis realization ‐ 26 separate analysis is realized to achieve the aim of this research (A, B1-B5, C1-C10, D1-D10),
- Conclusion formulation ‐ based on the results achieved, the limitations and potential further research topics are formulated.
Comparison of the evaluated criteria in the light of moment characteristics
The use of criteria in a relative form (as a percentage) enables their simple comparison without the need for their normalization or other modification. The moment characteristics described above reflect the attributes of the individual criteria, which can be compared through the following pair of figures.
The first one (Fig 3) offers the results of the ANOM analysis (analysis of mean), i.e. it compares the average of individual criteria with the common average for all criteria. From this point of view, the K1 criterion is dominant, as the arithmetic mean is at the level of 18%. We observe the opposite trend in the case of criteria K4 and K5, for which the average oscillates around the zero value; this is due to the high number of district towns with a zero value for the monitored criterion.
The mean value shown on the previous figure also indicates the distribution of individual observations (district towns) across the variation range of that criterion. The most even use of the available variation range is also offered by the criterion with the highest mean value, i.e. total debt (K1), as mentioned above. In the case of criteria K4 and K5, we observe the most subjects with the same value (zero), which is reflected in the quantile plot in the form of a significant vertical curve for both criteria.
The individual criteria have significantly different attributes, which indicates their added value and justification for subsequent use within the intended multi-criteria analysis, as they are unique in their structure and do not repeat each other. This fact is also documented by the following figure (Fig 4), with contains the results of the rank correlation analysis. Only in one case is there a positive rank correlation between the pair of criteria K1 and K2, but we can call it small.
Combinations of entry criteria as a structure of our own research
The primary result of the performed analysis is given by the TOPSIS technique using all 5 criteria, which represents the “benchmark” of individual analyses. Thus, the set of criteria is given, and as such represents combination A (see Table 2).
The results obtained with a smaller number of criteria are compared with these results in the following structure. In the first part of the analysis, one criterion is removed from these 5 criteria each time, i.e. multicriteria analysis is processed using 4 criteria a total of 5 times (combinations B1 –B5). The structure of each combination is captured in the following table (Table 3).
In the second part, the results are obtained using three monitored criteria, i.e. in each case, 2 of the 5 criteria are removed. A total of 10 combinations are thus created, each of which represents a separate calculation, or analysis (see Table 4).
In the third and last step, the TOPSIS technique is calculated using only a pair of criteria, while the other 3 criteria are not considered. By choosing 2 criteria out of 5, we get 10 combinations, which represent further processed combinations C1 –C10 within the presented research (see Table 5).
In total, each of the performed analyses consists of the performance of 26 calculations, taking into account the above-mentioned structure of entry criteria. The obtained results are compared with each other using a wide set of mathematical and statistical methods described in other parts of this section.
The importance of criteria in the context of their decreasing number
The presented research works with an approach to determining the importance of evaluated criteria, which represents a representative of objective methods for determining the importance of criteria. This approach works with the absolute expression of the variability of the evaluated criterion, while the obtained results are therefore influenced by the values of the moment characteristics of the level and position. Its previous successful usages can be found.
The group of objective methods consists of methods that work with the moment characteristics of the criteria in order to determine their importance. The simplest way is to use the variability of the criteria, either in absolute or relative terms. All other approaches require additional calculations, increasing the complexity on the part of the decision maker [77–83].
The importance of entry criteria is determined based on variability in absolute as the simplest approach for weight of criteria determination. The importance of the evaluated criteria is determined based on the standard deviation method (i.e. standard deviation method, hereinafter referred to as “SD”).
(1)
where: SDj = standard deviation of j(th) criterion
n = number of criteria
Using this approach and the moment characteristics described in the previous section, the importance of the criteria in individual combinations is calculated.
The principle of this method used for determining the importance of criteria is based on measuring their absolute variability through one of the momentary characteristics of variability, namely the standard deviation. Within combination A, district towns are evaluated using 5 criteria, where this approach gives the highest importance to the first of these criteria, i.e. Total debt (w1 = 0.602). This is followed by a pair of criteria with importance at the levels of 15.21% and 21.32%. From this point of view, the significance of the remaining two criteria is minimal and, in both cases, does not exceed 2% (see Fig 5).
In the next step, we work with 4 criteria (combinations B1-B10), where the first criterion is always dominant (criterion first in order, not K1). Its importance, apart from combination B5, exceeds 60%, while in combinations B3 and B4 its dominance is even higher (w1 = 0.765 and w1 = 0.710, respectively). In all these cases, it is one and the same criterion, namely Total debt (K1), which mostly affects the results of the multi-criteria evaluation using the TOPSIS technique. We see a different situation in the last combination, B5, where the dominance of the first criterion decreases; this is caused precisely by the absence of the K1 criterion. However, the importance of the last criterion is always minimal, which is also illustrated by the following figure (Fig 6).
Combinations marked C consider the three entry criteria, see Fig 7. In most cases (C1-C8), we observe the dominance of the first criterion, which is one of the criteria K1, K2, K3. The combination C3 is interesting, in which the dominance of the first criterion is most pronounced when the weight of the other 2 criteria does not exceed 3%. With combinations C9 and C10, it is possible to talk about the balance of the two most important criteria. The small influence of the third criterion on the overall results of the TOPSIS technique persists regardless of the degree of dominance of the first criterion.
Combinations D1 –D10 are mostly characterized by the dominance of one of the two evaluated criteria, see Fig 8. In five cases (D3, D4, D7, D8, D9), the importance of the second criterion does not even reach the level of 10%, i.e. the overall result is primarily influenced by the more significant of the given pair of criteria. Only in the case of the D10 combination, including criteria K4 (Liabilities past due to income) and K5 (Liabilities at least 60 days past due), can we call these criteria relatively equivalent.
From the point of view of individual combinations, we observe significant differences in the importance of the criterion determined by the SD method. The dominance of criterion K1, which was the criterion with the highest importance values in 15 out of 26 combinations, is interesting. On the other hand, it was not the least important criterion in any case, regardless of whether 5, 4, 3 or 2 criteria were evaluated. We observe the opposite situation in the case of Liabilities at least 60 days past due (K5), which is the exact opposite of the above-mentioned criterion. This criterion is not the most important in any combination, but in 15 cases it was the criterion with the lowest importance (i.e. in all combinations of which it is a part), see Table 6.
Description of the apparatus of the mathematical and statistical methods used
A wide range of mathematical and statistical methods is used for processing the obtained results, a description of which can be found in several sources [84–88]. The only differences are the designations used in individual publications.
The basic prerequisite for the selection of appropriate methods in any statistical investigation is to identify whether or not the investigated data come from a normal distribution. This assumption subsequently serves as a starting point when choosing from parametric or non-parametric statistical methods. The normality of the distribution of the TOPSIS technique, resulting from the use of individual combinations (A, B1-B5, C1-C10, D1-D10), is monitored by the Shapiro-Wilk test [89].
The starting point and thus the tested sets for statistical processing are the results obtained by applying the TOPSIS technique. The comparison of the order obtained in this way is carried out by means of Kendall’s rank [62] correlation coefficient (hereinafter referred to as “Kendall’s coefficient”). This coefficient represents a non-parametric alternative to the frequently used Pearson’s correlation coefficient [90] (hereinafter referred to as “Pearson’s coefficient”), which also tracks the correlation between two numerical variables.
The strength of the Kendall coefficient is lower, but it is useful in the event of extreme values that significantly affect the value of the regression coefficients and thus the estimation of the residuals. The Kendall coefficient is automatically used in the analysis in the case of monitoring the correlation of two numerical variables when the condition of using the Pearson coefficient is not met, i.e. normality of both observed variables (tested by the aforementioned Shapiro-Wilk test).
Another subject of testing is the agreement of the results of the TOPSIS technique using five criteria (combination A) with other results (combinations B1-B5, C1-C10, D1-D10), which is established by testing the agreement of distribution functions using the Kolmogorov-Smirnov test [91]. This test has two variants; for the purposes of our own research, the second one is used, i.e. we compare two distribution functions and verify their agreement.
The agreement of the standard deviation of the two indicated above, i.e. homoscedasticity, is verified through the Levene test [92]. The agreement of the obtained results is also monitored at the level of the ranking of individual subjects, whose agreement is quantified by means of the Jaccard index.
When estimating the spatial association, first, the spatial relations between territorial units/areas must be defined. They describe their spatial weights calculated from point or polygon data, although the method of calculation differs. Spatial weight is a basic element of spatial statistics for measuring spatial ties. Spatial weight matrices represent the strength of potential interaction between individual places, or between spatial units [93]. Rook-type weights are used, so only areas adjacent to the investigated area are taken into account, which is also reflected in the neighbourhood matrix. After identifying the spatial weights, Moran’s local coefficient [94] is used, which determines whether there is spatial autocorrelation for a given set of regions, or territory.
The one-sided relationship between the dependent (results of the TOPSIS technique) and the independent variable (number of inhabitants) is verified on the basis of a simple regression model, whose explanatory power is quantified using the coefficient of determination [95]. The obtained results are described using selected moment characteristics [96].
All analyses are processed using the software MS Office Excel, Statistica and Statgraphics.
Results and discussion
At the determined level of significance, the normality of the results was confirmed (S-W = 0.965; p = 0.05), while the results themselves are slightly negatively skewed compared to the Gaussian curve, with a smaller concentration around the mean value (β = -0.498). The interquartile range at the level of 0.299 ci represents more than 36% of the total variation range, which makes it possible to expect significant differences between the evaluated district towns. No extreme or outlier value appears in the results, which is precisely due to the relationship of the above-mentioned ranges (see Fig 9).
From the point of view of the distribution of results in space, we note their randomness, or spatial uncorrelation (M = 0.0004). The best-rated district towns are located primarily in Central Slovakia, with the absolute best being Tvrdošín (0.980 ci), followed by Poltár (0.928 ci) and Nový Mesto nad Váhom (0.910 ci). Half of the 10 worst-rated district towns can be found in Eastern Slovakia; the rest are scattered across the entire territory of Slovakia. Table 7 contains its statistical description.
The relationship between the obtained results of the multi-criteria evaluation and the number of inhabitants of the district towns can be quantified through a simple regression model (using OLS method) as follows:
(2)
where: NI = number of inhabitants
The degree of variability of the dependent variable in such a model is significant (R2 = 95.69%), i.e. the number of inhabitants has a positive effect on the overall results of the SD-TOPSIS technique when including five entry criteria (combination A).
Evaluation of district towns using four entry criteria (combinations B1-B5)
The results of the SD-TOPSIS technique when including 4 out of 5 input criteria (combinations B1-B5) show similar properties regardless of the omitted criterion, as in combination A. In 4 out of 5 cases the results are slightly negatively skewed, which corresponds to the position of the mean and the median. The highest average value is achieved by the combination B3 (0.638 ci), while the lowest is achieved by the combination B5 (0.577 ci); however, this is due to the overall lower results of the second mentioned combination. We observe significant variability measured by either the coefficient of variation or the standard deviation, which is given by a lower concentration of results around the mean value and a considerable interquartile range. The rejection of the assumption of homoscedasticity of the results can be attributed to the combination B5, which is significantly different from the results of all other combinations (LE = 6.478; p < 0.01). For more details, see Fig 10.
The high degree of similarity in the structure of the results of B1-B4 with the results of combination A can be captured using correlation analysis, specifically the Kendall coefficient (see the following table), which can be used to classify it as almost perfect (min. rK = 0.933). A district town with a high rating using 5 criteria also achieves a high rating using 4 criteria. In the case of combination B5, this correlation is also statistically significant, but its strength is significantly lower.
The correlation analysis is supplemented by a comparison of distribution functions through the Kolmogorov-Smirnov test. As can be read from the following table, in the case of 3 combinations (B1, B2, B4) we can talk about a perfect match of the distribution of the obtained results. It can initially indicate the substitutability of the combinations. Differences in results can be assumed for the combination B5, for which a statistically significant difference was confirmed.
Taking a closer look at the results of specific district towns, it is possible to find significant differences across the evaluated combinations. The Jaccard index points to an agreement in the order of district results exceeding 90% in the case of combination B1 (97.18%) and combination B2 (94.36%); see Fig 11. In the other 3 combinations, this agreement is significantly lower, while in combination B5 it is at the level of 2.81%. The order is therefore significantly heterogeneous within the individual combinations, which enhances the importance of the selection of entry criteria when evaluating their importance using the standard deviation.
Due to the small differences in the results, the agreement in the order needs to be perceived more loosely, i.e. via accepting deviations. In the case of acceptance of the difference of one place in the overall ranking, the similarity of the results increases with the B1 combination on 100%, since Vranov nad Topľou and Martin differed by only 1 place. We also observe a strong agreement in the case of the combinations B2 (97.18%) and B4 (69.01%). If we accept a difference of 2 places in the overall order, the agreement in the case of combination B2 would also be 100%, while in combination B4 we get to 87.32%. The combination of B5 with 12.67% turns out to be completely inappropriate. For detailed results, see Fig 12.
When evaluating the results of the SD-TOPSIS technique in the context of the size of the evaluated subject, the comparison is carried out by a regressor and a coefficient of determination; see the following table. The high informative value of individual regression models remains, as well as the positive influence of the number of inhabitants on the evaluation results. Across the individual combinations, we do not record a significant difference in this influence, which is also documented by the stable level of the regressor.
Evaluation of district towns using 3 entry criteria (combinations C1-C10)
When evaluating the results of the SD-TOPSIS technique including 3 of the 5 entry criteria (combinations C1-C10), we observe differences in the position of the mean values, variability or shape (see Fig 13). The variation ranges from 0.726 ci (C9) to 0.922 ci (C6). In 4 out of 10 combinations, it does not include the results of all district cities (identified outliers). In an absolute sense, the best results are achieved with the C7 combination, where we also observe their slightly negative skewness. The C2 combination has the most significant differences between the evaluated subjects; however, this does not mark the individual results as outliers. These differences lead to the rejection of the assumption of homoscedasticity of the results, which cannot be confirmed (LE = 6.765; p < 0.01) due to significant differences in the total number of 20 pairs of combinations, e.g. C1-C9, C3-C4, C6-C10 and others.
The relationship between the results of the initial combination A and combinations using 3 criteria is captured using the Kendall coefficient, i.e. using rank correlation analysis. From this point of view, the evaluated combinations can be divided into two groups. The first one, which includes combinations C1, C2, C3, C5, C6 and C8, is characterized by a very strong and almost perfect linear connection of the results of the SD-TOPSIS technique. The largest significant correlation is measured in the pair A-C1, while the smallest has a value of 0.917 (A-C3). The second group, including combinations C4, C7, C9 and C10, has a statistically significant rank correlation, but with a much lower strength.
The agreement in the structure of the obtained results of the SD-TOPSIS technique is verified by comparing the distribution functions offered by the following table. The agreement of the distribution functions can be found in 6 out of 10 pairs, specifically when comparing the results of combination A with the results of combinations C1, C2, C3, C5, C6 and C8. In other cases, the differences are statistically significant, which points to differences in the results of individual subjects with these combinations. In the group of 10 combinations with 3 entry criteria, it is possible to find those that largely copy the distribution of the results of combination A (with confirmed agreement of distribution functions). In other cases, the distribution is different; the most significant differences are observed in the case of the C4 combination.
The relevant differences in distribution functions also portend a different degree of agreement in the order of district towns, which is quantified for the needs of the presented research by the Jaccard index (see Fig 14). Its highest values are observed in the case of combination C1 (90.10%), which is closely followed by combinations with an agreement oscillating around 30%, namely C6 (29.57%) and C8 (30.98%). In other cases, the differences in the order are striking; the most significant ones are observed in the combinations C7, C9 and C10 at the level of 2.81%.
By accepting the difference of one place in the order of results, the similarity of the order of district towns in individual pairs increases, and the combination C1 again has the highest one (95.77%). Bánovce nad Bebravou, Levice, Pezinok and Sabinov were added to the 64 subjects with an exact matching order. The highest increases are recorded for combinations C6 and C8 (by 38.02 p.p.), and C2 and C5 (by 36.61 p.p.), in the case of which we can see an agreement exceeding 50%. With a tolerance of a difference of 2 places, combination C1 completely copies the ranking of combination A, see Fig 15. For the needs of further research, it is also necessary to consider combinations C6 and C8, with a match exceeding 85%.
The results of the SD-TOPSIS technique can also be evaluated in the context of the size of the evaluated subject, while the comparison itself is carried out through the regressor and the coefficient of determination, i.e. parameters of a simple regression model. The high informative value of the individual models remains, as well as the positive influence of the number of inhabitants on the evaluation results. Across the individual combinations, we do not register a significant difference in this influence, which indicates a similar distribution of the size groups of the evaluated subjects.
Evaluation of district towns using 2 entry criteria (combinations D1-D10)
Significant differences can be observed when comparing the results of the SD-TOPSIS technique including only 2 of the 5 entry criteria (combinations D1-D10). As can be identified from the figure below, differences can be found across all moment characteristics of level and position, variability or distribution shape. The variability expressed through the coefficient of variation ranges from 15.63% (combination D10) to 40.34% (combination D6), see Fig 16. In the case of the results with the lowest variability we also observe the highest concentration around the mean value (β = 25.30). These differences lead to the rejection of the assumption of homoscedasticity of the results, which cannot be confirmed (LE = 8.781; p < 0.01) due to significant differences in a total of 21 pairs.
Using the Kendall coefficient, with the exception of combination 10, a significant rank linear correlation with the results of combination A is confirmed. An interesting group consists of 4 combinations (D1, D2, D3, D4), where we can talk about an almost perfect rank correlation of their results with the results of combination A, i.e. a combination with 3 additional criteria (5 in total). In other cases, the strength of the correlation is 0.400, so it can be called weak or, in some cases, insignificant.
The agreement of the distribution functions of the results using 2 entry criteria (combinations D1-D10), with the results including all criteria in 3 combinations, was confirmed. The Kolmogorov-Smirnov test did not identify significant differences in pairs A-D1, A-D2, A-D3 and A-D4. In other cases, based on the results below, it is not possible to confirm the agreement in the structure of the results of the application of the SD-TOPSIS technique. Larger differences can be observed especially in the case of the D6 and D10 combinations. However, in the set of 10 combinations, it is also possible to find ones that copy the distribution of the results of combination A.
Based on the above, combinations D1-D10 can be divided into two groups. The first group (combinations D1-D4) match the ranking of combination A at the level of around 20%, while the best is combination D2 (28.16%), see Fig 17. Among the 20 district towns with the same order, it is possible to include, for example, Komárno, Levice, Senec and Šaľa. The second group of combinations achieves diametrically opposed results when comparing the ranking of district towns, while in the case of the combinations D6 and D7 not a single entity with the same ranking was found.
In the case of accepting a difference of 1 place in the order of evaluation by the SD-TOPSIS technique, the agreement of the order ranges from 45.07% to 67.60% in the case of combinations D1-D4. For combinations D5-D10, the maximum agreement is at the level of max. 11.26% (D6). It is the combination D2 that has the highest agreement in the order, with a tolerance of a difference of 2 places, namely 85.91% (see Fig 18). For combinations D1, D3 and D4, the agreement oscillates around 70% (D1–73.23%, D3–70.40%, D4–70.40%). On the contrary, with the combinations D6 and D10, even with this tolerance of 2 places, the agreement does not reach over 10% (D6–2.81%, D10–2.81%).
When evaluating the results of the SD-TOPSIS technique in the context of the size of the evaluated subject, the comparison is carried out by means of a regressor and a coefficient of determination; see the following table. The high informative value of the individual models remains, as well as the positive influence of the number of inhabitants on the evaluation results. Across the individual combinations, we do not record a significant difference in this influence, which is also documented by the stable level of the regressor.
Comparison of the evaluation parameters of district towns across all combinations
The comparison of SD-TOPSIS multi-criteria evaluation parameters copies the structure of individual sub-analyses, with the intention of easier comparison or orientation in the text. The linear rank correlation of the results for combination A (5 input criteria) with the other results decreases in direct proportion to the decreasing number of criteria, see Fig 19. With 4 criteria, the significant correlation could be described as very strong or almost perfect only with 4 combinations. The average value is influenced by the result of the B5 combination out of the others (rK = 0.381). With 3 criteria, we observe an average decrease of 6.40%, while with 5 combinations, the strength of the correlation is still above the level of 0.90. By reducing the number of criteria to 2, we still find 4 combinations with such a significant rank correlation (D1, D2, D3, D4). However, as the number of criteria decreases, the differences between the combinations increase, i.e. the relative and absolute variability of Kendall’s coefficient results increases.
As the number of entry criteria decreases, the number of confirmations of distribution function matching also decreases, but relatively slowly. Their agreement was confirmed in 80% of cases for B combinations, 60% of cases for C combinations and 40% of cases for D combinations. The structure of the results of the SD-TOPSIS technique therefore partially changes, but there are combinations where significant agreement remains.
The average agreement in order quantified using the Jaccard index for B combinations reaches 48.73%, while it is significantly positively influenced by the results of combination B1 (97.18%) and combination B2 (94.36%). As the number of criteria decreases, the agreement decreases significantly, the exception being combination C1 with an agreement of 90.14%. In combinations with 2 criteria, the match is no longer acceptable, i.e. below 30% (in the case of combination D2). Accepting a difference of 1 place, we find in the case of combination B1 a complete agreement of the order, while the agreement in the case of combinations B2 and C1 exceeds 95%. With an accepted difference of 2 places in the order of district towns, we observe 100% agreement in the 2 mentioned combinations, while in the other 3 we reach the agreement level of min. 85% (specifically for combinations B4, C6 and D2). For detailed results, see Fig 20.
The above-mentioned differences do not affect the connection between the obtained results and the number of inhabitants of the district towns. The relationship of these variables across individual combinations proved to be highly relevant (the coefficient of determination does not fall below 93%) and constant. The rating of the SD-TOPSIS district town rises slightly with the growing number of its inhabitants, see Fig 21.
Based on the above-mentioned results of individual analyses, we can state that SD-TOPSIS is applicable with several combinations of entry criteria. Their suitability is determined by the degree of accepted deviation. In the case of an exact match in order, it is a combination of B1, B2 and possibly C1. The last-mentioned combination must be taken into account in the case of accepting a difference of one or two places. In the case of this combination of entry criteria, relevant results are achieved in comparison with combination A, which enables a reduction in the number of monitored criteria. Due to the identified combinations, criteria K4 ‐ Liabilities overdue to income and K5 ‐ Liabilities at least 60 days past due, could be considered.
Comparison of overall results
Within the SD-TOPSIS combination, which determines the importance of entry criteria in the evaluation of district cities based on their absolute variability, it is possible to mark Tvrdošín (0.980 ci), followed by Poltár (0.928 ci), Nové Mesto nad Váhom (0.910 ci) and Topoľčany (0.859 ci) as the best-rated district towns. The results show no spatial correlation, with the best-rated district towns being located primarily in Central Slovakia, in the Žilina and Banská Bystrica self-governing regions. Based on the analyses carried out, we can state that SD-TOPSIS is applicable with a smaller number of criteria in the case of three combinations–B1, B2 and C1, in which criteria K4 ‐ Liabilities overdue to income and K5 ‐ Liabilities at least 60 days past due (simultaneously or separately), are absent. In the case of these combinations of entry criteria, relevant results are achieved compared to combination A, and are significantly correlated (B1: rK = 0.970; p < 0.01; B2: rK = 0.996; p < 0.01; C1: rK = 0.992; p < 0.01) with confirmed agreements in the distribution functions (B1: K-S = 0.251; p = 1; B2: K-S = 0.167; p = 1; C1: K-S = 0.251; p = 1) and a similar positive effect of the number of inhabitants on overall results. The order of district towns in all three cases, with a tolerance of two differences, is 100% consistent with the order of SD-TOPSIS with the inclusion of all 5 criteria in the analysis (combination A).
In the case of the upper quartile of evaluated municipalities, the agreement of both combinations is complete, i.e. the district town, which was ranked among the top 25% for combinations A, was also ranked in this group for combinations B1. These entities include Banská Štiavnica, Pezinok, Senec and Stropkov. The majority part is located in Central and Western Slovakia, with the exception of Stropkov and Medzilaborce, located in the Prešov self-governing region, i.e. in Eastern Slovakia (see Fig 22).
A similar situation, with the agreement of the identified subjects, can also be observed in the case of the lower quartile, i.e. 25% of the worst-rated district towns. Each of the district towns located in this group with the A combinations is also located in it if criterion K4 or K5 is not included (i.e. combinations B1, B2 and C1). Such entities include Čadca, Sobrance, Snina and Nitra. From the point of view of spatial distribution, we observe the dominant representation of Eastern Slovakia (7); see the following figure/map (Fig 23).
In the case of the SD-TOPSIS technique, combinations B1, B2 and C2, with the absence of criteria K4 and K5, are full replacements.
As a result, the multi-criteria evaluation of territorial self-government subjects (in our case, district towns) proved to be highly applicable. However, the result itself is largely determined by the structure and number of entry criteria. Based on the performed analyses described in the previous sections, we can state that there are significant differences resulting from the reduction in the number of entry criteria. Each such reduction had an impact on the overall results, but it is possible to find combinations that deviate from this conclusion (combinations B1, B2 and C2). They could be the subject of further research, or verification on another research sample.
Conclusions
As a result, the multi-criteria assessment of local government entities using TOPSIS method (in our case, district towns in Slovakia) proved to be highly applicable. However, the result itself is largely determined by the structure and number of criteria. On the basis of the analyses, we can observe significant differences resulting from a reduction in the number of criteria. Based on the results of our own research, we can conclude that, each criteria reduction had an impact on the overall results, but it is possible to find combinations that escape this conclusion (combinations B1, B2 and C2). These are the ones that could be the subject of further research or verification on a different research sample.
Several applications of MCDM methods in local government conditions work with a heterogeneous number of criteria [4, 5, 8, 15], especially in Slovakia [10–15]. However, none of these applications think about the number of criteria used. The novelty of the research findings is demonstrated on two levels:
- the use of 5 specific criteria that have not yet been processed using MCDM analysis methods,
- the introduction of the idea of reducing the number of criteria used in order to simplify the multi-criteria models used so far.
A similar approach to identifying opportunities for reducing the number of criteria can be applied across the private and public sectors. We see an opportunity in financial analysis, HR management, management assessment, logistics and emphasis on supplier selection, customer behavior, and others.
The results obtained should be seen with the following limitations:
- the simplest method (or one of the simplest methods) for determining the importance of criteria was used; it is based on the variability of the criteria in absolute terms,
- approaches to determining the importance of criteria are innumerable [41, 65–82], and different results can be expected with their application,
- it is the same with MCDM methods, the TOPSIS method is used, the use of which in the conditions of (Slovak) territorial self-government has been proven [4, 5, 8, 10–15].
References
- 1. Serrano Cinca C, Mar Molinero C, Bossi Queiroz A. The measurement of intangible assets in public sector using scaling techniques. Journal of Intellectual Capital. 2003; 4(2): 249–275.
- 2. Ter Bogt HJ, van Helden GJ, van Der Kolk B. Challenging the NPM ideas about performance management–Selectivity and differentiation in outcome-oriented performance budgeting. Financial Accountability & Management. 2015; 3(3): 287–315.
- 3. Summermatter L, Siegel JP. Defining Performance in Public Management: Defining Performance in Public Management: Variations over time and space. 2009. https://www.semanticscholar.org/paper/Defining-Performance-in-Public-Management%3A-over-and-SummermatterSiegel/9d604811759ce3e4b8e1d03f3f086ce31bbf8dd9.
- 4. Peková J, Jetmar M, Toth P. Veřejný sektor, teorie a praxe v ČR. Prague: Wolters Kluwer; 2019.
- 5. Muñoz FY, Martos LP, Pavón NP. Rethinking andalusian RIS3 strategy design through regional benchmarking. Journal of Regional Research. 2021; 51: 5–30.
- 6. Miceikiené A, Skauroné L, Krikštolaitis R. Assessment of the Financial Autonomy of Rural Municipalities. Economies. 2021; 9(3): 105.
- 7. Hallerberg M, Strauch R, Von Hagen J. The Design of Fiscal Rules and Forms of Governance in European Union Countries. 2004. https://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp419.pdf.
- 8. Selcuk Kilic H, Selcuk Yalcin A. Comparison of municipalities considering environmental sustainability via neutrosophic DEMATEL based TOPSIS. Socio-Economic Planning Sciences. 2021; 75: 100827.
- 9. Douša M. Trvalo udržateľný rozvoj miest slovenskej republiky. Košice: UPJŠ; 2021.
- 10. Mihaliková E. Finančná situácia a výkonnosť v samospráve. Košice: UPJŠ; 2021.
- 11. Papcunová V, Balážová E. Majetok obcí. Nitra: SAPV; 2006.
- 12. Kraftová I. Finanční analýza municipální firmy. Prague: C. H. Beck; 2002.
- 13. Vavrek R. Weight of Topsis Technique Parameter and its Impact on Assessment of Municipalities in Slovak Republic. Scientific Papers of the University of Pardubice. 2017; 24(39): 236–246.
- 14. Vavrek R, Adamišin P, Kotulič R. Multi-criteria evaluation of municipalities in Slovakia–Case study in selected districts. Polish Journal of Management Studies. 20217; 16(2): 290–301.
- 15. INEKO. Ako hospodária obce a VÚC. 2022. http://www.hospodarenieobci.sk/.
- 16. Zavadskas EK, Mardani A, Turskis Z, Jusoh A, Nor KM. Development of TOPSIS Method to Solve Complicated Decision-Making Problems–An Overview on Developments from 2000 to 2015. International Journal of Information Technology & Decision Making. 2016; 15(3): 645–682.
- 17. Maccrimmon K. Decision making Among Multiple-Attribute Alternatives: A Survey and Consolidated Approach. Santa Monica: RAND Corporation; 1968.
- 18. Brauers WK, Zavadskas EK. Robustness of the Multi-Objective MOORA Method with a Test for the Facilities Sector. Technological and Economic Development of Economy. 2009; 15(2): 352–375.
- 19. Hui CL, Masud A. Multiple Objective Decision Making—Methods and Applications. A State-of-the-Art Survey. Heidelberg: Berlin: Springer; 1979.
- 20. Guitouni A, Martel JM. Tentative Guidelines to Help Choosing an Appropriate MCDA Method. European Journal of Operational Research. 1998; 109(2): 501–521.
- 21. Zanakis S, Solomon A, Wishart N, Dublish S. Multi-Attribute Decision Making: A Simulation Comparison of Select Methods. European Journal of Operational Research. 1998; 107(3): 507–529.
- 22. Olson DL. Comparison of Three Multicriteria Methods to Predict Known Outcomes. European Journal of Operational Research. 2001; 130(3): 576–587.
- 23. Roubens M. Preference relations on actions and criteria in multicriteria decision making. European Journal of Operational Research. 1982; 10(1): 51–55.
- 24. Korhonen P. A hierarchical interactive method for ranking alternatives with multiple qualitative criteria. European Journal of Operational Research. 1986; 24(2): 265–276.
- 25. Korhonen P, Wallenius J, Zionts S. Solving the Discrete Multiple Criteria Problem Using Convex Cones. Management Science. 1984; 30(11): 1336–1345.
- 26. Marcotte O, Soland RM. An Interactive Branch-and-Bound Algorithm for Multiple Criteria Optimization. Management Science. 1986; 32(1): 61–75.
- 27. Olson DL, Fliedner G, Currie K. Comparison of the REMBRANDT system with analytic hierarchy process. European Journal of Operational Research. 1995; 82(3): 522–539.
- 28. Van Huylenbroeck G. The conflict analysis method: bridging the gap between ELECTRE, PROMETHEE and ORESTE. European Journal of Operational Research. 1995; 82(3): 490–502.
- 29. Vavrek R. Evaluation of the Impact of Selected Weighting Methods on the Results of the TOPSIS Technique. International Journal of Information Technology & Decision Making. 2019; 18(6): 1821–1843.
- 30. Ciardello F, Genovese A. A comparison between TOPSIS and SAW methods. Annals of Operation Research. 2023; 325(2): 967–994.
- 31. Opricovic S, Tzeng GH. Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European Journal of Operational Research. 2004; 156(2): 445–455.
- 32. Chang TY, Ku CCY. Fuzzy filtering ranking method for multi-criteria decision making. Computers & Industrial Engineering. 2021; 156: 107217.
- 33. Kumar P, Sarkar P. A comparison of the AHP and TOPSIS multi-criteria decision-making tools for prioritizing sub-watersheds using morphometric parameters’ analysis. Modeling Earth Systems and Environment. 2022. 8(3): 3973–3983.
- 34. Tramarico CL, Mizuno D, Salomon VAP, Marins FAS. Analytic hierarchy process and supply chain management: A bibliometric study. Procedia Computer Science. 2015; 55: 441–450.
- 35. Ilbahar E, Cebi S, Kahraman C. A state-of-the-art review on multi-attribute renewable energy decision making. Energy Strategy Reviews. 2019; 25: 18–33.
- 36. Rozentale L, Blumberga D. Methods to Evaluate Electricity Policy from Climate Perspective. Environmental and Climate Technologies. 2019; 23(2): 131–147.
- 37. Suharevska K, Blumberga D. Progress in Renewable Energy Technologies: Innovation Potential in Latvia. Environmental and Climate Technologies. 2019; 23(2): 47–63.
- 38. Siksnelyte I, Zavadskas EK, Bausys R, Streimikiene D. Implementation of EU energy policy priorities in the Baltic Sea Region countries: Sustainability assessment based on neutrosophic MULTIMOORA method. Energy Policy. 2019; 125: 90–102.
- 39. BDjordjević B, Krmac E. Evaluation of energy-environment efficiency of European transport sectors: Non-radial DEA and TOPSIS approach. Energies. 2019; 12(15): 2907.
- 40. Vavrek R, Bečica J. Capital City as a Factor of Multi-Criteria Decision Analysis- Application on Transport Companies in the Czech Republic. Mathematics. 2020; 8(10): 1765.
- 41. Vavrek R, Bečica J. Similarity of TOPSIS results based on criterion variability: Case study on public economic. PLoS ONE. 2022; 17(8): e0271951. pmid:35925960
- 42. Bečica J, Vavrek R, Galecka M, Smolny K. Application of Multi-Criteria Analysis on Theatres’ Efficiency–Czech and Polish Comparative Case Studies. Croatian and Comparative Public Administration. 2021; 21(3): 423–455.
- 43. Chang ChH Lin JJ, Lin JH Chiang MCh. Domestic open-end equity mutual fund performance evaluation using extended TOPSIS method with different distance approaches. Expert Systems with Applications. 2010; 37(6): 4642–4649.
- 44. Behzadian M, Khanmohammadi Otaghsara S, Yazdani M, Ignatius J. A state-of the-art survey of TOPSIS applications. Expert Systems with Applications. 2012; 39(17): 13051–13069.
- 45. Luan B, Yin R, Xu P, Wang X, Yang X, Zhang L, et al. Evaluating Green Stormwater Infrastructure strategies efficiencies in a rapidly urbanizing catchment using SWMM-based TOPSIS. Journal of Cleaner Production. 2019; 233: 680–691.
- 46. Ma L, Long J, Chen K, Tu S, Zhang Y, Liao L. Green growth efficiency of Chinese cities and its spatio-temporal pattern. Resources, Conservation and Recycling. 2019; 146: 441–451.
- 47. Wu Y, Xu C, Zhang B, Tao Y, Li X, Chu HF, et al. Sustainability performance assessment of wind power coupling hydrogen storage projects using a hybrid evaluation technique based on interval type-2 fuzzy set. Energy. 2019; 179: 1176–1190.
- 48. Lamba M, Munjal G, Gigras Y. Ranking of Classification Algorithm in Breast Cancer Based On Estrogen Receptor Using MCDM Technique. International Journal of Information Technology & Decision Making. 2023; 22(2): 803–827.
- 49. Jeni Seles Martina D, Deepa G. Operations on Multi-Valued Neutrosophic Matrices and Its Application to Neutrosophic Simplified-TOPSIS Method. International Journal of Information Technology & Decision Making. 2023; 22(1): 37–56.
- 50. Pavic Z, Novoselac V. Notes on TOPSIS Method. International Journal Of Engineering Research and General Science. 2013; 1(2): 5–12.
- 51. Kandakoglu A, Celik M, Akgun I. A multi-methodological approach for shipping registry selection in maritime transportation industry. Mathematical and Computer Modelling. 2008; 49: 586–597.
- 52. Shih H, Shyur H, Lee ES. An extension of TOPSIS for group decision making. Mathematical and Computer Modelling. 2007; 45: 801–813.
- 53. Bhutia PW, Phipon R. Application of AHP and TOPSIS Method for Supplier Selection Problem. Journal of Engineering. 2012; 2(10): 43–50.
- 54. Zyoud SH, Fuchs-Hanusch D. A bibliometric-based survey on AHP and TOPSIS techniques. Expert Systems with Applications. 2017; 8: 158–181.
- 55. Noryani M, Sapuan SM, Mastura MT. Multi-criteria decision-making tools for material selection of natural fibre composites: A review. Journal of Mechanical Engineering and Sciences. 2018; 12(1): 3330–3353.
- 56. Seyedmohammadi J, Sarmadian F, Jafarzadeh AA, Ghorbani MA, Shahbazi F. Application of SAW, TOPSIS and fuzzy TOPSIS models in cultivation priority planning for maize, rapeseed and soybean crops. Geoderma. 2017; 310: 178–190.
- 57. Dutta D, Dao SD, Martínez L, Goh M. An evolutionary strategic weight manipulation approach for multi-attribute decision making: TOPSIS method. International Journal of Approximate Reasoning. 2021; 129: 64–83.
- 58. Pekár J, Furková A. Prípadové štúdie z viackriteriálneho rozhodovania. Bratislava: Ekonóm; 2014.
- 59. Hatefi MA. A Typology Scheme for the Criteria Weighting Methods in MADM. International Journal of Information Technology & Decision Making. 2023; 22(4): 1439–1488.
- 60. Liu J, Yin Y. An integrated method for sustainable energy storing node optimization selection in China. Energy Conversion and Management. 2019; 199: 112049.
- 61. Keršuliene V, Zavadskas EK, Turskis Z. Selection of rational dispute resolution method by applying new step‐wise weight assessment ratio analysis (Swara). Journal of Business Economics & Management. 2010; 11(2): 243–258.
- 62. Kendall MG. Rank Correlation Methods. London: Griffin; 1970.
- 63. Fisher RA, Yates F. Statistical Tables for Biological, Agricultural and Medical Research. London: Oliver and Boyd; 1963.
- 64. Zavadskas EK. Multiple Criteria Evaluation of Technological Decisions of Construction. Moscow: Moscow Civil Engineering Institute; 1987.
- 65. Cambazoğlu S, Yal GP, Eker AM, Şen O, Akgün H. Geothermal resource assessment of the Gediz Graben utilizing TOPSIS methodology. Geothermics. 2019; 80: 92–102.
- 66. Diaz F, Cilinskis E. Use of Multi-Criteria TOPSIS Analysis to Define a Decarbonization Path in Colombia. Environmental and Climate Technologies. 2019; 23(2): 110–128.
- 67. Polikarpova I, Lauka D, Blumberga D, Vigants E. Multi-Criteria Analysis to Select Renewable Energy Solution for District Heating System. Environmental and Climate Technologies. 2019; 23(3): 101–109.
- 68. Indzere Z, Melvere M, Muizniece I, Blumberga D. The Evaluation of Factors Affecting Bioeconomy Development Using Transdisciplinary Approach. Environmental and Climate Technologies. 2019; 23(3): 360–369.
- 69. Wang D, Shi Y, Wan K. Integrated evaluation of the carrying capacities of mineral resource-based cities considering synergy between subsystems. Ecological Indicators. 2020; 108: 105701.
- 70. Paradowski B, Shekhovtsov A, Bączkiewicz A, Kizielewicz B, Sałabun W. Similarity Analysis of Methods for Objective Determination of Weights in Multi-Criteria Decision Support Systems. Symmetry. 2021; 13: 1874.
- 71. Ouerghi H, Mourali O, Zagrouba E. Non-subsampled shearlet transform based MRI and PET brain image fusion using simplified pulse coupled neural network and weight local features in YIQ colour space. IET Image Processing. 2018; 12(10): 1873.
- 72. Yuan J, Luo X. Regional energy security performance evaluation in China using MTGS and SPA-TOPSIS. Science of The Total Environment. 2019; 696: 133817. pmid:31446291
- 73. Zhang L, Zhang L, Xu Y, Zhou P, Yeh CH. Evaluating urban land use efficiency with interacting criteria: An empirical study of cities in Jiangsu China. Land Use Policy. 2020; 90: 104292.
- 74. Vavrek R, Chovancová J. Assessment of economic and environmental energy performance of EU countries using CV-TOPSIS technique. Ecological Indicators. 2019; 106: 105519.
- 75. Singla A, Sing Ahuja I, Sing Sethi A. Comparative Analysis of Technology Push Strategies Influencing Sustainable Development in Manufacturing Industries Using Topsis and Vikor Technique. International Journal for Quality Research. 2018; 12(1): 129–146.
- 76. Pekkaya M, Zilifli V. A Preliminary Study on Generating Criteria Priorities Series via AHP for Decision Process of Commercial Credit Applications in Turkey. International Journal of Information Technology & Decision Making. 2023; 22(6): 1843–1846.
- 77. Więckowski J, Kizielewicz B, Paradowski B, Shekhovtsov A, Sałabun W. Application of Multi-Criteria Decision Analysis to Identify Global and Local Importance Weights of Decision Criteria. International Journal of Information Technology & Decision Making. 2023; 22(6): 1867–1892.
- 78. Hatefi MA. A Typology Scheme for the Criteria Weighting Methods in MADM, International Journal of Information Technology & Decision Making. 2023; 22(4): 1439–1488.
- 79. Ay S, Can GF, Toktas P. A Novel Multi-Criteria Decision-Making Approach Proposal Based On Kemira-M With Four Criteria Groups International Journal of Information Technology & Decision Making. 2023; 22(3): 991–1032.
- 80. Davoudabadi R, Meysam Mousavi S, Zavadskas EK, Dorfeshan Y. Introducing MOWSCER Method for Multiple Criteria Group Decision-Making: A New Method of Weighting in the Structure of Cause and Effect Relationships. International Journal of Information Technology & Decision Making. 2023; 22(2): 641–677.
- 81. Aytaç Adalı E, Öztaş T, Özçil A, Zeynep Öztaş T, Tuş A. A New Multi-Criteria Decision-Making Method Under Neutrosophic Environment: ARAS Method With Single-Valued Neutrosophic Numbers. International Journal of Information Technology & Decision Making. 2023; 22(1): 57–87.
- 82. Dawood KA, Zaidan AA, Sharif KY, Ghani AA, Zulzalil H, Zaidan BB. Novel Multi-Perspective Usability Evaluation Framework for Selection of Open Source Software Based on BWM and Group VIKOR Techniques. International Journal of Information Technology & Decision Making. 2023; 22(1): 187–277.
- 83. Yalcin E, Unlu U. A Multi-Criteria Performance Analysis of Initial Public Offering (IPO) Firms Using Critic and Vikor Methods. Technological and Economic Development of Economy. 2018; 24(2): 534–560. doi:https://doi.org/10.3846/20294913.2016.1213201.
- 84. Anděl J. Základy matematické statistiky. Prague: Matfyz; 2007.
- 85.
Cyhelský L, Kahounová J, Hindls R. Elementární statistická analýza. Prague: Management Press; 2001.
- 86. Hindls R, Hronová S, Seger J. Statistika pro economy. Prague: Professional Publishing; 2002.
- 87. Pacáková V. et al. Štatistické metódy pre ekonómov. Bratislava: Iura Edition; 2009.
- 88. Vavrek R. Effective Management Analysis of Financial Resources of Municipalities in SR and CR. Presov: University of Presov in Presov; 2015.
- 89. Shapiro SS, Wilk MB. An Analysis of Variance Test for Normality (Complete Samples). Biometrika. 1965; 52: 591–611.
- 90.
Pearson’s Correlation Coefficient. In: Kirch W. (eds) Encyclopedia of Public Health. Dordrech: Springer, Dordrecht; 2008.
- 91. Test Kolmogorov–Smirnov. In: The Concise Encyclopedia of Statistics. New York; 2008.
- 92. Levene H. Robust Tests for Equality of Variances. In: Contributions to Probability and Statistics. Palo Alto: Stanford University Press; 1960.
- 93. Kusendová D, Solčianska J- Testovanie priestorovej autokorelácie nezamestnanosti absolventov vysokých škôl okresov Slovenska. http://gis.vsb.cz/GIS_Ostrava/GIS_Ova_2007/sbornik/Referaty/Sekce6/solcianska_kusendova_po_recenzii.pdf.
- 94. Zhang T, Lin G. On Moran’s I coefficient under heterogeneity. Computational Statistics & Data Analysis. 2016; 95: 83–94.
- 95. Slavík V, Grác R, Klobučník M. Priestorová autokorelácia–metóda vymedzovania a klasifikácie regiónov v kontexte sociálno-ekonomickej regionalizácie Slovenskej republiky. Sociológia ‐ Slovak Sociological Review. 2011; 43(2): 183–204.
- 96. Vavrek R. Metodológia, graf a číslo. Košice: Equlibria; 2022.