Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Similarity of TOPSIS results based on criterion variability: Case study on public economic

Abstract

In the real world, acceptance of a decision is conditional on the availability of a great volume of data. Selection of a suitable solution on the basis of this data represents a problem that multi-criterial methods (MCDM) are applied to. The issue of which of these should be favoured during their use involves a specification of the importance of the assessed criteria. The goal of the presented research is to quantify the differences (symmetry) in assessment using selected MCDM methods (Technique for Order of Preference by Similarity to Ideal Solution–TOPSIS), while applying an absolute and relative variability of the assessed criteria to a determination of their importance. The obtained results indicate that the order of the assessed subject (alternative) is not directly influenced by the method of determining the variability of the assessed criteria. We can also state that the degree of concurrence in the order of application of the TOPSIS technique, in combination with both approaches expressed by the Jaccard index, is relatively low.

Introduction

At a time of increasing global competition, which we do not identify as just the 21st century, it is necessary to pay increasing attention to the effective expenditure of funds [1, 2] or the identification and selection of alternative sources [3]. Decisions on the basis of multiple criteria are gaining in popularity, and the application of this method can be found in various areas of the public and private sectors. The problem that multi-criteria decision-making methods (MCDM) are engaged in resolving is to find and assess the best alternative from the available options [4]. According to [5], these methods represent the process of selecting one alternative from the available options on the basis of a pre-defined set of criteria, which are usually of differing importance. Their purpose is to combine selected criteria and the method of determining their importance into one assessment indicator [6]. This takes into consideration the preferences of the decision-maker, and therefore the actual result, order or recommendation can be modified depending on these preferences [7]. We can encounter a wide range of MCDM methods in practice, as for example PROMETHEE–Preference Ranking Organization Method for Enrichment Evaluations [8], ELECTRE–Elimination and Choice Expressing the Reality [9], TOPSIS–Technique for Order of Preference by Similarity to Ideal Solution [10], VIKOR–Vlse Kriterijumska Optimizacija Kompromisno Resenje [11], IDRA–Intercriteria Decision Rule Approach [12], EVAMIX–Evaluation of Mixed Data [13], and others. According to [14], despite the large number of methods, there is no “perfect” method which could be used to resolve all problems.

Consequently, multi-criteria methods can be classified into two main categories: the discrete MCDM or MADM (Multi-Attribute Decision-Making) and continuous multi-objective, decision-making (MODM) optimization methods [15]. In a more complex manner, we can find a division into 4 groups [16, 17]. One of the simplest taxonomies including the TOPSIS technique is offered by [18], see the following Fig 1:

The presented research focuses on evaluation of 72 subjects (district towns) using 6 unique criteria. The TOPSIS technique, which is an appropriate tool for decision-making on the basis of incomplete data, according to [19], is therefore used for this purpose. According to [20, 21], the scope of the chosen data is not determinative for use of this method, i.e., data of any scope can be used. This method proposes the minimization of the distance with respect to the ideal and, simultaneously, the maximization of the distance with respect to the anti-ideal [22]. The strong points of this method include the directness and simplicity of calculations [22, 23], and the ability to work with all types of criteria [21] and complexity [24]. [20, 21] consider the absence of the option to allocate weights to the monitored criteria and the absence of consistent supervision by the decision-maker to be the main disadvantages of the TOPSIS method. For this reason, this method depends on the process of establishing the importance of various attributes with regard to achieving the specified goal, which is also the area of interest of the presented research (as it is possible to choose methods according to the purpose of one’s own research).

The aim of this paper is to quantify the differences (symmetry) in assessment using the TOPSIS technique while using an absolute and relative variability of the assessed criteria to a determination of their importance. The selected Coefficient of Variance (CV) and Standard Deviation (SD) methods were chosen for this purpose, i.e., a relative and absolute assessment of the variability of the assessed criteria. Both represent moment characteristics of variability, but their calculation itself is different. The actual case study is preceded by an execution of the theoretical foundation of the MCDM issue, focusing on the TOPSIS technique and the methods of determining the importance of the assessed criteria. The research sample, criteria from the aspect of public economics, is described within the terms of the methodology. Attention is also focused on determining importance, as determined by means of the two aforementioned methods and the mechanism of the mathematical-statistical methods used. In the conclusion, the obtained results are summarised and potential future research is simultaneously outlined.

The TOPSIS technique as a multi-criteria assessment tool

TOPSIS is one of the basic methods for multi-criteria decision-making, and its primary application is for resolving various types of decision-making issues. This method is one of the most frequently used methods, according to [25, 26], who also cite the above-mentioned AHP, ANP or PROMETHEE methods as an alternative. A summary of its application is described, for example, by [27], who registered the annually increasing number of incidents of research/articles in which the application of TOPSIS and other techniques can be found. Selection of the TOPSIS method for the purposes of our research was based on its previous successful use in resolving decision-making problems of a similar nature. Its application can be found in the environmental [2830], transport [31, 32], logistical [33] and many other fields (see [34, 35]).

[27] expresses the principle of the TOPSIS technique using Fig 2, to which [36] also refers during a description of this method. Each white ball represents a concrete alternative, i.e., one assessed subject. The grey ball represents a Negative Ideal Solution (NIS) alternative, i.e., a real or hypothetical alternative (subject) with the worst values of the individual criteria. The black ball represents a Positive Ideal Solution (PIS) alternative, i.e., a real or hypothetical alternative (subject) with the best values of the individual criteria. The best rated is the alternative (one of the white balls) which is the furthest away from the grey ball (NIS) and also the closest to the black ball (PIS). For example, [3739] discuss an actual calculation of the TOPSIS technique, during which time the individual steps are as follows:

  1. In the first step, one needs to create a basis for the decision-making matrix, consisting of individually assessed subjects (alternatives) and previously defined criteria: (1)
    where: Ai = i(th) alternative
       xij = value of the j(th) criterion reached by the i(th) alternative
  2. In practice, the assessed criteria may have various parameters (expressed by the various level-of-moment characteristics), for the assessment of which the matrix is standardised in the subsequent step: (2)
    where: rij = normalized value of the j(th) criterion
       xij = value of the j(th) criterion reached by the i(th) alternative
  3. The decision-maker’s preferences are taken into consideration in the third step of the calculation, which represents the application of the results of the selected method for determining the importance of the assessed criteria: (3)
    where: vij = weighted normalized value
       wij = criterion weight
       rij = normalized value
  4. Given the fact that the TOPSIS technique works with Euclidean distance, the Positive Ideal Solution (PIS) and Negative Ideal Solution (NIS) must be identified: (4)
    where: Hj = PIS
       Dj = NIS
  5. After identification of these two alternatives, the Euclidean distance between each real alternative (subject) and the mentioned PIS and NIS alternatives, is calculated: (5)
    where: d+ = distance from PIS
       d- = distance from NIS
  6. The resulting assessment parameter is the relative distance to the PIS alternative, by means of which the subjects are assessed: (6)
    where: ci = relative distance from PIS

Within the terms of each MCDM method, one of the most important steps in the calculation is to determine the importance of the assessed criteria; this process takes place in the third step of the calculation for the TOPSIS technique. The approach to resolving this issue is discussed in the following section.

Variability as a factor for determining the importance of criteria

In general, it can be stated that the result of applying the MCDM method is directly determined by the decision-maker and also by, among other things, which approach or method he selects to determine the importance of individual criteria. One of the classifications of these methods is presented by [40], who identifies 2 groups of methods. [39] works with 3 groups and [41] uses 4 groups of approaches for classifying weights into 4 groups, which represents an expansion of the preceding classifications, during which time the groups in question are as follows:

  1. subjective,
  2. expert,
  3. objective,
  4. integrated.

Subjective methods reflect the personality of the decision-maker and his individual preferences (the weight of the indicator is determined on the basis of subjective opinion). Expert assessment is executed by a smaller number of experts in the specific field, during which time the previous application is described by [4244]. The Fuller method, or Fuller triangle, is the most commonly used of these groups of methods. The third group, i.e., the group of objective methods, assigns a weight to individual criteria on the basis of a previously determined mathematical model unique to each method. This means that the decision-maker does not have direct influence on determining the importance of criteria; a selection is made depending on his preference of the properties of the used data, e.g., according to variability or relations between criteria. The last group is integrated methods, which represent a combination of the approaches described above. The group of objective methods (third group) can include methods such as the Mean Weight method [45, 46], Standard Deviation method [47], Mahalanobis-Taguchi System method [48], λ bi-capacity model [49], Coefficient of Variance method [30], and many others (see [50, 51]). Variability can also be found in research in other forms, e.g., as a moment characteristic [52] or control CV charts [53, 54], or as a method for determining the importance of the assessed criteria [55]. We work with the following two methods for the purposes of this research:

  • method utilising relative variability—the Coefficient of Variance method (CV): (7)
    where: CVj = coefficient of variance of the j(th) criterion
  • method utilising absolute variability—the Standard Deviation method (SD): (8)
    where: SDj = standard deviation of the j(th) criterion

Research methodology

The purpose of the presented paper is to quantify the differences (symmetry) in assessment using the TOPSIS technique, while applying an absolute and relative variability of the assessed criteria to a determination of their importance. The fulfilment of this purpose should lead to the solution of the problem posed by the absence of a comparison of this type (with a finite number of alternatives, criteria and external determination of criteria importance using selected methods).

The case study is realised within the environment of public economics, in the context of which the research sample and monitored criteria are described. The following section represents a description of the research sample and criteria and an assessment of the importance of selected criteria, utilising two methods for determining their importance, namely the CV and SD methods. The mechanisms of the mathematical-statistical methods used are described in the last sub-section.

Research sample and criteria from the aspect of the public economy

The research sample for the presented research is a sample of all district towns in the Czech Republic (72 subjects), which are reported on an international scale within the terms of statistical surveys as natural centres of the LAU1 statistical unit (district centre). The specified statistical units originated within the territory of what is now the Czech Republic in the time of the Austro-Hungarian Empire, under the rule of the Habsburg Monarchy, after acceptance of the so-called Kroměříž Constitution in 1848, and their number was gradually modified in the following decades. Within the terms of analysis, the cost aspect of the sector classification of the budgets of a total of 72 district towns, which forms the full list of towns that are the seats of a district in the Czech Republic, is examined. Although a total of 77 districts (LAU1) are registered in the Czech Republic in relation to official statistics, in practice, the administrative territory of a total of 5 districts (Prague-East, Prague-West, Brno–Country, Plzeň –South and Plzeň-North) is actually the relevant territory of one of three major cities in the Czech Republic (Prague, Brno and Plzeň), and these cities are actually the seats of a multiple number of districts. The areas of the districts remained valid in the Czech Republic even after the extensive reforms to the public administration system in 2003, when their actual budgets (the budgets of district authorities) were abolished, but these administrative centres remained valid for the requirements of courts of law, the police, archives, employment offices, etc. At present, they are primarily utilised as a basic statistical unit in a territory, while the official district code book is created and maintained by the Czech Statistical Office.

The district is a part of the public administration system and can be viewed from various angles. Understanding the entire scope and importance of the function of self-governing municipalities, and subsequently the entire public administration system within a defined territory (state, region, municipality), requires a multi-disciplinary approach [56, 57]. This is applied by administrative science [58], which also defines the approaches to the provision of public services, which are defined by content, tasks and specific goals, and which usually differ between individual public administration units according to the defined area and actual scope of provision [59]. On the basis of the Czech constitution, a municipality is considered to be a basic territorial self-governing community of citizens, forming a territorial unit, and is defined by a border or the cadastral area of the municipality. According to Act No. 128/2000 Sb., on municipalities, each municipality in the Czech Republic should assure the universal development of its territory and meet the needs of its citizens. These needs include public services, which are provided to the residents entirely, or for partial payment, are based on acknowledged public interests, and may differ depending on the defined territory (state, region, municipality) of their provision [60, 61]. In practice, the specific needs of citizens are never specified in detail, and the management of the municipality makes decisions at its deliberation. This results in situations where municipalities of the same size report different results to each other, even on a longer time scale. Differences may be evident between municipalities from the income aspect [62] of the budget (typically on the level of non-tax and capital revenue) [63], particularly in relation to individual cost categories [64]. Income and costs are individual in the budgets of municipalities, but the contents are standardised according to the valid budget composition, which is specified by Decree of the Ministry of Finance of the Czech Republic No. 323/2002 Sb. Act No. 128/2000 Sb., on municipalities, which also stipulates that during execution of their tasks, municipalities must endeavour to protect the public interest. The valid legislation (particularly the constitution of the Czech Republic) indicates that the municipality is a public corporation with its own budget and property, which it disposes of on the basis of adherence to specific laws for decisions made by elected members of the municipal council. Municipalities in the Czech Republic enter into legal relations in their own name and bear the responsibilities arising from these relations. Each part of the territory of the Czech Republic is part of the territory of one of a total of 6,258 self-governing municipalities, which are typical in that they have one or more cadastral areas and a council elected by the citizens of this municipality.

The foundations of the Czech municipal system are based on operation within a so-called mixed system, during which the municipalities, as local self-governing units, have their own authority (self-government) and transferred authority. The state may only intervene in independent (self-government) authority if required by law, and only in a manner stipulated by the law. The advantage of independent authority is that municipalities are able to cooperate when conducting activities. The transferred authority of municipalities is essentially de-concentrated state administration entrusted to municipalities by laws on the basis of the provisions of Article 105 of the Czech constitution, which states that execution of state administration can be entrusted to self-governing bodies only if stipulated by the law. This therefore concerns indirect state administration, in which the municipality appears similarly to purely executive state administration bodies, and the realised activity is of a sub-legal, executive and ordered nature. In relation to transferred authority, municipalities in the Czech Republic follow valid laws, and the result is state assistance during execution of matters within its jurisdiction and powers, during which time they must follow not only the legal regulations to the degree stipulated by the law, but also government resolutions and guidelines from central administrative authorities. With regard to the mixed system for local public administration, it can be stated that the complexity of the existing system of organisation of territorial public administration in the Czech Republic is also substantially increased by the common regulations for individual areas, in relation to both independent and transferred authority. From the viewpoint of the ordinary citizen, independent and transferred authority subsequently intermingles and overlaps in municipalities. Within the terms of this paper, changes to the cost structure in individual budgets during the period from 2010–2020 for all district towns in the Czech Republic, calculated per permanently residing citizen in the relevant year and registered in the specific territory, are assessed. Classification of the budget components by sector complexly covers all the executed expenses of the selected sample of municipalities. Expenses are classified by individual common attribute into sectors (agriculture, industry, services for residents, social matters, safety and public administration) according to the sector classification of the budget, which is given by the valid decree of the Ministry of Finance of the Czech Republic on budget composition.

Assessed criteria from the aspect of the public economy

Within the terms of assessment of the budgets of the above-mentioned district towns, the authors of this paper selected individual categories of costs depending on sector classification of the budget composition, which is binding for all municipalities in the Czech Republic on the basis of the valid legislation, specifically:

  • R1 – Agriculture, forestry and fishing (in CZK per capita),
  • R2 – Industrial and other economic sectors (in CZK per capita),
  • R3 – Services for residents (in CZK per capita),
  • R4 – Social matters and employment policy (in CZK per capita),
  • R5 – State security and legal protection (in CZK per capita),
  • R6 – General public administration and services (in CZK per capita).

While the scope of a great number of activities under independent authority depends on decisions made by the municipality and on local conditions, the scope of their execution is not clearly determined in the case of transferred authority. In relation to sector classification of the budget composition (R1 –R6), this concerns expenses for public administration, in which we also classify administration regarding the hearing of offences against public order in state administration and in territorial self-government, offences against public order, offences against property, and offences against co-existence between citizens (unless these were committed by the breach of special legal regulations on road traffic), as well as offences in the sphere of the seeking out, protection, use and further development of natural therapeutic resources, including sources of natural mineral water, and spa locations. The municipal authorities of municipalities with expanded authority, which also includes the district towns assessed below, hear offences in matters they administer, along with other offences, if other administrative bodies do not have the authority to hear them.

How accurately it is possible to correctly determine the distribution of a municipality’s expenses throughout individual sectors is the subject of assessment, and the criteria for the requirements of multi-criteria assessment are therefore not of an absolute or relative value. Year-on-year changes, or more precisely progress, which is calculated as follows, is assessed.

The importance of the assessed criteria from the aspect of their variability

As specified above, one approach to determining the importance of criteria is based on the variability of the data/criteria. For the requirements of the presented research, relative variability is recorded using the CV (Coefficient of Variance) method, and absolute variability is recorded using the SD (Standard Deviation) method. The method of quantifying variability was expressed in the differences between the allocated weights (importance) recorded in Fig 3.

thumbnail
Fig 3. Average importance of the assessed criteria (whisker: max—min).

https://doi.org/10.1371/journal.pone.0271951.g003

There are no statistically significant differences in importance from the aspect of individual criteria (during assessment of the distribution function, mean value–median and standard deviation). The exception to this rule is the third criterion (R3), where the obtained sets of weights have a different distribution function (K-SR3 = 0.8; p < 0.01) and mean value (WR3 = 12; p < 0.01). With the exception of the period of 2015/2016, the standard deviation is homogeneous (LEEP6 = 5.3405; p < 0.05). Differences in the distribution function across individual years are not statistically significant, and the same applies in the case of mean values–medians. However, differences can be observed in selected moment characteristics, e.g., in the variation range or minimum and maximum values, see Table 1.

thumbnail
Table 1. Comparison of selected moment characteristics of criteria weights (CV vs. SD).

https://doi.org/10.1371/journal.pone.0271951.t001

The mean value of the allocated weights is mostly higher in the case of the measurement of relative variability, i.e., using the CV method. Using this method, the lowest importance of 4 out of 6 monitored criteria (R2 –R5) is higher, during which time in absolute results we can observe a higher importance during use of the SD method. The assessment of the variation range, which is higher for all criteria, is clearly the result of the use of the SD method.

The basic prerequisite for quantification of the results of variability measurement on the results of multi-criteria assessment is therefore partially proven in the heterogeneous determination of the importance of individual criteria. The result of these differences is quantified in the analytical section, i.e., the third chapter.

Mathematical-statistical methods

The basic method used in the executed analysis is the TOPSIS technique, the calculation of which is described in detail in one of the previous sections. The results, in combination with the CV and SD methods, are subsequently verified and interpreted using selected moment characteristics and the following methods:

  • Levene test (9)
    where: k = number of values of the observed categorical variable
       N = number of observations
       Ni = number of observations in the i(th) group
       Yij = measured value of the j(th) unit of the i(th) group
        = average value of the i(th) group
        = median of the i(th) group
       Z = average of groups Zij
       Zi. = average of Zij for the i(th) group
  • Kruskal-Wallis test (10)
    where: n = number of observations and sample size
       ni = number of observations in the i(th) group
        = total number of orders in the i(th) group
  • Jaccard index (11)
    where: A = set 1 (list of subjects)
       B = set 2 (list of subjects)

The first two tests represent commonly used non-parametric tests for testing homoscedasticity, or agreement between the mean values. The Jaccard index is one of the most frequently used indices [6567] and is used to assess the order of the results of CV-TOPSIS and SD-TOPSIS. All analyses are executed in MS Excel, Statistica 13.4 and Statgraphics XVIII software.

Results

In the preceding section, differences in importance determined on the basis of the CV and SD methods, i.e., on the basis of relative and absolute assessment of the variability of monitored criteria, among other differences, were established. In this section we gradually present the results of the multi-criteria assessment using the TOPSIS technique (in combination with both the aforementioned approaches), while focusing on quantification of the established differences (see Fig 4).

Results from application of the CV-TOPSIS method show significant differences across the entire assessed period (see Fig 4). Subjects with considerably better assessment, i.e., outliers/extreme observations, can be found in each of the evaluated periods (EP). This fact became evident by rejection of the homoscedasticity of these results (LE = 2.0335; p < 0.05) and also by confirmation of the differences between mean values (Q = 350.115; p < 0.01). At the same time, on each occasion the results are positively skewed (min. γ1(EP8) = 1.062), and we therefore observe that a majority of the subjects have an above-average assessment.

Assessment based on the SD-TOPSIS method shows significant differences, which can be indicated on the basis of Fig 5. Statistically significant results became apparent from the aspect of the assessment of homoscedasticity (LE = 2.366; p < 0.05) and also during a comparison of mean values (Q = 492.664; p < 0.01). During each of the evaluated periods (EP), we observe multiple outliers/extreme observations, along with a high concentration of the majority around the expected value (max. γ2(EP5) = 44.087).

Identification of differences in the obtained results

The results of the multi-criteria assessment outlined above change substantially during the monitored period of 2010–2020. Although differences can be observed from multiple angles (e.g., through individual moment characteristics), this does not automatically mean confirmation of the differences between the results of the applied CV and SD methods. Assessment of these differences is carried out on the basis of absolute values (Fig 6), the order of individual subjects (Fig 7), and also the assessment of both aspects simultaneously (Fig 8). In the last step, agreement on the level of individual subjects is monitored by application of the Jaccard index (Fig 9).

thumbnail
Fig 6. Average differences in the absolute results of the TOPSIS technique (CV vs. SD).

https://doi.org/10.1371/journal.pone.0271951.g006

thumbnail
Fig 7. Comparison of the order of individual subjects on the basis of the results of the TOPSIS method (CV vs. SD).

https://doi.org/10.1371/journal.pone.0271951.g007

thumbnail
Fig 8. Structure of individual subjects on the basis of the results of the TOPSIS method (CV vs. SD).

https://doi.org/10.1371/journal.pone.0271951.g008

thumbnail
Fig 9. Comparison of the agreement of the results of the TOPSIS method using the Jaccard index (CV vs. SD).

https://doi.org/10.1371/journal.pone.0271951.g009

When monitoring absolute differences, we can confirm the dominance of the results of applying the CV method (see Fig 6). The results acquired in this manner are mostly higher/better, with the exception of EP5, which is given by the great evenness of the results of the SD-TOPSIS method in this period. During most of the evaluated period (EP), these results are better by a 0.129 to 0.90 relative distance to the PIS alternative (ci). In the case of better results using the SD method, their frequency is lower, but is also linked to a greater difference.

The differences arising from the use of various methodologies must also be considered in the context of the results of other assessed subjects, which is portrayed in Fig 7. In 44.44% of the cases for the entire monitored period, the placement of the assessed subject on the basis of the CV-TOPSIS method is better. Better results in relation to application of the other method (SD-TOPSIS) can be observed in 42.50% of cases, i.e., the order did not change in just 13.06% of the cases. From this aspect, it is impossible to identify one of these methods as dominant.

As summarised by Fig 8, in the majority of cases we observe better results if relative variability is used (CV method) to determine the importance of the assessed criteria. Paradoxically, these better results are linked to a decline in overall order (50% of cases), which we credit mainly to the specific situation during the EP5 assessed period.

From the aspect of individual subjects and their placement, we can state that the degree of agreement in their order on the basis of assessment using both methods is relatively low. The position of individual subjects is significantly heterogeneous during the individually assessed periods, when the degree of agreement quantified using the Jaccard index ranged between 4.17% and 20.83%. With regard to the methodology of this index, the “strength” of the heterogeneity determined in this manner cannot be interpreted in more detail (because deviation by one place in the overall order is sufficient).

Conclusions

Making decisions on the basis of multiple criteria is currently a frequently used approach that can be broadly employed in practice. Within the terms of execution of this method, adequate attention must be devoted to determining the importance of the assessed criteria; this can be approached in different ways. The submitted research presents the application of two methods for determining the importance of criteria, which are based on variability expressed as relative (CV method) or absolute (SD method), in relation to the TOPSIS method. The described contents of expenses by sector classification of budget elements clearly indicates that these cover the entire range of expenses of all municipalities within the terms of the budget. However, specific expenses may differ between municipalities on a year-on-year basis, which is caused by the different requirements for financing or ensuring specific public services for the residents in the specific territory. Application of the above-mentioned methods resulted in the following findings from a methodological aspect:

  • during application of the SD method, higher absolute differences in determined importance were measured in the case of all the monitored criteria,
  • during the use of relative variability to determine the importance of the assessed criteria (CV method), the subjects are usually rated more highly,
  • the order of the assessed subject is not directly influenced by selection of the CV or SD method,
  • the degree of similarity expressed by the Jaccard index is relatively low.

In general, it is necessary to respect the differences arising from both perspectives when determining the importance of the evaluated criteria (absolute and relative). The absolute view cannot be replaced by a relative one, or vice versa. Methods based on the relative importance of the criteria may indicate a better evaluation of the subjects, but without affecting their overall ranking.

In the future, the plan is to follow up on the obtained results by verification using other MCDM methods, such as VIKOR or PROMETHEE, and to verify these conclusions.

References

  1. 1. Hsieh JY, Fu K. Testing Municipal Reinvention on the Price of Municipal Governance. Lex localis–Journal of Local Self-Government. 2014; 12(2): 289–310.
  2. 2. Pevcin P. Productivity Changes in Slovenian Urban Municipalities. Lex localis–Journal of Local Self-Government. 2014; 12(3): 417–429.
  3. 3. Wu CM, Hsieh CL, Chang KL. A Hybrid Multiple Criteria Decision Making Model for Supplier Selection. Mathematical Problems in Engineering. 2013; 8: 324283.
  4. 4. Jahanshahloo GR, Hosseinzadeh Lotfi F, Izadikhah M. Extension of the TOPSIS method for decision-making problem with fuzzy data. Applied Mathematics and Computation. 2016; 181(2): 1544–1551.
  5. 5. Stanujkic D, Dordevic B, Dordevic M. Comparative Analysis of Some Prominent MCDM Methods: A Case of Ranking Serbian Banks. Serbian Journal of Management. 2013; 8(2): 213–241.
  6. 6. Čereška A, Zavadskas EK, Bucinskas V, Podvezko V, Sutinys E. Analysis of Steel Wire Rope Diagnostic Data Applying Multi-Criteria Methods. Applied Sciences. 2018; 8(2): 260.
  7. 7. Cinelli M, Kadzinski M, Gonzalez M, Słowinski R. How to support the application of multiple criteria decision analysis? Let us start with a comprehensive taxonomy. Omega. 2020; 96: 102261. pmid:33746337
  8. 8. Brans JP, Vincke P. Note—A Preference Ranking Organisation Method: The Promethee Method for Multiple Criteria Decision Making. Management science. 1985; 31(6): 647–656.
  9. 9. Roy B. Classement et Choix en Présence de Points de Vue Multiples. Revue fran. d’informat. et de recher. operation. 1968; 2(8): 57–75.
  10. 10. Hwang CL, Yoon K. Methods for multiple attribute decision making, Berlin: Springer; 1981.
  11. 11. Duckstein L, Opricovic S. Multiobjective optimization in river basin development. Water resources research. 1980; 16(1): 14–20.
  12. 12. Greco S. A new PCCA method: Idra. European Journal of Oper. Res. 1997; 98(3): 587–601.
  13. 13. Voogd H. Multicriteria evaluation with mixed qualitative and quantitative data. Environment and Plannning. 1982; 9(2): 221–236.
  14. 14. Guitouni A, Martel JM. Tentative guidelines to help choosing an appropriate MCDA method. European Journal of Operational Research. 1998; 109: 501–521.
  15. 15. Turskis Z. Multi-attribute contractors ranking method by applying ordering of feasible alternatives of solutions in terms of preferability technique. Technological and Economic Development of Economy. 2008; 14(2): 224–239,
  16. 16. Ustinovichius L, Zavadskas EK, Podvezko V. Application of a quantitative multiple criteria decision making (MCDM-1) approach to the analysis of investments in construction. Control and Cybernetics. 2007; 36(1): 251–268.
  17. 17. Brauers WKM, Zavadskas EK, Peldschus F, Turskis Z. Multi-objective decision-making for road design. Transport. 2008; 23(3): 183–193.
  18. 18. Jafaryeganeh H, Ventura M, Guedes Soares C. Effect of normalization techniques in multi-criteria decision making methods for the design of ship internal layout from a Pareto optimal set. Structural and Multidisciplinary Optimization volume. 2020; 62: 1849–1863.
  19. 19. Pavic Z, Novoselac V. Notes on TOPSIS Method. International Journal of Research in Engineering and Science. 2013; 1(2): 5–12.
  20. 20. Kandakoglu A, Celik M, Akgun I. A multi-methodological approach for shipping registry selection in maritime transportation industry. Mathematical and Computer Modelling. 2009; 49: 586–597.
  21. 21. Shih H, Shyur H, Lee ES. An extension of TOPSIS for group decision making. Mathematical and Computer Modelling. 2007; 45: 801–813.
  22. 22. Diaz-Balteiro C, González-Pachón J, Romero C. Measuring systems sustainability with multi-criteria methods: A critical review. European Journal of Operational Research. 2017; 258: 607–616.
  23. 23. Bhutia PW, Phipon R. Application of AHP and TOPSIS Method for Supplier Selection Problem. Journal of Engineering. 2012; 2(10): 43–50.
  24. 24. Zyoud SH, Fuchs-Hanusch D. A bibliometric-based survey on AHP and TOPSIS techniques. Expert Systems with Applications. 2017; 78: 158–181.
  25. 25. Noryani M, Sapuan SM, Mastura MT. Multi-criteria decision-making tools for material selection of natural fibre composites: A review. Journal of Mechanical Engineering and Sciences. 2018; 12(1): 3330–3353.
  26. 26. Zavadskas EK, Mardani A, Turskis Z, Jusoh A, Nor KM. Development of TOPSIS Method to Solve Complicated Decision-Making Problems—An Overview on Developments from 2000 to 2015. International Journal of Information Technology and Decision Making. 2016; 15(3): 645–682.
  27. 27. Tramarico CL, Mizuno D, Antonio V, Salomon P, Augusto F, Marins S. Analytic Hierarchy Process and Supply Chain Management: A Bibliometric Study. Procedia Computer Science. 2015; 55: 441–450.
  28. 28. Rozentale L, Blumberga D. Methods to Evaluate Electricity Policy from Climate Perspective. Environmental and Climate Technologies. 2019; 23(2): 131–147.
  29. 29. Suharevska K, Blumberga D. Progress in Renewable Energy Technologies: Innovation Potential in Latvia. Environmental and Climate Technologies. 2019; 23(2): 47–63.
  30. 30. Vavrek R, Chovancová J. Assessment of economic and environmental energy performance of EU countries using CV-TOPSIS technique. Ecological Indicators. 2019; 106: 105519.
  31. 31. Djordjević B, Krmac E. Evaluation of energy-environment efficiency of European transport sectors: Non-radial DEA and TOPSIS approach. Energies. 2019; 12(15): 2907.
  32. 32. Vavrek R, Bečica J. Capital City as a Factor of Multi-Criteria Decision Analysis—Application on Transport Companies in the Czech Republic. Mathematics. 2020; 8(10): 1765.
  33. 33. Chu TC. Selecting Plant Location via a Fuzzy TOPSIS Approach. The International Journal of Advanced Manufacturing Technology. 2020, 20, 859–864.
  34. 34. Chang CH, Lin JJ, Lin JH, Chiang MC. Domestic open-end equity mutual fund performance evaluation using extended TOPSIS method with different distance approaches. Expert Systems with Applications. 2010; 37(6): 4642–4649.
  35. 35. Behzadian M, Khanmohammadi Otaghsara S, Yazdani M, Ignatius J. A state-of the-art survey of TOPSIS applications. Expert Systems with Applications. 2012; 39(17): 13051–13069.
  36. 36. Vavrek R. An Analysis of Usage of a Multi-Criteria Approach in an Athlete Evaluation: An Evidence of NHL Attackers. Mathematics. 2021; 9(12): 1399.
  37. 37. Seyedmohammadi J, Sarmadian F, Jafarzadeh AA, Ghorbani MA, Shahbazi F. Application of SAW, TOPSIS and fuzzy TOPSIS models in cultivation priority planning for maize, rapeseed and soybean crops. Geoderma. 2018; 310: 178–190.
  38. 38. Vavrek R. Evaluation of the Impact of Selected Weighting Methods on the Results of the TOPSIS Technique. International Journal of Information Technology & Decision Making. 2019; 18(6): 1821–1843.
  39. 39. Dutta B, Dao SD, Martínez L, Goh M. An evolutionary strategic weight manipulation approach for multi-attribute decision making: TOPSIS method. International Journal of Approximate Reasoning. 2021; 129: 64–83.
  40. 40. Liu J, Yin Y. An integrated method for sustainable energy storing node optimization selection in China. Energy Conversion and Management. 2019; 199(1): 112049.
  41. 41. Keršuliene V, Zavadskas EK, Turskis Z. Selection of rational dispute resolution method by applying new step‐wise weight assessment ratio analysis (Swara). Journal of Business Economics and Management. 2010; 11(2): 243–258.
  42. 42. Kendall MG. Rank Correlation Methods, London: Griffin: London; 1980.
  43. 43. Fisher RA, Yates F. Statistical Tables for Biological, Agricultural and Medical Research, London: Oliver and Boyd; 1963.
  44. 44. Zavadskas EK. Multiple Criteria Evaluation of Technological Decisions of Construction, Moscow: Civil Engineering Institute; 1987.
  45. 45. Wang D, Shi Y, Wan K. Integrated evaluation of the carrying capacities of mineral resource-based cities considering synergy between subsystems. Ecological Indicators. 2020; 108: 105701.
  46. 46. Paradowski B, Shekhovtsov A, Bączkiewicz A, Kizielewicz B, Sałabun W. Similarity Analysis of Methods for Objective Determination of Weights in Multi-Criteria Decision Support Systems. Symmetry. 2021; 13: 1874.
  47. 47. Ouerghi H, Mourali O, Zagrouba E. Non-subsampled shearlet transform based MRI and PET brain image fusion using simplified pulse coupled neural network and weight local features in YIQ colour space. IET Image Processing. 2018; 12(10): 1873.
  48. 48. Yuan J, Luo X. Regional energy security performance evaluation in China using MTGS and SPA-TOPSIS. Science of the Total Environment. 2019; 696: 133817. pmid:31446291
  49. 49. Zhang L, Zhang L, Xu Y, Zhou P, Yeh CH. Evaluating urban land use efficiency with interacting criteria: An empirical study of cities in Jiangsu China. Land Use Policy. 2020; 90: 104292.
  50. 50. Singla A, Sing Ahuja I, Sing Sethi A. Comparative Analysis of Technology Push Strategies Influencing Sustainable Development in Manufacturing Industries Using Topsis and Vikor Technique. International Journal for Quality Research. 2017; 12(1): 129–146.
  51. 51. Yalcin E, Unlu U. A Multi-Criteria Performance Analysis of Initial Public Offering (IPO) Firms Using Critic and Vikor Methods. Technological and Economic Development of Economy. 20181 24(2): 534–560.
  52. 52. Sangnawakij P, Niwitpong S. Confidence intervals for coefficients of variation in two-parameter exponential distributions. Communications in Statistics: Simulation and Computation. 2017; 46(8): 6618–6630.
  53. 53. Muhammad ANB, Yeong WC, Chong ZL, LIM SL, Khoo MBC. Monitoring the coefficient of variation using a variable sample size EWMA chart. Computers and Industrial Engineering. 2018; 126: 378–397.
  54. 54. Tran KP, Heuchenne C, Balakrishnan N. On the performance of coefficient of variation charts in the presence of measurement errors. Quality and Reliability Engineering International. 2019; 35(1): 329–350.
  55. 55. Bhowate A, Aware M, Sharma S. Predictive Torque Control with Online Weighting Factor Computation Technique to Improve Performance of Induction Motor Drive in Low Speed Region. IEEE Access. 2019; 7: 42309–42321.
  56. 56. Vrabková I. Perspektivy řízení kvality ve veřejné správě. Ostrava: TU Ostrava; 2012.
  57. 57. Papcunová V, Balážová E, Agh, P. The Evaluation of the Relations Between the State Budget and the Local Self-Government Budgets (Case Study of the Slovak Republic), Proceedings of the 22ND International Colloquium on Regional Sciences, Velé Bílovice, Czechia, 12–14.6.2019, Klímová, V., Žítek, V, Masaryk University: Brno, 2019, 369–377.
  58. 58. Hendrych D. Správní věda: teorie veřejné správy. Prague: ASPI; 2007. https://doi.org/10.1364/ol.32.002339 pmid:17700778
  59. 59. Plaček M, Němec J, Ochrana F, Puček M, Krápek M, Špaček D. Do performance management schemes deliver results in the public sector? Observations from the Czech Republic. Public Money & Management. 2020; Early Access.
  60. 60. Peková J, Pilný J, Jetmar M. Veřejná správa a finance veřejného sektoru. Prague: ASPI: Prague; 2008.
  61. 61. Romanova A, Radvan M, Schweigl J. Constitutional Aspects of Local Taxes in the Slovak Republic and in the Czech Republic. Lex Localis—Journal of Local Self-Government. 2019; 17(3): 591–616.
  62. 62. Janoušková J, Sobotovičová Š. Property tax in the regions of the Czech Republic. Economics and Management (E+M) 2017, 20(4), 120–134.
  63. 63. Sedmihradská L., Bakoš E. Municipal Tax Autonomy and Tax Mimicking in Czech Municipalities. Lex localis–Journal of Local Self-Government. 2016; 14(1): 75–92.
  64. 64. Plaček M, Špaček D, Ochrana F, Krapek M, Dvořáková P. Does excellence matter? National quality awards and performance of Czech municipalities. Journal of East European Management Studies. 2019; 24(4): 589–613.
  65. 65. Hwang CM, Yang MS. New Similarity Measures Between Generalized Trapezoidal Fuzzy Numbers Using the Jaccard Index. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems. 2014; 22(6): 831–844.
  66. 66. Albatineh AN, Khan HMR, Zogheib B, Kibria GBM. Effects of some design factors on the distribution of similarity indices in cluster analysis. Communications in Statistics—Simulation and Computation. 2017; 46(5): 4018–4034.
  67. 67. Vavrek R, Bečica J, Papcunová V, Gundová P, Mitríková J. Number of Financial Indicators as a Factor of Multi-Criteria Analysis via the TOPSIS Technique: A Municipal Case Study. Algorithms. 2021; 14: 64.