1.1 Background

In response to fierce competitions in current business environment, enterprises surely need swift and precise data collection and analysis for decision-making and performance monitoring. In this context, the enterprise management cockpit as an integrated information platform has developed into a powerful tool for the management to better understand current enterprise status and trends. This study centers around the pivotal role of such cockpit in promoting efficient management practices through discussions of its design and implementation, together with relevant advice accordingly.

The National Education Work Conference in 2023 proposed to fully implement the digital education through national strategies, so as to inject new vitality to educational development [1]. More specifically, major tasks in 2023 for the Ministry of Education’s Higher Education Department can be summarized as “further promotion of ‘Four News’ (new majors, institutions, models, and paradigms)” along with a new step forward in independent talent cultivation. Furthermore, disciplinary integration is highlighted, featured by the development of new liberal arts, quality and organizational model innovations [2].

To this end, followed by the idea of new business, business schools with management-related disciplines have embraced the new education paradigm to cultivate talents in comprehensive cross-disciplinary enterprise simulation courses, thus addressing the inadequate support for higher operational efficiency of simulated enterprises. Traditionally, such issue is primarily manifested by the conflict between the complexity and urgency of the operational process and the guidance for value enhancement in simulated enterprises, along with the mismatch between the training for decision-makers and their actual performance. In addition, the simulation process highly needs a cloud-based evaluation and analysis tool that coordinate with overall enterprise operation. This tool can facilitate real-time integration of and insight into operation data and team-building evaluations. In addition, early warning through visible meters are available for decision-makers to rapidly assess their own capabilities, then realize efficient decisions and agile management.

Correspondingly, the simulated enterprise management cockpit from Guangzhou Huashang College is adopted in this study. By studying the indicators’ design system for the cloud-based simulated intern enterprise management cockpit, a combined weighting method based on cloud models can be utilized to integrate qualitative and quantitative data evaluation. Specifically, real-time quantitative data comes from databases, while timely qualitative data from robot collection of the enterprise’s operational processes and evaluation results of team-building activities. Then, the weighting algorithm is employed to evaluate relevant data by different levels and corresponding results are presented visually. Through timely diagnosis and early warning provided on the cloud platform, decision-makers can optimize their management, as shown in Figs 1 and 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Guangzhou Huashan College’s simulation intern enterprise management cockpit.

https://doi.org/10.1371/journal.pone.0305652.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Example of the segmented meters in the simulation intern enterprise management cockpit from Guangzhou Huashang College.

https://doi.org/10.1371/journal.pone.0305652.g002

In summary, considering current trends in information technology and management disciplines, it is of practical significance to deal with the building of the early warning evaluation system for the simulated enterprise management cockpit in higher education institutions from the perspective of cloud model and combined weighting, furthermore in promoting the visual methods of enterprise management empowered by data intelligence. To be noted, the simulated enterprise management cockpit from higher education institutions will be hereinafter referred to as “the enterprise management cockpit”, and Abbreviated as “EMC”.

The importance of this research of simulated enterprise management cockpit in this paper: (1) constructing a comprehensive operational safety evaluation index system, and comprehensively and scientifically evaluating the operating status and development trends of the enterprise through the combination weight method; (2) Through advanced data visualization and information technology, the core business and financial indicators of the enterprise are presented intuitively, enabling managers to respond quickly and combining macro and micro indicators to timely identify potential risks and achieve accurate decision-making; (3) By applying the principles of cloud models, it is possible to provide real-time feedback on the warning levels of enterprises under different operating conditions, provide targeted warning information, and help prevent and respond to potential business risks in a timely manner.

1.2 Research meaning of the early warning evaluation system for the EMC

1.3 Research methodology

2.1 Literature review of international researches

In terms of modern enterprise management, the theory of Business Intelligence (BI) has developed from data warehouse to Balanced Scorecard (BSC), further evolving into the panel-style management tool known as “management cockpits” supported by visual virtual information technology. Through this approach, an integrated strategic performance system can be built for a visual and comprehensive management application that covers the whole picture and details, thus supporting all decision-making by multidimensional indicator dashboards as follows:

First, the current status of international BI researches. The preliminary BI theory, including data warehouse, online analytical processing and data mining, is rooted in transforming existing data within enterprises into knowledge, thereby contributing to informed business operational decisions [3].

The next step forward primarily in the 1990s mainly developed in databases and data modeling, with the new concept of directed graph workflow mining models—data mining [4].

More recent advancements in this aspect refer to the theory of intuitive visualized management, namely the dashboard management theory. Thereby, relevant intelligent analytical concepts are more clear, including visual dashboard analysis, control and self-service analysis, integrated data product chains, cloud-based management data analysis, mobile terminal analysis and IoT data analysis [5].

Moreover, the rise of service-oriented personal cloud computing requires new technologies for asset discovery, exchange, and management. In response, a large-scale distributed system on unified personal clouds is then proposed, namely interactive cockpits that manage personal clouds and their alliances. In this way, there is less reliance on central infrastructure, while greater balance between service supply and consumption [6].

To deal with cross-functional communication and collaborative management within the management cockpit, a control and management process is established based on intellectual property and strategic orientation. This approach has been proved successful within enterprises and interdisciplinary management teams, enabling structural, categorized, and managing functionsv—with abundant visualized information along with extensive analysis and simulation options for comprehensive and sustainable performance management [7].

In terms of BSC’s application in higher education, the recommended method is to formulate strategies for implementing BSC within educational institutions, then apply and observe its effectiveness in organizational assessments. The aim is to define a management and tracking system for industry bench-marking [8].

During the strategic implementation phase, the management proposes to interpret strategic execution as a dynamic decision-making task. Learning from previous experience of solving complex problems and performance measurement, more explorations are conducted on whether factors such as BSC cockpit, intelligence, and knowledge can explain variations in strategic execution performance [9].

When considering differentiated decisions of implementing strategies and outcomes resulting from various cockpits, scholars proposed to integrate ideas related to strategic operations. Therefore, they design a closed-loop control task for strategic implementation and advocate a BSC strategy map cockpit. This cockpit provides participants with a concentrated and useful interface with both theoretical and practical contributions during implementation, also facilitating the modeling considering the impact of management performance measurements [10].

Second, The current international research status of evaluating enterprise management information systems. So far, popular evaluation methods for information systems worldwide include qualitative expert analysis using the Delphi method, fuzzy composite evaluation based on fuzzy matrices, the quantification of qualitative data through a hierarchical quantitative analysis using the Analytic Hierarchy Process, multi-criteria decision analysis, along with expectation models that emphasize customer satisfaction. Specifically, ideas on strategic performance evaluation are as follows:

In terms of performance management, relevant theories proposed to integrate an enterprise’s overall strategy with its financial indicators and non-financial information. To be noted, the crucial connection between overall strategy and performance indicators during evaluation part is fully highlighted, therefore a Performance Pyramid model is designed accordingly [11].

In addition, the reform-centered view of performance evaluation reveals enterprises’ sustained development capability, which is theoretically sophisticated, but rarely applied in practice [12].

The financial management system EVA (Economic Value Added) came out, which combines decision-making, incentive, and performance evaluation. In particular, it emphasizes enterprises’ costs of all capital inputs and their long-term development [13].

Serving as a performance assessment system, the BSC is of great significance in implementing enterprise strategies. It provides a framework that focuses on key management processes, assessing an enterprise’s past performance, together with its future development potential and competitiveness [14].

When evaluating enterprise performance, the “Four-Dimension” framework proposed by Robert Hall is adopted, including quality, lead time, resource utilization, and human resource development. These four non-financial indicators are included into the performance evaluation system and improvements in any of these dimensions should not sacrifice others, so as to reduce competitive risks.

Regarding decision support systems, the solution is to integrate visualize method and decision modeling capabilities with human insights and interaction abilities, thus developing a decision support system known as Forest Community-DSS. By leveraging techniques like DT (Decision Theater), FC-DSS and relevant technologies can enhance stakeholder involvement in decision-making through higher participation frequency, more input from stakeholders, support given to the development and evaluation of alternative solutions, and optimized selection of preferential alternatives, as highlighted by Boukherroub, D’amours, & Rönnqvist [15].

Third, Overseas research status of management cockpit systems. Among developed countries, the management cockpit system serves as sophisticated decision support within information systems, which demonstrates an advanced development stage of business intelligence systems that evaluated by the fourth-generation BSC concept. This system has now been widely adopted in various industries, with verified values and efficiency in enterprise competition and operations.

So far, prominent global suppliers of such cockpit information system mainly refer to SAP, Oracle, Microsoft, SAS, Qlik, Tableau and Microstrategy, basically monopolies in sectors such as telecommunications, finance, retail, and insurance.

2.2 Literature review of research status in China

In China, domestic researches on evaluating enterprise performance primarily focuses on local applications of advanced international models. Apart from the characteristics of Chinese enterprise s’ supply chains, scholars adopted a hierarchical approach based on BSC theory. They emphasize the need for comprehensive evaluations considering multiple factors, thus upgrading the indicator system that integrates BI theory.

First, the domestic BI research status. The initial BI research in China dates back to a relevant report in 1998, presenting several views discovered in the management community. After that, theoretical researches in specific areas emerged, such as starting from interpreting BI concept from management, technology, and application perspectives [16], then corresponding implementation strategies [17], and further control recommendations during BI implementation [18].

The cloud technology and business intelligence was integrated, featured by resource sharing, low costs, and real-time interactions [19]. In addition, a system dynamics model for strategic performance evaluation was introduced, which effectively combines relevant theories and research methods of BSC and system dynamics. In detail, this model had an extra BSC dimension of social responsibility, finally building a dynamic five-dimensional model with an indicator system and strategic map for evaluating enterprise strategic performance by simulation software [20].

To upgrade agile enterprise management, a management cockpit was proposed to integrate various business units and break down the barriers between departments. This efficient method functions well in handling with huge amounts of isolated cross-organizational business data at different management levels, thus mitigating “information silos” and enhancing the enterprise’s competitive prowess in dynamic markets [21]. What’s more, a multi-dimensional intricate system of budget analysis indicators was developed to support lean management and facilitate feedback. Particularly, the management cockpit works as a budget compass and baton, directing towards efficient resource allocation and continuous improvement [22].

Second, the domestic research status of information system evaluation. Despite being not mature, relevant domestic researches on information system evaluation have already yielded significant results. One previous study proposed an evaluation system for enterprise informatization, featured by 20 primary indicators and specific calculation methods. This study was also a pioneer in measuring the effectiveness of enterprise informatization by quantifying the correlation between the level of informatization and enterprise performance [23].

After several years of technology advances, a comprehensive life-cycle evaluation system was put forward, which utilized the hierarchical grey method and verified its scientific validity and applicability through case study [24].

In order to further upgrade the evaluation system, user-friendly idea prevailed, specifically valuation metrics of relevant users in different stages were established, including researches on the evaluation indicator system for building enterprise information system considering both enterprise management personnel and front-line operators [25].

Based on the multi-dimensional utility theory, scholars introduced a flexible information system evaluation model accordingly in multi-objective decision-making. Supported by a comprehensive and objective indicator system, a hierarchical structure for information system evaluation utilizing a multi-dimensional utility model, along with specific merging rules and weights, the overall effectiveness of information systems has been fully validated [26].

Among various relevant evaluation methods, scholars suggest a combination of two or more methods [27].

Then, scholars proposed an enhanced value evaluation model according to stakeholder theory. This model combines both financial and non-financial indicators in three dynamic dimensions that reflects the organization’s value and strategic performance, namely growth performance, profitability performance, and risk performance [28].

From the perspective of cultivating students’ innovation and entrepreneurship capabilities, the dashboard can be utilized, so that a pragmatic virtual simulation teaching platform can visualize relevant resources based on business management disciplines. In this way, traditional low efficiency in transferring theoretical knowledge into practical skills is no longer a problem. Instead, this method can motivate an active entrepreneurial mindset and participation, so as to enhance the entrepreneurial qualities of high-end management professionals [29].

Furthermore, through the above literature review research, the performance of business functions in evolving into intelligent enterprise management cockpit management services lies in: (1) the ability to utilize technologies such as big data, cloud computing, artificial intelligence, and automation to integrate ERP databases with other systems; (2) Can provide agile, scenario based, and intelligent management and decision-making services for enterprises, help them improve their digital capabilities, and achieve maximum data value; (3) It has the characteristics of low code, light configuration, easy operation, strong support, open integration, compatibility with multiple platforms, and emphasizes data security and privacy protection. In summary, this enterprise management service can provide rich and multi-dimensional custom data display, report visualization, and interaction capabilities, helping enterprises better manage and analyze data, making wiser decisions, and emphasizing data security and privacy protection.

3.1 Enterprise management cockpit

The enterprise management cockpit is a management information center system that provides "One Stop" decision support for senior management. It visually displays the key indicators (KPIs) of enterprise operation in the form of a cockpit, through various common charts (speedometer, volume bar, warning radar, and radar), intuitively monitors the operation of the enterprise, and can provide early warning and mining analysis of abnormal key indicators.

When enterprise managers have no time to query in the physical cabin, they can synchronously query the corresponding dial data with mobile phones and other mobile terminals under Internet conditions, so as to achieve an integrated online and offline application.

The management cockpit incorporates the management philosophy of the balanced scorecard, which unfolds the company’s vision and strategy through a strategic map layer by layer into a set of dynamic and measurable KPI indicators. The internal processes and team learning aspects emphasized by the balanced scorecard correspond to the blue wall content of the management cockpit, the business, finance, and operation aspects correspond to the black wall content of the management cockpit, and the product development and customer aspects correspond to the red wall content of the management cockpit;

3.2 Analytic Hierarchy Process

Analytic Hierarchy Process (AHP) is a qualitative and quantitative decision-making method proposed by Professor Sati in the early 1970s in the United States. Its characteristic is to conduct in-depth analysis of the essence, influencing factors, and internal relationships of complex decision-making problems, and on this basis, use quantitative information to mathematize the thinking process of decision-making, solve complex decision-making problems with multi-objective, multi criteria, or unstructured characteristics, thereby making decision-making methods more convenient. This method decomposes into different hierarchical structures in the order of the overall goal, sub goals at each level, evaluation criteria, and specific backup plans. Then, the method of solving the judgment matrix eigenvectors is used to obtain the priority weights of each element in each level for a certain element in the previous level. Finally, the method of weighted sum is used to gradually merge the final weights of each backup plan for the overall goal. The one with the highest final weight is the optimal plan.

In modern decision-making assistance applications, with the help of information technology, the elements related to decision-making are decomposed into levels such as goals, criteria, and plans through matching. The network system theory and multi-objective comprehensive evaluation method are applied to propose a hierarchical weight decision analysis method.

The management simulation platform of the Huashang School of Guangdong University of Finance and Economics is facing a panoramic simulation of socialized operations. In the face of centralized data at various levels, it is necessary to determine the impact weights and select the best ones, in order to select key indicator data that can affect operations.

3.3 Fuzzy comprehensive evaluation method

This comprehensive evaluation method transforms qualitative evaluation into quantitative evaluation based on the membership theory of fuzzy mathematics. The basic principle is: first, determine the set of factors (indicators) of the evaluated object and evaluate the set of levels; Secondly, determine the weights and membership degree vectors of each factor separately to obtain a fuzzy evaluation matrix; Thirdly, perform fuzzy operations on the fuzzy evaluation matrix and the weight vectors of factors, and normalize them to obtain the fuzzy comprehensive evaluation results.

The management simulation platform of Huashang School of Guangdong University of Finance and Economics requires enterprises to objectively reflect the actual situation of their business processes, including the assessment of their own human capital, timely feedback on their business operations, and business dealings with peripheral institutions such as industry and commerce, taxation, and auditing. Both quantitative and qualitative data are needed, and they need to be unified into a quantitative caliber. Therefore, modern technology and fuzzy mathematics are needed to integrate them together.

3.4 Cloud model theory

Cloud model theory is a combination evaluation method that combines qualitative knowledge with quantitative data to integrate uncertain data. It achieves quantitative description of concepts through the concept of clouds. The theoretical principle is that clouds are composed of many cloud droplets, each representing a possible value and its membership degree. This theory introduces probability theory and fuzzy set theory, and its advantage lies in dealing with uncertainty and fuzziness problems, especially in fields such as data mining, intelligent control, and decision support. It provides an effective tool and method for handling uncertain information, improving the accuracy and adaptability of decision-making.

4.1 Re-building of the early warning evaluation system for the EMC in higher education institutions

Considering a new competitive environment and the strategic evaluation needs for enterprises’ scale development, higher education institutions should highlight that the development of management practices is the condition for modern enterprises to employ new performance evaluation methods. In addition, comprehensive factors should be taken into consideration, including those aspects related to enterprise development goals, organizational structure, organizational capabilities, comprehensive budget management, incentive mechanisms, indicator quantification, internal controls, and financial analysis. Moreover, based on the core BSC ideas of enterprise performance evaluation, scientific methods of fuzzy judgment and analytic hierarchy process (AHP) were adopted to determine each indicator weight in the comprehensive evaluation function model. Finally, beyond operating as a teaching method for stronger enterprise management, this approach also contributes to a new regional/ global digital decision system.

More specifically, the first step is to cultivate relevant abilities and confirm board performance, then the next to optimize indicator selection combining the characteristics of simulation enterprise on the platform and expert opinions, so as to develop a comprehensive evaluation indicator system for early warning based on cause-and-effect relationship chain and strategic mapping. To be noted, the target, or the enterprise’s strategic goals, was initially divided by AHP into four layers from top to bottom. In detail, the first layer represents the comprehensive early warning evaluation layer (the comprehensive layer), showing the overall dashboard performance. Then this layer was further sub-divided into five specific strategic decision layers (the first-level dimensions), namely the dimensions of business, finance, and operation, customer management and marketing, internal operational objectives, product development strategies, along with team building and management. Furthermore, each strategic decision layer was broken down into tactical operational layers (the second-level dimensions), which further divided into specific management execution indicators (the third-level dimensions). Particularly, the relevance of each indicator to the indicator system was identified, finally shaping a four-layer comprehensive evaluation indicator system featured by a “one overall and three-dimensional” framework (as shown in Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Comprehensive early warning evaluation indicator system of simulated enterprise management cockpit from higher education institution.

https://doi.org/10.1371/journal.pone.0305652.t001

This study adopts a three-level indicator system to identify the positive and negative correlation of specific indicators at the third level. (1) Positive correlation indicates that the larger the indicator value is; the higher utility the indicator system is. (2) Negative correlation represents the opposite.

Specific indicators for business and finance include: (1) Financial warning, which measures operational risks from the financial perspective. (2) Budget implementation deviation, which measures the level of financial control in terms of budget effectiveness. (3) Business value-added efficiency, which reflects the value of investment to shareholders. (4) Operational incentive quality, which measures the operational efficiency from the financial perspective. (5) Level of total scale, which reflects the operational status in quantitative ways.

Specific indicators for customer management and marketing include: (1) Proportion of attention customers, which analyzes the contribution of different types of customers. (2) Key customer segmentation, which explains the effectiveness of management on important customers. (3) Customer satisfaction, which reflects to what extent products or services satisfy consumers.

Specific indicators for internal operational objectives include: (1) Business warning, which measures operational risks from a business perspective. (2) Operational support, which reflect a company’s operation efficiency in terms of turnover. (3) Risk prevention performance, which shows the effectiveness of social auditing, debt levels, and operational conditions from a view of risk control. (4) Internal operational efficiency, which shows management effectiveness according to a company’s development efficiency. (5) Compliance management level, which measures the level of management compliance with legal and ethical standards of a company.

Specific indicators for product development strategy include: (1) Overall product benefits, which reflects the contribution of product value. (2) Product structure, which exhibits the product mix from the perspective of quantity and value.

Specific indicators for team building and management include: (1) Team overall satisfaction, which values employees’ satisfaction on the company’s working environment. (2) Team organizational level, which measures the effectiveness of organizational management within the company. (3) The ability of being a learning organization, which reflects the progress of human capital.

4.2 Theory for determining the weight of comprehensive early warning evaluation indicators for the EMC

4.2.1 Subjective weights determined by AHP.

The Analytic Hierarchy Process (AHP) is a comprehensive analysis method integrating qualitative and quantitative analysis. Proposed by Professor A.L. Saaty of the University of Pittsburgh in the 1970s, this method applies network system theory to solve multi-objective comprehensive evaluation problems. By assigning weights to decision-related indicators in a hierarchical order, it provides a clear and straightforward solution to the ranking of multi-objective, multi-criteria, and multi-level factors. Specific implementation process is described in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Weight determination paths by AHP.

https://doi.org/10.1371/journal.pone.0305652.g003

4.2.1.1 Defining the hierarchical structure model for the set of indicators. According to Table 1, the comprehensive early warning evaluation indicator system proposed by this study is structured as follows. The top-level is the objective layer, denoted as U. The first-level indicators are known as U₁, U₂, …, U_m. The second-level indicators are recorded as U₁₁, U₁₂, …, U_mn. The third-level indicators are denoted as U₁₁₁, U₁₁₂, …, U_mnt.

4.2.1.2 Constructing comparative judgment matrices. Firstly, a judgment matrix A is constructed as (a_ij)_m×m for each hierarchical level corresponding to the three dimensions in Table 1. Then, technical experts and scholars are invited to compare the importance of each pair of indicators within the hierarchical level with a scale of 1 to 9 to determine the judgment matrices for each level.

4.2.1.3 Normalizing the judgment matrices at each level and obtaining the normalized vector w_i of the i-th indicator.

(1)

4.2.1.4 Determining the weight coefficient of each indicator by weighting the judgment matrices at each level.

(2)

4.2.1.5 Calculating the maximum eigenvalue λ_max of judgment matrix with MMULT function and checking the consistency of CR.

(3)

Calculating the consistency index CI (4)

When CI = 0, it shows that there is consistency in the judgment matrix, and the larger the value is, the worse the consistency is.

By looking up the table, we can get the average random consistency index RI.

(5)

When CR≤0.1, the judgment matrix has an acceptable degree of consistency as needed, otherwise it should be adjusted.

4.2.2 Determining the objective weight based on entropy weight method.

The entropy weight method, put forward by Shannon in the 1940s, is used to determine the weight based on the dispersion of objective data. Weights are assigned to indicators according to the information entropy formula, and then adjusted to ensure objectivity.

4.2.2.1 Normalizing the third-level indicator matrix for each second-level indicator. The second-level indicators are classified into positive and negative ones according to the attributes of their relevant third-level indicators as shown in Table 1. For positive indicators, the higher score the comprehensive evaluation result is, the more the indicator contributes to the one at a higher level. For negative indicators, the higher score the comprehensive evaluation result is, the less the contribution it makes. As the third-level indicators under the second-level ones have different measurement, it is necessary to standardize them to ensure comparability. The specific approach is as follows: first, the absolute values of indicators are converted into relative values to avoid the influence of dimensional difference on evaluation results. By doing so, normalized values would fall within the range of [0, 1]. The indicator system should be normalized using the min-max method. The positive indicator is applied with Eq (6) and negative one, Eq (7). The formulas are as follows: (6) (7) Where,

x_j = the value of the j-th indicator;

x_max = the maximum value among different evaluation series of the same indicator;

x_min = the minimum value among different evaluation series of the same indicator.

4.2.2.2 Data shift. In case meaningless data such as zero and negative values affect the calculation of subsequent data, this study adopts a data shift strategy to resolve this problem, by adding a correction value α to each normalized datum.

(8)

4.2.2.3 Determining the weight y_ij of the third-level indicators under corresponding second-level indicators.

(9)

4.2.2.4 Calculating the j-th information entropy value e_j for each indicator. (10) (11) Where, K is a constant determined by the number of samples.

4.2.2.5 Calculating the j-th information utility value d_j for each indicator.

(12)

The information utility value for a particular indicator, denoted as x_ij, depends on the difference between 1 and the information entropy value e_j of that indicator. This information utility value directly influences the weight assigned to the indicator x_ij. A higher information utility value indicates greater importance of the indicator and such indicator is assigned with a high weight.

4.2.2.6 Calculating the weight w_j for the third-level indicators under the corresponding second-level indicator.

(13)

4.3 Designing the comprehensive early warning evaluation system based on the cloud model

The model of the comprehensive early warning evaluation system of the college simulation enterprise management cockpit is designed based on the principle of cloud modeling. It aims at diagnosing qualitative and quantitative uncertainty transformation of the talent cultivation capacity system in large-scale interdisciplinary simulation platforms.

5.1 Calculating the weight of evaluation model indicators

This paper studies the case of an integrated Guangzhou Huashang College Internship Simulation Platform for business operation of companies. In each period, a total of 54 manufacturing enterprises and 36 distributors provide internship opportunities for students. As the business operation is cycled, the competition is fierce, and companies cannot assess its operational status in a timely manner, it is urgent to put in place a visual tool to track the real-time operation process of the companies, which can provide diagnostic warnings against business risks. Key risks include cash budgeting, bankruptcy, capital structure, operational efficiency, product development, customer management, team building, learning ability and growth. To address the issue in a scientific-based approach, this study adopts a weight mix method to score relevant indicators. A total of 12 experts from the college and 3 experts from Kingdee Precision Information Technology Service Co., Ltd. are invited to score the subjective weighting. Aggregation intervals for scatter plot warnings referenced by objective indicator scoring are selected. Furthermore, the calculation results are adjusted to ensure their validity. It is worth noting that the following indicators correspond to column U in Table 1.

5.2 Calculating the weight of the evaluation model indicators using AHP and entropy weight method

The first-level indicators are paired up to evaluate their relative importance by experts and the importance of the indicator weight is scored. The second-level and third-level indicators should do the same. Subjective weights for indicators of three levels are calculated according to Eqs (1) to (5). Next, based on the data of 20 manufacturing enterprises that have performed well in the past three years according to the Guangzhou Huashang College Simulation Platform, combined with practical experience, knowledge, and individual opinions about enterprise management of experts on and off campus, each indicator of the 20 companies in Table 1 is assessed. Based on the positive and negative correlation of the indicators and using scatter plots, the evaluation dataset is obtained and then normalized to calculate the entropy weight according to Eqs (6) to (13) as objective weight. Then, the weight mix is calculated according to Eq (14) (the three types of weights are shown in the left half of Table 2). Finally, the indicators in the left half of Table 2, labeled as “indicator weight (unnormalized) before normalization by U-weight”, are normalized according to the weight of the corresponding first-level indicators to obtain the final data, as are shown in the right half of Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Weight calculation of three indicator levels.

https://doi.org/10.1371/journal.pone.0305652.t002

Regarding specific calculation, this study adopts the subjective weight solving process of 11 three-level indicators of financial early warning U11 as an example to illustrate the consistency test of AHP, as is shown in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Calculation of the subjective weights of 11 three-level indicators of financial early warning U11.

https://doi.org/10.1371/journal.pone.0305652.t003

According to important of the paired indicators, the 11 third-level indicators of financial warning U11 in Table 3 are scored, and subjective weights are calculated. According to Table 4, it is found that the RI value is 1.5135. Then, using Eq (5), CR is calculated as 0.0522, which is less than 0.1, indicating that the dataset has acceptable consistency. This example demonstrates that the overall subjective weighting of experts complies with the consistency test. Based on the left half of Table 2, the subjective weights of the indicators are multiplied by the first-level indicator weight of U1 (0.4072) and then by the second-level indicator weight of U11 (0.2493). By doing so, the normalized weights for this group are generated, aligning with the weights of the 11 indicators that belong to U11 in the right half of Table 2. This accounts for how sample data are calculated and normalized.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Average random consistency index RI.

https://doi.org/10.1371/journal.pone.0305652.t004

5.3 The evaluation of the comprehensive cloud model warning

5.3.1 Determining the criteria for the evaluation of the cloud model comprehensive early warning for enterprise management cockpit.

There are three enterprise warning levels for the college simulation platform, which are categorized using a digital grading system, where higher values indicate greater safety. Based on the actual operation of enterprises on the simulation platform, each warning level ranges from (0–60], (60–70], and (70–100] respectively. In this case, the value of K isset as 0.1, and a dial is set in the Kingdee Light Analysis Software, as shown in Fig 4. In this study, He is determined as 0.1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. An example of the warning area division on the management dashboard of internship simulation platform for enterprise operation at Guangzhou Huashang College.

https://doi.org/10.1371/journal.pone.0305652.g004

Based on the characteristics of the warning intervals in Fig 4, and in accordance with Eqs (16) to (18), the cloud model for the warning levels are calculated, as shown in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Cloud model for enterprise warning levels on the college simulation platform.

https://doi.org/10.1371/journal.pone.0305652.t005

Then, the standard cloud of early warning level is programmed by MATLAB to obtain the standard cloud map, as shown in Fig 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Standard cloud image of enterprise management cockpit.

https://doi.org/10.1371/journal.pone.0305652.g005

5.3.2 Computing the evaluation cloud set for all sample indicators.

According to Eqs (15), (19) to (22), the evaluation cloud set (Exj, Enj, Hej) of scores for each level indicators of the sample enterprises are calculated, as presented in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Evaluation cloud for indicators at each level.

https://doi.org/10.1371/journal.pone.0305652.t006

5.3.4 Determination of the warning level for enterprises on the simulation platform.

The evaluation cloud map Z (73.2, 9.12, 2.18) for the early warning levels are drawn based on the comprehensive evaluation cloud of enterprises on the Guangzhou Huashang College Simulation Platform. It is then compared with the standard cloud map. By programming the standard cloud map using MATLAB, the results are shown in Fig 6. For the warning level of “General” and “Good”, the comprehensive cloud map is further analyzed to study its numerical features. It is determined that the cloud map is most similar to the “Good” standard one. Therefore, the safety evaluation level for enterprises on the Guangzhou Huashang College Simulation Platform is determined to be “Good”. This indicates that these enterprises can operate in a good state; however, continuous improvement is needed in key areas such as the main business profit margin U145, operating leverage ratio U317, current ratio U333, quick ratio U334, cash ratio U339, and cash flow ratio U3310, in case of a shift to the dangerous level.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Comparison between composite indicator evaluation values and standard cloud evaluation values.

https://doi.org/10.1371/journal.pone.0305652.g006

6.1 Main conclusions

Firstly, the enterprise management cockpit is a tool that utilizes data visualization and information technology to present key performance indicators of an organization. It allows the management personnel to access key data on an integrated platform, so as to better understand the condition and development of the enterprise. Researches have shown that this management approach is of great significance in improving the overall operational management level of enterprises on the college simulation platform. It helps the management personnel to gain a better understanding of the business through macro system and micro indicators and find risks in advance, and supports their decision-making.

Secondly, an evaluation indicator system for the operational safety of enterprises is established based on five management dimensions: integration of business and finance, customers, internal operations, products, and teams. This system reflects various factors necessary to assess the safety of enterprises on the simulation platform. By using the subjective weights derived from the AHP and the objective weights calculated through the entropy weight method, and the weight mix of each indicator is obtained in proportion. This approach reflects the importance of each indicator in an objective way and provides reliable data for the comprehensive warning evaluation of the college simulation enterprise management cockpit cloud model. It bridges the gap in the digitalization of the platform throughout the warning diagnosis process.

Thirdly, the application of cloud models in establishing a safety evaluation model for simulation platform enterprises enables a comprehensive understanding of the overall operational safety. It allows real-time feedback on the warning levels under different operational conditions of enterprises.

6.2 Recommendations

The enterprise management cockpit of the university management simulation platform can provide comprehensive and real-time operation monitoring and decision support data query, analysis, and early warning assistance. Its limitations lie in the complexity of data integration, limitations on model accuracy, and system scalability. In response to these limitations, further research areas can focus on empirical verification of the multi indicator scientific comprehensive system of combination and cloud models, and compare the latest AI functions and automation technologies to improve data integration efficiency, optimize model accuracy, and explore more flexible and scalable system architectures to break through bottlenecks in this field and promote further development of research.