2.1 Research on policy assessment means

Policy assessment is a factual evaluation process of scientific measurement of policy effectiveness and implementation process, as well as a value evaluation process of judging the benefit distribution and welfare changes generated by policies on social economy [1]. Process comparison method, value analysis method and ‌questionnaire survey is often used in policy assessment, and there are two main categories. The first species is based on econometric model to analyze the impact of a certain polices on the variables related to economy and society by quantitative or qualitative methods. It includes the top-down model represented by FEEM-CMCC model, the bottom-up technical model represented by Agent-based model, and the regression model represented by vector autoregressive model [2,3]. In practical application, such methods can get more detailed results from the actual economic operation data, but whether the actual effect of the policy can be truly separated from the data changes, and the degree of coherence between theoretical results and reality is easy to be questioned. The second category is to evaluate the rationality and feasibility of the policy content itself, which mainly includes qualitative evaluation methods represented by fuzzy comprehensive evaluation and policy literature measurement methods gradually emerging in recent years. The policy text is transformed into structured data that can be calculated, and the policy document and its implementation process and effect are systematically and comprehensively investigated and analyzed. Then a reasonable evaluation conclusion is drawn. This paper mainly reviews the second type of methods.

At present, the evaluation of policy content itself is mostly carried out by the scoring method. Susana et al. [4] applied Delphi method and analytic hierarchy process to build an innovation policy evaluation index system. The financial macro network method proposed by Stolbova et al. [5] can effectively supplement the shortcomings of existing climate policy assessment methods. Martinho et al. [6] proved through research that literature metrology, literature review and quantitative methods can better explore the dimensions related to soil legislation and policy in the context of the European Union. Helming et al. [7] conducted an empirical analysis by establishing a general equilibrium model. Nag et al. [8] divided policies into three categories of urgent, mandatory and general policies from the perspective of policy objectives and characteristics, and made corresponding evaluation indicators. Abotah et al. [9] evaluated and selected existing policies through a hierarchical decision-making model. BENNETT [10] developed the logical framework of policy evaluation in the field of public administration that consider factors in multiple stages from resource input to final goal output. Compared with traditional policy evaluation methods, PMC index model [11] is a relatively advanced policy quantitative assessment way in the world. Kuang et al. [12] established a PMC index model of cultivated land conservation policy, also compared it with other indexes to show the curve characteristics of this kind of policy, so as to evaluate it objectively. In recent years, the comprehensive scoring method based on PMC index has been tentatively applied to the quantitative assessment of innovation policy in science and technology [13], carbon emission reduction policy [14], insurance policy [15], etc., showing good applicability to this kind of research. Although this method has a more general quantitative assessment of policies, similar to other subjective evaluation methods, it is highly subjective in evaluation variable assignment and scoring, and the results are easy to be questioned. At present, the commonly used policy evaluation methods at home and abroad mainly include rooted theory, entropy weight TOPSIS method, LDA subject model, differential differential model, PMC index model and so on. Among them, the advantages of PMC index model are mainly manifested in three aspects: (1) The research object focuses on the policy document itself, which can not only conduct quantitative evaluation of a single policy, but also analyze the consistency of multiple policies; (2) The setting of variables at all levels of the model refers to the results of text mining, effectively avoiding the arbitrariness of subjective evaluation; (3) Abundant evaluation indicators can be used to analyze the internal heterogeneity and pros and cons of policies in multiple dimensions, and the policy evaluation results can be visually displayed through PMC curved graph, which is convenient to find the weak points of policies.

Policy text analysis method is a kind of policy analysis method that have emerged in recent years, including qualitative analysis represented by interpretive analysis, discourse analysis and comparative analysis of policy text and quantitative analysis represented by content analysis, quantitative analysis and content mining of policy text. Based on co-word analysis and cluster analysis methods, scholars Huang et al. [16] analyzed the change of thematic key point of the science and technology innovation policy and found that such policies have undergone significant policy thematic changes in the four fields of " international collaboration ", " manpower resource", "mechanism reform" and "research and progress priorities". Scholars Tan et al. [17] used social network analysis, semantic network analysis, correspondence analysis and other methods to analyze the differences and similarities of high-frequency words in these documents of the five-year plan for coal industry from 1996 to 2016, and obtained the changing trend of the direction and focus of such policies. Although this kind of policy text analysis method is still in its infancy, it enriches the data sources of policy analysis and is more objective and fairer than the traditional qualitative research of policy interpretation or subjective scoring methods.

2.2 Research on sports related polices

Accompanied by improvement of the status of sport and sports industry in national development, many scholars have carried out research on sports related policies, focusing on mass sports, school sports and competitive sports [18]. Wang [19] and Zhang [20], taking competitive sports practice as the direction, pointed out that the development characteristics of competitive sports policy are stay in step with the overall development of sport in China. Li and Guo [21] pointed out that the development of school physical education policy in China is characterized by the change of policy value orientation from the national standard to the human standard, and from the emphasis on decision-making guidance to the emphasis on science. Hu and Tang [22] analyzed the changes of the sports industry from 1993 to 2015, and concluded that the evolution direction of the sports industry was to strengthen the leading role of sports industry, improve market environment, and enhance the legal consciousness. They found that in the formulation of sport policy, three aspects should be considered: decision-making, implementation and evaluation. Wang et al. [23] elaborated the formulation, objectives, implementation agencies, measures and achievements of China’s sports industry policy through in-depth exploration. Lin [24] found through statistical data research that China’s sports industry has different levels of policy advantage tilt and advanced development. These indicators reflect the government’s preference in resource allocation and can be used to measure the effect of policies.

To sum up, there are still some breakthroughs in the existing research on policy evaluation. From the content level, the focus on the special issue of sports industry policy evaluation is insufficient, and the lack of research on policy content makes most of the existing research only applicable to post-evaluation. From the method level, the existing evaluation model has some problems such as lack of comprehensiveness in factor selection, strong subjectivity in valuation and low precision. Therefore, this study constructs a PMC index model, calculates index assignment based on intelligent text similarity algorithm, and makes quantitative evaluation on the existing representative sports industry policy content. It aims to provide reference for policy makers to analyze and adjust existing policies or to issue new policies. The technology roadmap is shown in Fig 2.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. PMC index model construction and variable assignment method flow of policy evaluation.

https://doi.org/10.1371/journal.pone.0316206.g002

3.1 PMC index analysis

The PMC index claims that any relevant variables should be taken into account, so the variables should be selected as widely as possible without ignoring any possible relevant variable. We can understand the internal consistency of a policy from each dimension, and its surface formed by measuring the PMC index can intuitively reflect the situation of each dimension of the policy. It also can conduct multidimensional evaluation of the policy effectiveness and conduct specific analysis of single indicators.

First of all, evaluation variables are determined and parameters are set. For the sake of evaluating China’s sports industry policy in a more targeted manner, variables in this study are set according to of reference to the policy assessment indicators put forward by Estrada [11], united with the results of sports industry policy text content mining, the first and second indexes are determined.

Secondly, the multi-input-output table is constructed. As a data analysis tool, this table can reserve a mass of data, measure and evaluate various indicator variables multidimensional, and reflect the developing procedure of a specific policy. The table is composed of a number of first-level indicators and unrestricted second-level indicators, the weight of every indicator is equal and independent of each other. According to formula, the secondary variables are assigned and put into the input-output table. The score of the first-level indicators is calculated on the basis of Formula ③. The values of the first-level indicators are the ratio of the total score of the second-level indicators to the amount of second-level indicators.

Finally, the PMC index is calculated and the surface is constructed. Finally, the PMC index of the sports industry policy is calculated on the basis of the formula, that is, total up all the scores of first-level indicators of one policy.

①

②

③

④

Where i represents the first-level indicators, j represents the second-level indicators, Xij represents a second-level indicators, m represents the amount of first-level variables, and n or n (Xij) represents the amount of second-level variables under a first-level variable. According to Estrada [11], the PMC-Index could be divided into four levels of consistency: inferior (0–3.99), acceptable (4.00–6.5), great (6.51–7.50) and excellent (7.51–9.00), which provides basic criteria for comparing different policies. The PMC surface represents the value of PMC index in the form of stereoscopic images, which can intuitively display the advantages and disadvantages of sports industry policies. PMC surfaces are always dependent on the matrices. The PM matrix is a matrix of 3 rows and 3 columns, and the PMC matrix corresponding to the PMC surface is calculated by the Formula ⑤.

⑤

3.2 Index assignment based on text similarity algorithm

A policy often involves multiple policy objectives or policy tools, and it is easy to arbitrariness to judge the topic solely through text reading or score from the title. Therefore, text mining is applied to the index assignment for analysis. Establish policy database V(d) for sports industry-related policies, where the text information in each policy text pi is w_i(d). First of all, it is necessary to conduct word segmentation processing for the policy text, and adopt a more mature maximum matching method to match the maximum symbol string of the text with the dictionary entry. If it fails to match, a Chinese character will be cut off and continue to match until the corresponding word is found in the dictionary. According to TF-IDF method [25,26], set TF_i as word frequency, N as total number of documents, and frequency of text appearing in document set as n, then: ⑥

The traditional spatial vector model for text similarity calculation uses the Angle cosine of the policy text feature vector to represent the distance: ⑦

When assigning values to quantitative evaluation variables, the Formula ⑦ is used to classify the documents, and W represents the word frequency of each word in the word frequency vector. According to the policy text corresponding to each variable, that is, the learning sample, through the intelligent calculation method, if a new policy text is similar to it, it is considered that the new policy is assigned a value of 1 on the evaluation variable; otherwise, it is 0. However, in the actual classification process, there are no mature classification samples for reference for sports industry policy research. Consequently, this paper adopts the learning sample classification method. [27], as shown in Fig 3. Aiming at the classification accuracy of learning samples to be 100%, theoretical classification is first given, and part of the learning samples are predictive classified. If it is inconsistent with the theoretical classification, the original classification is adjusted, and the learning sample for subsequent similarity calculation is finally obtained through repeated process. Given a threshold, if the similarity is greater than the threshold, the indicator is assigned a value of 1.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Process of constructing learning samples for policy assessment.

https://doi.org/10.1371/journal.pone.0316206.g003

4.1 Analysis of the evolution of sports polices in China

Since the reform and opening policy was promulgated, Chinese government has shifted its focus to economic construction, and the sports industry has also been on the "fast track" of development in line with the requirements of The Times. According to the number and focus of China’s sports industry policies, this study takes the evolution history of sports industry in China as basis, and takes the year of 1978 as the starting point. Evolution and change process of sports industry policy is divided into the four phases: germination period (1978–1992), active exploration period (1993–2000), rapid development period (2001–2013), and comprehensive development period (2014-present).

1. Germination Period (1978–1992)
   Since the year of 1978, China’s economy started to get a rapid expansion. In 1979, China regained its legal identity in the International Olympic Committee, and the industrialization of China’s sport market gradually took shape. The reform of school sport, mass sport, competition training, sport science and technology was fully launched, and relevant policies are constantly seeking directions and actively exploring, trying to make sport realize industrial development. In 1984, State Physical Culture and Sports Commission of China issued the “Notice on the Further Development of Sport”, which clearly proposed the increase of sport funds and infrastructure construction, and included it in the national economic plan, and the "Decision on economic System Reform" promulgated by the government of China in the same year also made the sporting goods manufacturing industry get the initial bud. The sportswear industry represented by Putian in Fujian province emerged. In 1986, the State Sport Commission issued the "Decision on the reform of the sport system", pointing out that the use of "multiple management" methods to strengthen the construction of sport venues, and began to take the lead in the reform of individual sport. The introduction of a few policies cannot form a joint force and break the "institutional cage" that restricts the evolution of the sports industry, but under active exploration, industrialization attribute of the evolution of sport is gradually highlighted. In 1992, in response to the "Decision to accelerate the development of the Tertiary Industry" promulgated by the government of China, the then National Sport Commission formally proposed the concept of sports industry. At this point, the conception of sports industry was taken by degrees, but in this period, sports industry only serves as a way to promote economic development, and its position in economic development has not been paid enough attention.
2. Active Exploration Period (1993–2000)
   As the last ten years before the 21st century, the market economy had gradually stabilized. The focus of the relevant sports industry policies formulated by the state is mainly on the establishment and improvement of related laws and regulations system. China started to strengthen standardized management of the sports industry. In 1993, the "Opinions on Deepening the reform of Sport" was issued, and it marked the official beginning of the reformation of China’s sport system. This document also clearly proposed the goal of the development of sports industrialization, which is mainly guided by macro policies. In the year of 1995, “The Outline for the Development of the Sports industry (1995–2010)” was issued, and it clarified the composition of sports industry. In the same year, the first law regulating the high-quality development of sport came into being, bringing the evolution of sport in China into a new stage that governing sport according to law. Subsequently, various policies on the sports industry have been promulgated. At this stage, the status of the sports industry in economic development was established, and the policy content is more targeted. With the coordination of resource allocation and government tax support, the sports industry policy system started scaling.
3. Rapid Development Period (2001–2013)
   At the early 21st century, China’s sports industry ushered in new development opportunity as it enters the new century. In July 2001, China successfully got the opportunity to host the Olympic Games, and in November of the same year, China successfully joined the WTO. In this context, China’s sports consumption continues to be active, the sport market system continues to improve, the evolution of the sports industry was highly valued by government departments of different levels, and sports industry policy also followed the current, seizing the opportunities of The Times. The biggest feature of the policy at this stage is that the main body of the policy gradually changed from the unitary main body dominated by the former State Sport Commission to the diversified main body dominated by departments at all levels. The State’s emphasis and planning on the sports industry provided strong support and target for the development of the sports industry. The General Office of the State Council issued the "Guiding Opinions on Accelerating the Development of the Sports industry" in 2010, and the General Administration of Sport issued the "12th Five-Year Plan for the sports industry" in 2011, Chinese government issued special policies and special plans at the national level for the first time to support the evolution of the sports industry. At this stage, the central departments at all levels, mainly the General Office of the State Council, issued significantly more policies than the previous stage, through further market supervision, tax incentives and other aspects of support, the sports industry ushered into a rapid development period.
4. Comprehensive Development Period (2014-present)
   Since the year of 2014, the national level to promote the evolution of sports industry policy documents frequently. In the year of 2014, the State Council officially issued the "Several opinions on Accelerating the development of sports industry to promote sports consumption", and promoted the evolution of sports industry to the state strategy level, China’s sports industry development ushered into the "second spring", into an overall development period. “Guiding opinions of the general office of the state council on accelerating the development of the fitness and leisure Industry” in 2016 and “Guiding opinions of the general office of the state council on accelerating the development of the sport competition and performance industry” in 2018 had positive impact on the evolution of China’s sports industry. At this stage, the number, subject and impact of the polices are much higher than the previous stages, and the policy content is also more plentiful, and it began to pay attention to the evolution of emerging industries such as sport tourism and exhibition, and promotes the radiation scope of China’s sports industry to be broader. Under the influence of the policy dividend, the evolution of China’s sports industry repeatedly achieved good results, the total amount increased year by year, and the status in the national economy also continued to improve, in the year of 2018, China’s sports industry was more than 1% of GDP for the first time. With the continuous optimization of policies, it is necessary to accelerate the development and improve the level of the sports service industry through openness. Now, due to policy incentives, a large number of top foreign clubs come to China every year to explore the market. From large to strong and forming global competitiveness is the inevitable path for industrial development.

The sport policy stage evolution is shown in Fig 4.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Policy stage evolution diagram.

https://doi.org/10.1371/journal.pone.0316206.g004

4.2 PMC index analysis

4.2.1 Text mining and policy selection.

The policy text of this study includes both central government policies and local government policies. Since provincial governments in China are subject to the leadership and restriction of the central and national government [28], this political relationship not only ensures the consolidated implementation of major strategies of the government, but also stimulates the initiative and sense of responsibility of local governments. Therefore, this paper selects 230 high-frequency words in central and national policies as the basis for constructing quantitative evaluation variables of policies. The central and national texts were imported into ROST software for word frequency statistics, and general service words and adverbs that were not significant to this field were screened and filtered. In this paper, based on relevant literature research and expert opinions, high-frequency words are screened. The higher the word frequency, the more important the word was.

As can be seen from Fig 5, high-frequency words like “School”, “Athletes”, “Youth” and “Seniors” appearing, shows that the development of the sports industry has a positive impact on people of different ages. The appearance of high frequency words such as society and school show that with the idea of lifelong sport, social sports into the school has become the core concern of government. The emergence of high-frequency words such as coaches, training, and sport venues shows that government focus more on the supporting conditions for the evolution of the sports industry.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Word cloud of some high-frequency words.

https://doi.org/10.1371/journal.pone.0316206.g005

On the basis of referring to the existing PMC policy evaluation model, this study modified some evaluation dimensions according to the high-frequency words analysis results, we finally established 9 first-level indicators and 47 second-level indicators of the PMC index model, which is shown in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Setting of policy quantitative evaluation variables.

https://doi.org/10.1371/journal.pone.0316206.t001

Finally, this study selects 6 policies that at the national level that cover a wide range of areas, have a large time span and involve more areas. The selected indicators are more suitable for evaluating comprehensive policies, and some adjustments should be made to specific policies. The reason for choosing these policies is that P1 has for the first time elevated nationwide fitness as a state strategy, which is an important document leading the evolution of China’s sports industry in the course of comprehensively building moderately prosperous society and deepening the reform comprehensively. P2 is an important policy jointly issued by 12 departments, which defines the goals and tasks, main work, and investment guarantee measures of the elderly sport work under the new situation, is a practical measure for the functional departments of the national government to promote establishing a social system for healthy elderly care, and is an important guiding document for the elderly sport work in the new era. A leading sporting nation is the goal and task of the reform and development of China’s sport work in the new period, P3 is the embodiment of implementing the spirit of the national important conference, and it is to further clarify the goals, tasks and measures for the construction of sports power, further defines the goals, tasks and measures of the construction of sport power. P4 is a guiding document for the implementation of the important discourse on education, it has made an overall design for the reform, development and high-quality construction of school aesthetic education under the new background of times. P5 is a policy document issued by the State Council specifically for the first time that for the construction of fitness facilities and mass sport, which fully reflects the great importance to developing mass sport and improving the standard of public services for nationwide fitness. P6 summarizes the achievements of China’s sports industry during the "13th Five-Year Plan" period, systematically plans top-level planning document for the evolution of China’s sports industry during the "14th Five-Year Plan" period, and makes a comprehensive deployment for the evolution for the next five years, which is an important policy document in this period. The list of policies is shown in Table 2.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. List of typical policies.

https://doi.org/10.1371/journal.pone.0316206.t002

After variable parameters are determined, text similarity algorithm is used to assign values to the input and output matrix. Other central and national policies other than the above six policies are selected to establish a policy database. Based on word segmentation, maximum matching method is adopted to classify the policy texts. According to the corresponding policy texts of each variable, that is, learning samples, if a new policy text is similar to it, the threshold is set at 0.8 after testing, and if the similarity is greater than 0.8, the variable is assigned as 1. The multi-input and output matrix is obtained, as shown in Table 4. Eq (4) was applied to calculate the PMC-Index value of each strategy successively. Results are shown in Tables 3 and 4.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Multi-input-output table for 6 sports industry policies.

https://doi.org/10.1371/journal.pone.0316206.t003

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. PMC Index evaluation of six policies.

https://doi.org/10.1371/journal.pone.0316206.t004

For visualization to present the differences and disadvantages of sports industry policies, we apply the Origin software to curve the PMC surface map, shown as Figs 6–11 In these graphs, the numbers 1,2, and 3 on the left matches the X-axis of the matrix, and the numbers 1,2, and 3 on the right matches the Y-axis of the matrix.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. PMC surface of P1.

https://doi.org/10.1371/journal.pone.0316206.g006

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. PMC surface of P2.

https://doi.org/10.1371/journal.pone.0316206.g007

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. PMC surface of P3.

https://doi.org/10.1371/journal.pone.0316206.g008

Download:

• PNG
  larger image
• TIFF
  original image

Fig 9. PMC surface of P4.

https://doi.org/10.1371/journal.pone.0316206.g009

Download:

• PNG
  larger image
• TIFF
  original image

Fig 10. PMC surface of P5.

https://doi.org/10.1371/journal.pone.0316206.g010

Download:

• PNG
  larger image
• TIFF
  original image

Fig 11. PMC surface of P6.

https://doi.org/10.1371/journal.pone.0316206.g011

In addition, concave and convex of the surface indicates different PMC index values. The more convex of the surface shows that the index value corresponding to the policy is higher, and the more concave of the surface shows that the index value corresponding to the policy is lower. Three dimensions correspond to the PMC matrix.

4.2.2 Typical policy analysis

From the calculation results and efficiency, it can be seen that the policy features are more abundant than the traditional tool-industrial value chain two-dimension analysis framework. From the title and theme, we can see that the types of the six selected policies are different. The rank of PMC-Index value is as follows: P6 > P1 > P3 >P5 > P2 > P4, in which policy P6 is excellent. High-quality policies can promote the evolution of the sports industry, improve industrial structure and people’s well-being. According to the current policy evaluation results, the structure and direction of China’s sports industry policies are generally reasonable. To be specific, the amount of sports industry policies in China is growing rapidly recently, reflecting the unprecedented level of national importance attached to sport, which will have a far-reaching impact. At the present stage, the typical policies in terms of policy tools, domain, function, audiences and methods have achieved the role of accelerating the construction of China’s sport power, accelerating the integrated development of the industry and encouraging the innovation of new quality and productivity, and can provide support for the establishment of a sports industry framework with scientific and reasonable layout, fully function and complete classification. According to the 2021 PwC Sports Industry Survey Report released in December 2021, the global sports industry has had a compound annual growth rate of 4.9% over the past 3–5 years, and is expected to maintain a growth rate of 4.9% over the next 3–5 years; The sports industry in North America is expected to maintain a strong growth trend in market size over the next 3–5 years; Meanwhile, against the backdrop of technological progress and policy support, the sports industry in Asia, Africa, and South America has enormous development potential. Therefore, Chinese government needs to work harder on improving its sports industry policy. In addition, the average value of policy timeliness (X2) is low, indicating that the six policies do not fully take into account the significance of multi-long-term strategies and short-term goals.

Two typical policies, P6 and P1, were selected for analysis: (1) The PMC index value of policy P6 was 9, ranking first among the six policies. It can be seen from Fig 8 that, except for the policy timeliness (X2), the other surfaces of policy P6 are smooth, and it means that policy P6 has defects in terms of policy timeliness and policy level. This is because the issuing agency of policy P6 is the General Administration of Sport of China, and the formulation of the plan is based on the functions of the General Administration of Sport, without multi-subject interaction. This policy combines the economic and social changes and the development status of China’s sports industry, and makes effective design arrangements in terms of policy audiences, priorities, methods, areas, functions, tools, etc. It is a relatively scientific and feasible policy. (2) The PMC index value of policy P1 is 7.09, ranking second among the six policies. The P1 policy is designed to promote the sports industry as a significant power for China’s economic restructuring and upgradation. As can be seen in Fig 4, (X4) policy tools, (X6) policy functions, (X7) policy audiences, and (X9) policy methods are significantly convex, with a darker color and a higher value of 1.00. It shows that policy P1 has the features that of diversified tools, comprehensive functions and wide audience, and considers the combination of ways to promote the implementation of policies. However, similar to policy P6, policy timeliness (X2) and policy level (X3) showed a significant downturn. The reason is that P1 policy is relatively macro and the critical part of the policy is single, and it lacks consideration of how to coordinate the implementation of the policy content among various departments at various stages, and it needs to support the specific implementation of the policy.