2.1. Evolution of environmental policy evaluation

Environmental policy assessment has long been an important means for understanding policy effectiveness and impact. Early research primarily relied on qualitative descriptions and case studies, which had limitations in capturing the complex interactions of policies. Since the 1990s, assessment methods have become more sophisticated, introducing quantitative approaches. For example, Dijk et al. (1996) developed an integrated framework using micro-simulation models (NFM) combined with agricultural census data to evaluate environmental policies. This highlighted the importance of considering the complex interactions between economic activities and the environment in policy assessment [15]. Similarly, Frenken et al. (2004) examined the development of environmentally friendly vehicle technology using patent data, underscoring the importance of technological and organizational diversity in advancing environmental technological innovation [16]. Such models are powerful in predicting potential outcomes of policy interventions but typically require large-scale datasets and specialized expertise.

2.2. The rise of computational methods in policy analysis

In recent years, with the explosive growth of digitized textual data, computational methods, especially text mining, have gained significant attention in the field of policy analysis. Text mining is the process of exploring essential patterns and associations from large volumes of unstructured textual information and extracting valuable insights. By analyzing textual data such as policy documents, meeting records, and media reports, researchers can systematically identify themes, sentiments, and trends that human readers might easily overlook. For instance,.Rodriguez et al. (2008) used text mining to quantitatively evaluate China’s new energy vehicle policies and extracted key topics and trends from policy documents. This approach provides an objective and replicable pathway for tracking temporal changes in policy discourse and prioritiesutilize [17]. The robust mathematical planning technique proposed by Doole and Pannell (2011) provides a novel solution to address the issue of inadequate information in environmental policy assessment [18]. Recent advances in natural language processing have enabled more sophisticated thematic extraction from policy documents. Danesh et al. (2021) [19] demonstrated the effectiveness of Latent Dirichlet Allocation (LDA) for thematic analysis in large document corpora, providing a retrospective and prospective framework for understanding topic evolution over time. Similarly, Danesh and Dastani (2023) [20] illustrated how Python-based NLP techniques can systematically extract patterns from historical document collections, offering methodological insights applicable to policy text analysis.

Additionally, Wegner and Pascual (2011) criticize the constraints of cost-benefit analysis tools and advocate adopting a pluralist framework to assess policy impacts comprehensively [21]. Regarding the quantification and systematization of policy assessment, Bel and Holst (2018) introduce a quantitative tool for evaluating transport policies [22]. This tool analyzes the effects of bus rapid transit (BRT) systems using difference-in-differences techniques. Subsequently, Kuminoff (2018) introduces a conceptual model of consumer categorization in the housing, labour, and healthcare markets and discusses the application of benefit transfers, offering a new theoretical framework for examining consumer behaviour [23]. Furthermore, Costa et al. (2018) utilize an epidemiological quasi-experimental statistical approach to evaluate the influence of land-use policies on deforestation rates, providing a new statistical methodology for assessing land-use policies [24]. Zeiger et al. (2019) delineate a framework for evaluating sustainable development endeavours, measuring environmental planning systems and environmental outcomes, offering a comprehensive framework [25].

2.3. Index models for policy assessment: the PMC framework

While text mining excels at exploratory analysis of policy content, assessing the quality and consistency of policy design itself requires a more structured framework. Here, the Policy Modeling Consistency (PMC) index model emerges as an effective tool. Developed by Schoenefeld and Jordan (2019), the PMC index is a quantitative policy assessment method used to evaluate the strengths and weaknesses of policy documents [26]. The model decomposes policies into multidimensional variables (such as policy types, objectives, constraints, incentives) and assesses each variable with binary values (0 or 1) based on its presence or absence. As it does not require subjective weighting, this method reduces subjectivity in the assessment process and facilitates comparisons between different policies.

Yimsuk and Thammaboosadee (2024) applied the PMC index model to quantitatively assess China’s farmland protection policies, revealing their effectiveness and internal consistency. This approach objectively demonstrates the comprehensiveness and logical consistency of policy design [27]. Regarding empirical research and model development for policy assessment, Chen Die et al. (2019) developed an economic policy assessment model to analyze network car policies, providing a new framework for policy assessment within the emerging economic paradigm [28]. Similarly, Hynes and O’Donoghue (2020) employed a novel value function transfer method to evaluate eco-efficiency, introducing a new quantitative tool [29]. Moreover, Kuang et al. (2020) assessed the effectiveness of arable land protection policies utilizing the PMC-Index model, introducing an innovative quantitative approach for evaluating the efficacy of these policies [30]. Bouma (2021) conducted an extensive review of the utilization and value of experimental methods in environmental policy assessment, highlighting the potential of employing mixed methods and providing theoretical underpinnings for advancing innovative policy assessment methods [31].

Similarly, Chen et al. (2021) utilized the PMC index model to screen policy texts and develop a model for assessing the support of policy measures, offering an innovative model for quantitative analysis of policy texts [32]. Tan and Mao (2021) examined the influence of the central environmental protection inspection policy on air quality, providing empirical analyses for evaluating environmental regulatory policies [33]. Mentzakis and Sadeh (2021) offer a new experimental economics perspective for evaluating environmental economic policies, examining the effects of incentives on risk aversion and discounting tasks for monetary and environmental goods [34].

Regarding the systematization and quantification of policy evaluation, Lu et al. (2021) introduce a new systematic approach to evaluating public health emergencies by analyzing the resumption policy during an epidemic in China using a policy evaluation model [35]. Similarly, Yang et al. (2021) utilized text mining to quantitatively evaluate China’s new energy vehicle policy, developing an innovative text analysis tool for assessing new energy vehicle policies [36]. Dai et al. (2021) evaluated the green development policy of the Yangtze River Economic Belt, proposing theoretical recommendations for enhancing regional green development policies and offering theoretical support for optimizing such policies [37]. Meanwhile, Brouwers et al. (2022) investigated the understanding of impacts among participants during the assessment of nature policies in the Netherlands, providing a fresh perspective on participant analysis in the policy assessment process [38]. Robson-Williams et al. (2022) utilized a causal framework to evaluate water resources management, proposing an innovative framework for assessing international water resources management policies [39]. Meanwhile, Peng et al. (2022) assessed the sustainability of long-term care insurance in China, offering a fresh perspective for the long-term assessment of social security policies [40]. Lastly, regarding the integrative and strategic aspects of policy assessment, Ferraro and Miranda (2014) utilize statistical techniques to replicate the results of water use experiments, offering a new statistical approach to assessing water management policies [41].

2.4. Integration and identification of research gaps

Based on the existing literature review, environmental policy assessment has evolved from qualitative methods to diverse quantitative approaches including micro-simulation, text mining, and the PMC index. However, the integrated application of these methods, particularly text mining and the PMC index, remains a relatively new field. Geng et al. (2015) employed this integrated approach to evaluate the scientific rigor of China’s wind power industry policies and proposed a new analytical framework. This study further develops existing research by applying this integrated approach to the unique context of Macao’s environmental policies [42]. Macao’s policies are influenced by special factors such as its status as a Special Administrative Region, highly urbanized environment, and role in the Guangdong-Hong Kong-Macao Greater Bay Area. To date, research on Macao’s environmental policies has been primarily descriptive, without systematic and quantitative studies assessing their design quality and effectiveness. Therefore, this study aims to fill this research gap by empirically identifying the main themes and priorities of Macao’s environmental policies through text mining and utilizing these results to construct and validate variables for the PMC index model. This hybrid research methodology promises to provide a more comprehensive and nuanced understanding of the strengths and weaknesses of Macao’s environmental governance.Additionally, Xu et al. (2015) proposed an integrated environmental policy, including measures such as constructing biomass power plants based on a dynamic planning model, introducing a novel approach to the systematic design of environmental policies [43]. Kunstler and Vasileiadou (2016) explored the effects of using multiple assessment methods concurrently on daily assessment practices, offering practical guidance for method selection and application [44]. Meanwhile, Medeiros (2017) conducted a geographical analysis of the impact of EU cohesion policy, providing empirical insights into the geographical variability of regional policies [45]. Additionally, Yang et al. (2017) utilized a qualitative approach to evaluate the effectiveness of policy initiatives by exploring farmers’ perceptions, introducing a novel psychosocial dimension to policy assessment [46]. Pereda and Lucchesi (2022) proposed two frameworks from a global holistic perspective to evaluate mitigation policies in the maritime sector, offering a fresh outlook on assessing global climate change policies [47]. In a related context, Yang et al. (2022) introduced a novel quantitative tool for evaluating traditional medicine policies based on the PMC index model, enabling a comprehensive assessment of policies related to the development of traditional Chinese medicine [48]. Wang and Xing (2022) also made a significant contribution to policy assessment by integrating text mining techniques with PMC index models to evaluate the scientific rigour of policies derived from China’s wind power industry policies, introducing a new framework for assessing renewable energy policies [49]. Additionally, Liu and Zhao (2022) developed indexes for assessing the effectiveness of textile industrial policies based on document analysis, integrating theory with the PMC index model and introducing a novel quantitative method for evaluating traditional industrial policies [50].

The review of existing literature reveals that prior research efforts have often utilized various quantitative techniques such as microsimulation models, PMC indices, and mixed-method approaches to assess environmental policies, providing diverse perspectives on policy impact [51]. However, the application of these techniques to Macao’s environmental policies remains limited. This study builds upon the foundations laid by these works, employing the PMC index model for consistency evaluation and integrating text mining for qualitative analysis. Specifically, the quantitative approach adopted here draws from Kuang et al.’s (2020) [52] use of the PMC-Index for cultivated land policies, while the integration of text-mining follows the methodology introduced by Yang et al. (2021) [53] for evaluating China’s new energy policies. Together, these approaches are adapted to address the unique context of Macao’s environmental landscape, incorporating insights from prior studies that emphasize the value of mixed methods to achieve a comprehensive policy assessment.

2.5 research status of macao’s environmental policies

3.1 Data corpus and selection criteria

This study analyzes 12 environmental policy texts issued by the Macao Special Administrative Region government between 2010 and 2024. These policy texts were obtained from the official database of the Environmental Protection Bureau (DSPA) of the Macao Special Administrative Region to ensure data authenticity and comprehensiveness. This dataset covers the evolution of environmental policies since Macao’s return to China, laying a solid foundation for this research. The selection criteria were as follows: (1) policies issued between 2010 and 2024 (2) directly related to environmental management, pollution prevention and control, or energy (3) representing different types of policies such as regulations, plans, and standards to ensure diversity. The original texts were written in Chinese and Portuguese, but official English translations were used to ensure consistency in Natural Language Processing (NLP) analysis.

3.2. Text preprocessing

To conduct meaningful analysis, raw textual data needs to be preprocessed. This process includes text cleaning and normalization. Text classification and preprocessing are critical for ensuring analysis accuracy. Following best practices outlined by Danesh and Dastani (2023) [62], we implemented a multi-stage preprocessing pipeline including tokenization, stopword removal, and domain-specific filtering. Their work on COVID-19 publication classification demonstrates that careful preprocessing significantly improves downstream analysis quality, a principle we applied to our policy document analysis.

The specific steps are as follows:

1. (1). Tokenization: Breaking down text into individual words.
2. (2). Removing punctuation: Eliminating punctuation marks and special characters not needed for analysis.
3. (3). Removing stopwords: One of the most important steps is removing stopwords (e.g., “the”, “is”, “in” and other common words that are grammatically necessary but have relatively low semantic value). Using only standard stopword lists may not be sufficient in the context of specific domains. Therefore, this study first used the standard English stopword list provided by the Python Natural Language Toolkit (NLTK) library and expanded upon it.

I created a custom stopword list. This custom list includes words identified in preliminary analysis as meaningless in the context of this research (e.g., “conclude”, “specifically”, “government”), as well as routine administrative terminology that does not reflect substantive policy content (e.g., “regulation”, “article”). This customized approach enhances the accuracy of the analysis and ensures that the resulting keywords more precisely reflect the core content of Macao’s environmental policies.

3.3. High-frequency word analysis

After preprocessing, text mining methods were used to identify important terms in policy documents. Further analysis of word frequency using the Term High-frequency word method highlights the importance. These terms, Policy text high frequency words statistics in Table 1, with a detailed visualization provided in Fig 1. These findings confirm that these terms not only occur frequently but also carry substantial importance as keywords in shaping the discourse of Macao’s environmental policies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Policy Text High-Frequency Words Statistics (Frequency > 100).

https://doi.org/10.1371/journal.pone.0336786.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. High Frequency Word Chart.

https://doi.org/10.1371/journal.pone.0336786.g001

From the distribution characteristics of high-frequency words, Macao’s environmental policies present distinct management-oriented features. “Administration”, as the most frequent word, fully reflects the high emphasis on administrative management functions in policy texts. This phenomenon is consistent with findings in research literature that word frequency analysis can reflect the core concerns and policy orientation of policymakers.Combined with the high frequency of legal terms such as “Regulations” and “Directive”), it constructs a policy system framework with administrative management at its core and legal regulations as support.

In the field of environmental pollution control, high-frequency vocabulary shows obvious focusing characteristics. The high frequency of “Gas”, “Electricity”, and “Emission” reflects Macao’s environmental policies’ key focus on energy use and pollutant emission control. This vocabulary distribution pattern aligns with the method mentioned in the literature of identifying policy priority areas through high-frequency word analysis. Particularly noteworthy is the prominent position of transportation-related vocabulary such as “Vehicle” and “Charging”, revealing the unique challenges Macao faces as a high-density city in traffic pollution control.

From the perspective of policy tool application, economic means occupy an important position. “Funding”, as the second most frequent word after administrative management, combined with economic vocabulary such as “Cost” and “Grant”, reflects Macao’s environmental policies’ emphasis on promoting environmental protection goals through economic incentive mechanisms. In the regulatory enforcement dimension, the high frequency of words like “Fine”and “Inspection” demonstrates the strict enforcement characteristics of Macao’s environmental policies. These words form an organic combination with administrative management vocabulary, constructing a complete policy chain of “formulation-implementation-supervision”. As stated in the literature, semantic network analysis can reveal the structural relationships between high-frequency words, and the co-occurrence relationship between these regulatory vocabulary and management vocabulary reflects the systematization and rigor of policy implementation.Regarding infrastructure construction, vocabulary such as “Pipeline”, “Facility”, and “Installation” reflects Macao’s environmental policies’ emphasis on environmental infrastructure construction. This vocabulary distribution characteristic is consistent with the methodology mentioned in research that “word frequency statistics can identify important themes in policy texts”, reflecting policymakers’ attention to hardware facility investment.Particularly noteworthy is the appearance of regional and institutional vocabulary such as “Environmental Protection Agency”and “Macao Special Administrative Region”, which reflects the local characteristics and institutional arrangements of Macao’s environmental policies. This vocabulary feature echoes the viewpoint in the literature that “policy text analysis needs to consider regional characteristics and institutional background”.Comprehensive analysis shows that the high-frequency vocabulary of Macao’s environmental policies presents obvious hierarchical structure and internal connections. From management systems to specific measures, from economic means to technical support, it forms a multi-dimensional, all-round policy system. This vocabulary distribution pattern not only reflects Macao’s practical needs for environmental governance in limited space but also embodies its unique path of exploring environmental governance modernization under the “One Country, Two Systems” framework. Through in-depth analysis of these high-frequency words, it provides important textual evidence for understanding and evaluating the characteristics and effects of Macao’s environmental policies.

3.2 Semantic network topic modeling

To identify broader themes beyond individual words in policy documents, we applied semantic network analysis and topic modeling (as shown in Table 2), Social Network Semantic Graph. The table displays node-level properties (degree, weighted degree, modularity class) rather than pairwise co-occurrence frequencies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Social Network Semantic Graph.

https://doi.org/10.1371/journal.pone.0336786.t002

Technical explanation: From this network, we calculated: (1) Degree: number of unique terms each word co-occurs with; (2) Weighted: sum of co-occurrence frequencies; (3)Category: thematic community assignment from Louvain clustering. These metrics reveal each term’s semantic centrality and thematic role.

A thematic clustering diagram (Fig 2), constructed using the semantic web co-occurrence matrix and a web semantic model, identifies five critical themes in the quantitative assessment of environmental policies in the Macao Special Administrative Region (Macao SAR). These themes collectively form the framework for Macao’s environmental protection initiatives and reflect the multidimensional efforts of the Macao SAR government in environmental governance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Thematic clustering of the semantic network.

https://doi.org/10.1371/journal.pone.0336786.g002

3.6. Integration of text mining and PMC index

The methodological innovation of this study lies in integrating two methods: text mining and the PMC index. This integration is based on a sequential explanatory mixed research design.(1) Exploratory Phase (Text Mining): As an initial step, text mining was used for exploratory analysis of the entire corpus of Macao’s environmental policy documents. As previously mentioned, during this phase, major concepts, priorities, and means in policy discourse were identified (high-frequency words, semantic themes). These qualitative findings empirically revealed “what” Macao’s policies focus on.(2) Evaluation Phase (PMC Index): Subsequently, we directly applied insights obtained from text mining to construct and validate the PMC index model. The PMC model requires a series of structured variables to evaluate policies. The results of text mining provided an empirical foundation for ensuring these variables are relevant and comprehensive in the Macao context. For example, text mining highlighted themes such as “pollutant emission management” and “financial management,” confirming the rationale for specifically including X₆ (Policy Focus) and X₈ (Policy Themes) in the variables of the PMC model.

Through this integration, this study was able to move beyond merely describing policy content or conducting context-free abstract assessments. Text mining established the assessment framework on a data foundation, while the PMC index quantitatively evaluated the systematization and consistency of identified themes in policy design. Thus, these two methods complemented each other, generating synergy between qualitative insights and quantitative rigor in the dynamic assessment of Macao’s environmental policies.

4.1. Policy examples for PMC analysis

After text mining analysis of 36 policies, we selected 12 policies representing multiple aspects of Macao’s environmental governance for more detailed quantitative assessment. The selection of these policies was based on the following criteria: (1) covering diverse environmental domains including air, water, waste, energy, etc.; (2) representing policy developments from 2011 to 2022, capturing policy evolution; (3) policies considered to have significant impact on environmental outcomes. These policies originate from the legal department of the Macao Special Administrative Region and are summarized in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Summary of the sample of Macao’s environmental policy effectiveness evaluation.

https://doi.org/10.1371/journal.pone.0336786.t003

This study evaluates the effectiveness of environmental policies in the Macao SAR by employing the Policy Modeling Consistency Index (PMC Index) and incorporating the Omnia Mobilis hypothesis proposed by Ruiz Estrada et al. [52]. he PMC index model, which utilizes binary values (zeros and ones) to quantify each variable, provides an objective framework for analyzing the strengths, weaknesses, and internal consistency of policies [53].

The selection of the 12 representative environmental policies was guided by several criteria to ensure comprehensive representation. These criteria included: (1) Diversity of environmental focus areas to capture the spectrum of Macao’s policy efforts, including pollution control, energy management, and public engagement; (2) Representation of different stages of policy evolution from 2010 to 2024 to include both foundational and contemporary policy measures; (3) Perceived impact and significance of the policies in shaping environmental outcomes; and (4) Availability of sufficient documentation to enable consistent application of the PMC index model and text mining analysis. The 12 selected policies are intended to serve as a representative cross-section of Macao’s environmental strategies, addressing key areas such as emissions control, energy efficiency, and waste management. However, it is acknowledged that certain specific areas—such as biodiversity conservation and community-specific initiatives—may have limited representation in this selection. While these 12 policies provide a broad overview of Macao’s environmental governance efforts, the complexity and multifaceted nature of environmental policy-making necessitate further research to include other areas that may also play a significant role in the overall sustainability strategy of Macao.

4.2. PMC index modeling and variable definition

Based on detailed analysis of relevant literature and the results of text mining, this study establishes a refined policy assessment framework consisting of 9 primary indicators and 50 secondary indicators. These indicators provide a comprehensive foundation for evaluating the effectiveness and scope of environmental policies in the Macao Special Administrative Region.

1. Policy Type (X₁): Assesses the role and function of policies through 6 secondary aspects such as prediction, regulation, consultation, description, guidance, etc.
2. Policy Timeliness (X₂): Focuses on the time frame of policies, including three categories: long-term (over 10 years), medium-term (5-10 years), and short-term (1-5 years).
3. Policy Constraints (X₃): Includes 11 secondary indicators such as regulations, emission limits, supervision, penalties, standards implementation, funding, subsidies, etc., used to evaluate policy incentive and constraint mechanisms.
4. Policy Domain (X₄): Assesses the application areas and scope of impact of policies through 5 aspects such as quality management, green development, risk response, citizen participation, etc.
5. Policy Evaluation (X₅): Evaluates the rationality and scientific basis of policies through 4 aspects such as sufficient evidence, clear objectives, scientific planning, etc.
6. Policy Focus (X₆): Covers 10 important aspects such as environmental protection, energy management, urban greening, waste treatment, etc., evaluating priority areas.
7. Policy Scope (X₇): Assesses the geographical and administrative applicability of policies, emphasizing their comprehensive application range and universal relevance.
8. Policy Themes (X₈): Explores the emphasis on major themes such as pollutant emission management, energy efficiency, environmental regulations, etc.
9. Policy Openness (X₉): Evaluates policy transparency, ensuring public access to information, thereby fostering trust and inclusiveness.

To clearly distinguish between X6 (Policy Focus) and X8 (Policy Themes), this study defines them as follows. “Policy Focus” refers to the broad strategic priority areas pursued by policies (e.g., environmental protection as an overall goal). “Policy Themes,” on the other hand, are more specific and operational issues explicitly articulated in policy texts (e.g., pollutant emission monitoring, financial management). Through this distinction, we can assess both the strategic intent and specific content of policies.To ensure the legitimacy of the selection and definition of each variable, we constructed a detailed indicator system. Table 4 shows the primary and secondary variables, their assessment criteria, and the literature or theoretical basis supporting their adoption.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Scoring criteria for secondary variables.

https://doi.org/10.1371/journal.pone.0336786.t004

4.3. Model construction and calculation

Using the MSAR environmental policy evaluation indicator system, a multi-input, multi-output table can be easily derived for analyzing the content of various policy texts. This table serves as a structured data analysis framework for quantitatively assessing variables in policy texts. To maintain balance between variables, the assessment primarily uses binary values. Based on this table, the PMC index value for each policy can be calculated.

The environmental policy texts of the Macao Special Administrative Region were analyzed, and values for secondary variables were assigned according to formulas (1) and (2). Here, the values of secondary variables range from 0 to 1. The values of primary variables were calculated using formula (3), with scores ranging from 1 to 9. Finally, formula (4) calculates the PMC index of Macao’s environmental policies by summing the primary variable scores for each policy.

This approach provides a systematic and quantitative means of evaluating the effectiveness of environmental policies, ensuring a balanced and objective analysis while offering insights into areas for potential improvement.

(1)

(2)

(3)

(4)

Using the PMC index calculation method in conjunction with text mining, the input-output table for the 12 Macao environmental policy portfolios is derived, as shown in Table 5. Policies are evaluated and rated according to scores and evaluation criteria detailed in Table 6. Subsequently, the PMC index for each environmental policy is calculated and summarized in Fig 3, referred to as the Macao Environmental Policy Score Table. In line with the specified criteria and requirements, the scores for each policy dimension are visually represented in Fig 3, providing a clear depiction of the performance across various dimensions of Macao’s environmental policies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Policy rating scale.

https://doi.org/10.1371/journal.pone.0336786.t005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Scores of the 12 environmental policies of Macao.

https://doi.org/10.1371/journal.pone.0336786.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Evaluation of the Effectiveness of Environmental Policies in the Macao SAR.

https://doi.org/10.1371/journal.pone.0336786.g003

To ensure the robustness of the PMC index-based policy evaluations, we have incorporated a sensitivity analysis to examine the influence of changing the weightings of different variables on the overall policy scores. This analysis aims to assess the extent to which variations in variable weights impact the consistency and effectiveness ratings of Macao’s environmental policies. By doing so, we can evaluate whether certain variables disproportionately influence the outcomes and identify areas that require careful consideration during policy assessment. We have now strengthened the integration between our text mining findings and PMC index results. In Table 7 Mapping The Correspondence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Mapping The Correspondence.

https://doi.org/10.1371/journal.pone.0336786.t007

To conduct the sensitivity analysis, we adjusted the weights of the nine primary variables by ±10% and recalculated the PMC indices for all twelve policies included in the study. The goal was to determine whether significant changes in the weights of these variables would result in substantial shifts in the final policy scores and rankings. The sensitivity analysis revealed that the rankings of policies remained generally consistent, even when primary variable weights were varied by ±10%. The sensitivity analysis has provided important insights into the robustness of the PMC index model used for evaluating Macao’s environmental policies.

To enhance the visualization of the PMC index results, a PMC surface can be created once the PMC index has been calculated. Given that there are nine first-level variables, these are organized into a 3 × 3 matrix for the surface representation. The calculation of the PMC surface is performed using Equation (5), which provides a structured approach to mapping the results into a visual format.

(5)

4.4 Quantitative results assessment

4.5. Sensitivity analysis

To ensure the robustness of policy assessments based on the PMC index, we incorporated sensitivity analysis to study the impact of different variable weights on overall policy scores. This analysis aims to evaluate the extent to which fluctuations in variable weights affect the consistency and effectiveness of evaluating Macao’s environmental policies.

To conduct the sensitivity analysis, we adjusted the weights of the 9 primary variables by ±10% and recalculated the PMC index for the 12 studied policies. The goal was to determine whether substantially changing the weights of these variables would have a material impact on the final policy scores and rankings.

The results of the sensitivity analysis indicate that even with ±10% fluctuations in the weights of the primary variables, policy rankings remain largely consistent.

5.1 Conclusion

This study investigated the policy conditions of the Macao Special Administrative Region government after its return through text mining and PMC index analysis, providing quantitative assessments of 12 representative environmental policies. Text mining revealed that Macao’s environmental policies primarily focus on core themes such as regulation and governance, energy and emissions, incentives and sanctions, infrastructure and technology, and transportation and vehicles. The PMC index analysis results indicate that the evaluated policies were all rated at an “acceptable” level, showing reasonable policy design, but with room for improvement in aspects such as policy timeliness (lack of long-term planning) and scope (limitations in regional cooperation).

This study makes several important contributions.Methodological Contribution: This study developed and applied an innovative framework integrating text mining and the PMC index model for dynamic assessment of environmental policies in the unique governance environment of a Special Administrative Region. This mixed research approach combines data-driven discovery of policy content with systematic quantitative assessment of policy design, surpassing traditional evaluation methods.This study conducted the first comprehensive and quantitative assessment of Macao’s environmental policies since its return. This empirically clarified specific strengths (e.g., effectiveness of evaluation, openness) and weaknesses (e.g., lack of long-term vision) of policies that were previously understood mainly in descriptive ways.This study provides evidence-based tools and specific recommendations for Macao’s policymakers to refine existing policies and guide future environmental governance. For example, the research identifies areas requiring specific interventions for “policy constraints” and “policy focus” aspects that received lower PMC scores.

The assessment results of this study provide valuable insights for the continuous improvement of environmental policies in the Macao Special Administrative Region. Policymakers should address weaknesses identified in the PMC model, particularly strengthening long-term strategic planning and regional cooperation.International best practices provide valuable insights into areas where Macao’s environmental policies can be further developed.

Germany’s Energy Transition (Energiewende): Through sustained policy support, such as the Renewable Energy Act (EEG), Germany has successfully increased the proportion of renewable energy significantly. In 2023, renewable energy accounted for more than half of Germany’s electricity consumption for the first time, setting a historic record. Macao can learn from Germany’s experience about the importance of long-term incentive mechanisms and stable legal frameworks for accelerating the introduction of renewable energy.

Japan’s Circular Economy Framework: Japan was one of the first countries to introduce circular economy policies, beginning with the “Basic Act for Establishing a Sound Material-Cycle Society” in 2000. Japan’s framework emphasizes Extended Producer Responsibility (EPR) through specific laws such as the Container and Packaging Recycling Law and the Home Appliance Recycling Law, and thoroughly manages waste throughout product lifecycles. The implementation of the “Act on Promotion of Resource Circulation for Plastics” in 2022 further strengthened these efforts. Macao can reference this comprehensive and targeted legal approach to enhance waste management and resource efficiency.By learning from these international cases, Macao can move toward a more advanced environmental policy framework that addresses both regional needs and global sustainability challenges.

The quantitative methods adopted in this study provide a structured and replicable means of evaluating policy consistency and focus, but this approach has inherent limitations. Particularly in capturing the complete picture of policy effectiveness, it is difficult to encompass stakeholders’ subjective experiences and challenges encountered during implementation. The absence of qualitative insights may limit understanding of how policies are perceived and accepted by the public and relevant industries.To more comprehensively assess policy effectiveness, it is recommended to employ qualitative methods, such as stakeholder interviews, in addition to quantitative analysis. Interviews with policy implementers, industry representatives, local residents, and non-governmental organizations would provide valuable insights into subjective experiences, challenges, and successes related to policies. These perspectives help identify gaps that cannot be discovered through quantitative analysis alone and provide a more complete comprehensive picture of the strengths and weaknesses of Macao’s environmental policy framework. By adopting a mixed research approach, we can gain a more comprehensive understanding of the effectiveness of Macao’s environmental policies.

5.2 Policy recommendations

To address the significant deficiency in Policy Timeliness (X2). Propose a three-tiered temporal planning framework. First, Macao should establish rolling five-year environmental master plans with mandatory review cycles, institutionally linked to the SAR government’s five-year development plans to ensure policy continuity and political commitment. Second, the government should develop comprehensive 2030 and 2050 environmental target frameworks with specific, measurable milestones for carbon neutrality, air quality standards, and waste reduction targets that align with China’s national environmental goals while accommodating Macao’s unique characteristics as a Special Administrative Region. Third, long-term policy implementation requires stable financial support; therefore, we recommend creating dedicated long-term funding commitments through establishment of an Environmental Development Fund with multi-year budget allocations insulated from annual fiscal volatility.

Regarding the weakness in Policy Scope (X7), reflecting Macao’s limited regional integration, propose comprehensive measures to strengthen cross-border environmental cooperation within the Greater Bay Area framework. The first priority involves formalizing environmental policy coordination mechanisms through three specific institutional arrangements: establishing quarterly environmental policy coordination meetings with Guangdong and Hong Kong authorities to ensure regular communication and joint planning; creating joint air quality monitoring networks with shared real-time data platforms that transcend administrative boundaries; and harmonizing emission standards for cross-border vehicles and vessels to eliminate regulatory arbitrage and improve overall regional environmental performance. Beyond institutional mechanisms, Macao should pursue tangible cross-border environmental infrastructure projects, including joint waste treatment facilities with capacity-sharing agreements to address land constraints, integrated renewable energy grid connections to enhance energy security and sustainability, and coordinated coastal water quality management systems leveraging the shared marine environment of the Pearl River Delta.

Finally, to operationalize regional cooperation, Recommend implementing three categories of policy pilot programs: cross-recognition of environmental certifications across the Greater Bay Area to reduce bureaucratic barriers and promote best practices; joint green procurement standards for government projects to create economies of scale and market transformation; and coordinated environmental impact assessment procedures for regional infrastructure projects to ensure comprehensive evaluation of transboundary environmental effects. These specific, actionable recommendations transform the general observation that policies require “strengthening long-term planning and regional cooperation” into concrete measures with clear implementation pathways, institutional arrangements, and expected outcomes.

5.3 Applicability analysis of international best practices to Macao

Germany’s Energiewende offers important lessons regarding the effectiveness of long-term policy frameworks and renewable energy incentives. However, direct application to Macao faces substantial structural constraints that necessitate careful adaptation. The most fundamental challenge stems from Macao’s severely limited territory, which precludes the large-scale solar farms and wind installations that characterize Germany’s renewable energy infrastructure. To address this constraint, Macao must focus on spatially efficient alternatives, including building-integrated photovoltaics that utilize existing vertical surfaces, offshore wind collaboration with neighboring regions that leverage shared marine areas, and cross-border renewable energy procurement agreements within the Greater Bay Area framework that allow Macao to meet environmental targets through regional cooperation. Japan’s comprehensive Extended Producer Responsibility framework presents a particularly relevant model for Macao, though implementation requires substantial adaptation to account for institutional and scale differences. The structural similarities between Japan and Macao are striking. Both jurisdictions face severe land constraints, extraordinarily high population densities, and limited capacity for traditional landfill-based waste management.

This applicability analysis demonstrates that while international best practices provide valuable conceptual frameworks and proven policy instruments, successful policy transfer requires critical contextual analysis that accounts for Macao’s unique characteristics: land constraints, Special Administrative Region governance structure, tourism-dependent economic model, and strategic position within the Greater Bay Area regional system. Effective borrowing policy is therefore not imitation but rather adaptive translation that preserves core principles while fundamentally reconceptualizing implementation mechanisms to fit local realities.