Background

In this section, we present the main concepts necessary for a better understanding of this work. We begin by describing some of the areas related to computer science that we made use of; then, we overview the methods used for topic modeling and document classification.

Supervised learning for document classification.

In supervised learning, classification refers to assigning a category to each item [12]. Classification problems are (1) mono-label multiclass, i.e., two or more classes and each item getting only one label; or (2) multi-label multiclass, i.e., two or more classes, and each item getting two or more labels. Regardless of the type, document classification requires first obtaining numerical representations for text and training a classifier. A simplification of the standard process is presented in Fig 1, based on [18].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Supervised learning for document classification.

https://doi.org/10.1371/journal.pone.0271741.g001

The process has two parts: training and testing. When dealing with supervised learning, it is crucial to test the models on samples other than those from the training set, as the training optimizes the results for its own collection. Therefore, one more set, the test one, is kept apart from the first and used for the final evaluation of the model.

Both training and testing also include pre-processing. Pre-processing often includes removing stopwords (recurrent vocables that may not contribute to the classification like“the”, “a”), applying lemmatization, and normalizing document length. Approaches like tf-idf and embeddings are commonly used for textual representations, whereas classification strategies like SVMs are generally employed for modeling.

In this work, we explore tf-idf in tandem with SVM as well as word embeddings together with convolutional and recurrent neural networks for decrees classification. We adopted this approach because tf-idf combined with SVMs [2, 19] and certain neural network architectures are known to perform text classification accurately [20].

Document classification and topic modeling

The longitudinal and large N-data of presidents’ administrative decisions enable researchers to uncover how the government is driving policies, executing laws, and allocating budget resources. However, these administrative decisions’ volume and complexity have inhibited their regular use by political scientists, legislators, lawyers, experts, and whoever is affected by these decrees. Since 1990, more than ten thousand decrees have been issued by Brazilian presidents. Tools for automated text methods have been increasingly applied to overcome these limits. According to the researcher’s substantive interests, these decisions’ complexity requires more sophisticated text-as-data methods for estimating valid measures. Such methods are crucial to understanding presidential policy-making, especially in Brazil since decrees do not identify their respective policy areas.

Topic modeling is a relevant NLP approach to extract information out of a textual dataset in an unsupervised manner. LDA is one of the algorithms that can be used to perform this task, however its results depend on the amount of text given to the algorithm. Specifically in the case of LDA, samples with short texts, e.g. tweets [21], can lead to less coherent results than samples with long texts. Therefore, some works discuss different procedures to outperform traditional LDA specifically in short sample cases [22, 23]. Another possible approach to deal with short texts is to merge micro texts into longer texts before performing the topic modeling, as done by Mehrotra et al. [24] for tweets. The discussion of how the input of the LDA affects its coherence becomes particularly relevant in cases like ours, in which there are two representations of the data: a short abstract with denser information and the full text of the administrative decree (see Materials and methods).

Another approach to explore massive textual content is through text classification. Text classification has already been explored in the legal domain. Sulea et al. [7] develop an SVM ensemble for predicting the law area of cases and rulings from a collection of documents from the French Supreme Court. Their work reports 96.6% of f1-score in this task, focusing on the benefits of automatic text classification to support law professionals’ decisions.

Also in the Judicial Branch, Undavia et al. [9] discuss some approaches with neural networks for text classification in a United States Supreme Court dataset. The authors also explore taxonomies, similarly to what we have proposed for administrative decrees. In their case, two taxonomies were explored: one broader with 15 classes, similar to ours, and another with finer-grained classes, with more than 250 labels. Their convolutional neural network (CNN) presents results with more than two times accuracy over the broader taxonomies than over the fine-grained. For the broader, the authors report 72.4% of accuracy with the CNN, while for the fine-grained, 31.9%. In our case, instead of exploring the effects of different granularities, we explored the differences between two taxonomies with different focuses.

As for the Legislative Branch, O’Neil et al. [25] applied topic modeling in legal texts of the United Kingdom. To tune the LDA parameter in a fair manner, they counted on a legal practitioner that manually assessed their models. The legal practitioner was in charge of defining the number of topics of the LDA and assisting in the definition of the main parameter of the LDAvis [26] visualization tool. Their approach evidence how computational methods should be combined with the specific knowledge from the research area in order to get comprehensive insights about the constitution of the data explored through topic modeling. In turn, Chalkidis et al. [8] present a comparison of approaches for classification of the European Union documents. Their work tends to an extreme multi-label text classification task, as they have dealt with thousands of labels, some of them also with few or zero samples in the test set. They tested many approaches, including HAN, a convolutional network, and some other models like BERT [27]. Including all labels, their best result was given by BERT, with 73.2% of micro-f1.

Regarding the Executive Branch, Ruhl et al. [10] discuss topic modeling for analyzing a collection of executive orders from the United States of America. They manually explore the coherence and semantic meaning of the generated topics. For extensive collections of texts, they also advocate for machine learning in the legal domain as tools to accelerate analysis and give insights about the content of documents, though still not replacing humans, only facilitating some processes. Kaufman [6] in turn presents potential applications of machine learning in the future of political science analysis in the Executive Branch of the United States.

As the language particularities are factors that can influence many textual approaches, it is important to also take a look at what has been done in Portuguese, especially in Brazil and for texts from the political science domain. In Brazil, most of the work applying computational methods to political science documents has been done in the Judicial Branch. Luz de Araujo et al. [4] present the Brazilian Supreme Court database and several approaches for automatic classification in this domain, including an SVM, an LSTM, and a CNN. On their medium dataset, the best result for document classification is given by their LSTM, with 70.92% macro-f1. Da Silva et al. [28] address the same problem, focusing on the primary studies performed with a standard convolutional neural network for text classification that has gotten 91% of f1-score in a small dataset. Both works refer to the Supreme Court’s initiative to speed up the internal process of text categorization in the courts of Brazil.

Finally, using structural topic modeling (STM) for an unsupervised machine learning model, Inácio and Recch [29] identified and compared major policy areas targeted by Brazilian presidents. Notable differences in the prevalence of topics reveal that presidential attention varies considerably when presidents take unilateral actions by issuing decrees. While some presidents issued plenty of decrees to implement social policies, others prioritized changes in regulatory policies and state reforms.

Complex networks

The analysis of complex networks has also been employed in many domains with the intentions of characterizing the structure created by links in a collection, exploring specific groups, and finding important individuals. In the context of information retrieval, for example, the web has been represented as a graph by Broder et al. [30]. This representation allowed the authors to explore this network’s formation from different perspectives, from micro to macro levels. In the political science realm, models based on graphs also have been extensively used. Ward et al. [5], for instance, present some of those applications and the available methods for network exploration, indicating the possibilities and advances for mapping the relations between individuals and institutions from the political environment.

Networks have been used to describe the relations among decisions described in political documents. In New Zealand, records from the Legislative Branch have been modeled through a network [31]. Similarly, Bommarito et al. [32] discuss different perspectives of the networks generated by the courts’ documents of the United States. In turn, Mazzega et al. [33] present the structure of French legal codes based on the existing citations between those texts. Based on that, the authors also notice a rich club of documents, which tend to be more cited and therefore may perform a more significant role.

Also, Koniaris et al. [34] describe the citation network comprising legal texts from the European Union, focusing on the macro aspects of this network. The authors observe general elements of this structure and relate it with the network formation throughout time. Also, a parallel can be drawn between the macroscopic properties of this network and the one presented by Bommarito et al. [32], as both networks are well represented by a bow-tie decomposition. We did not find any specific work in Brazil applying complex networks’ approaches to model collections of administrative decrees. Table 1 summarizes the works selected in this section.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Related work.

https://doi.org/10.1371/journal.pone.0271741.t001

Data gathering and standard pre-processing

Decree links are grouped by year on the Executive Branch website. We accessed each of those links and crawled the decrees. As usual in static pages on the web, the HTML pattern available at each year page was not standard; therefore, manual fixes had to be done on the crawling algorithm.

We used web scraping to obtain and filter the data available at decrees pages automatically. The terms and conditions for the website applied while the decrees were extracted are detailed by Ordinance number 1,492 from October 5, 2011 (see http://www.planalto.gov.br/ccivil_03/Portaria/P1492-11-ccivil.htm—accessed on April 23, 2022). This ordinance defines that the use of the platform is granted for sharing and creating non-profit derivatives, provided the source and the date of access and not including national symbols. No specific extraction method is interposed by those terms. As specified by them, we also state that the texts presented in the website and extracted for this work have only informational purposes.

Our main goal while crawling those pages was to extract the full text of the decree and its abstract, loading it on the local database. We also removed all appendixes from the final text during this process and saved every HTML anchor link available in the document. Finally, we focused on obtaining the related ministries, decree identifications, and the president who has signed the decree for each table. All this data has been structured in an SQLite database.

Before running any machine learning technique over decrees, we applied a tailor-made pre-processing that removes signatures at the end of each document, lower-cases texts, removes special non-Latin characters, and standardizes punctuation. Signatures are removed to ensure that signatories of decrees do not influence classifications and insights given by the models.

Data enrichment

Though the table of each decree already presents a tagging system (in this paper, we refer to tags as the terms that are used to describe the decrees on the original databases, and to labels as the descriptions given after the creation of our taxonomies) based on subjects and law fields, these tags present low granularity, with thousands of unique terms for our collection of decrees. Therefore, searching or classifying decrees through these tags is challenging and involves dealing with highly rare or ambiguous terms. Instead, we opted to enrich the dataset with tags provided by other instances of the government.

In Brazil, the Legislative Branch plays a primary role in gathering all data from legal documents, including the Executive Branch documents. The Brazilian Senate presents an Application Programming Interface (API) that includes information from administrative decrees, with another tagging (labeling) system that deals with fewer tags (approximately 500) and more aggregated information. Using the API calls, we matched the Senate tags with each of the collected decrees and proposed a new taxonomy based on these tags named Senate-based taxonomy.

Unsupervised text analysis with LDA

We applied topic modeling in two distinct representations of each decree using a LDA model. First, we have explored the behavior of this model only with the abstract of each decree, a less specific representation, but with denser information. Second, we have worked with the concatenation of the abstract with the decree’s full text, a representation with wider information. Moreover, we have defined the number of topics for each approach using the best parameter selection search with the Grid Search algorithm, the method discussed by Feurer et al. [35]. Grid Search constructs multiple LDA models for all possible combinations of parameter values and compares them until finding the highest set of parameter values. In this case, we have maximized the statistical model log-likelihood, which measures the coherence of unseen data with data already processed [11], to evaluate the similarity between topics.

After the LDA process, we have used the LDAvis. In this step, we have searched for the best presentation of the data that shows a meaningful list of terms in the topic. Therefore, we have set the optimal λ to 0.4 through a case study observing the results given by different λ across all topics and labeling each one manually.

Taxonomies

Based on the unsupervised experiments’ insights and on the tags and information already available and gathered on our database, we present two taxonomies for the classification of administrative decrees.

The first is the cabinet-based taxonomy, which is based on the nomenclature of the ministerial portfolios in each administration at the federal level of government. The identification of those portfolios is based on the ministers who sign each decree. The second taxonomy is based on groups of tags, related to the fields of law, provided by the Senate API.

Both taxonomies present high granularity. It is worth noting that we can increase sampling and cope with changes in the Executive organization by grouping related areas, including nominal and cabinet portfolio changes. However, the flip side is that we lose the specificity of each area.

The creation of the taxonomies is illustrated in Fig 3. Basically, we have manually organized the set of ministerial portfolios in groups. The same process was performed for the Senate-based taxonomy, but in that case, using the set of the Senate tags.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Process of creation of taxonomies.

https://doi.org/10.1371/journal.pone.0271741.g003

The process of creation of taxonomies considered the composition of the presidential cabinet over time; the fields of law in each decree; and the topics identified in the analysis of the LDA. Therefore, we consider the LDA model an exploratory step to get a full comprehension of the administrative decrees’ dataset before creating the taxonomies. An important caveat is that we do not compare LDA models with the two taxonomies, as their nature and methodologies are different.

Supervised text classification

The organization of administrative decrees in taxonomy groups provides a simple but relevant indicator of the actions and priorities of the Executive Branch over time. However, maintaining the entire Brazilian database updated with those indicators is a process that takes both time and money. In this context, supervised machine learning techniques can provide mechanisms to reproduce those taxonomies for new decrees. The models generated by those techniques allow us to immediately infer which areas are affected mainly by the recent acts of the Executive and also provide a consistent point of view between different time changes in the organization of cabinets, portfolios, and themes.

The challenge is this: given a set of decrees and their classes, can we reproduce the logical mechanisms that link those decrees to their taxonomies? To address that, we made use of three well-established supervised learning techniques. One is based on SVM [2], while the other two are based on neural networks, respectively, a convolutional neural network inspired in that of [28] and a hierarchical neural network, designed for document classification [3]. We also ensemble the results of those three methods in a majority voting classifier. Besides, we used both the abstract and the decree’s full text as input to the models, based on the results given by the LDA analysis.

To further test the quality of the models obtained, we begin by separating 20% of the data for evaluation. The rest of the data is broken into four other sets used for training and tuning the hyper-parameters of the models based on a 4-fold-cross validation. For hyper-parameter tuning, we made use of Bayesian optimization for efficiently searching in wide spaces of parameters. For that, each of the four sets was used as validation for a model trained with the other three. The mean of the macro F1-score across these models was used as the metric to evaluate the performance of the hyper-parameters in each iteration of the optimization.

The technique’s approach and classes’ distributions are described in S2 File, together with the distributions of the classes of each set and with the best hyper-parameters found during optimization. Broadly, we used the same stratified sets for all techniques. We also applied pre-processing methods to those decrees according to what we found most appropriate based on the literature of each approach.

Network analysis

We also have modeled the collection of decrees as a graph. We begin by creating a set of nodes N with all decrees of our collection. Then, given a node a representing a decree D_a and a node b representing a decree D_b, a directed edge exists from b to a if and only if D_a was published earlier than D_b and D_b has a link to D_a. Links are obtained directly from the HTML of each decree. The set of pairs (b, a) originates the set of directed edges, E_d, of our graph. Finally, we keep only nodes with one or more incident edges to them. The final graph is then given by G_d(N, E_d).

The first level of analysis in this graph is made by looking at the most “central” nodes. Different centrality measures have been used to estimate the importance of each node of a network, but two simple varieties are the degree centrality and the betweenness centrality. The degree of a node is given by the number of nodes adjacent to it. In our case, as we have modeled a directed graph, we give degree centrality by the sum of in-degree (edges coming into the node) and out-degree (edges coming out of a node) of a node. So, in practice, degree centrality indicates how much nodes are connected to other nodes. To compute betweenness, in turn, one uses all the shortest paths between any two nodes that include the target one and outputs an indicator of how much the network structure would be affected if the target node had been removed. For more details on betweenness and degree centralities, please refer to [36].

The second level of analysis is performed by exploring the mesoscopic structure of our network through the perspective of modularity classes. Modularity indicates community structures in a network [37]. Particularly, we made use of the algorithm given by [38], implemented on Gephi (see Gephi 0.9.2 visualization and exploration software: https://gephi.org/—accessed on November 20, 2021), to separate our network into clusters. Then, we analyze the structure given by those clusters from our taxonomies’ perspective. We do not perform any specific parameter tuning during this analysis (the results were obtained using resolution 1.0 and with randomness by default), and we emphasize that the approach implemented by Gephi is a heuristic with results that can vary due to randomness. Hence, we face these results from an analytical perspective, with the primary intention of finding and exploring specific regions of the graph, and also suggest further studies in this direction.

Unsupervised text analysis with LDA

In this section, we present the results of the LDA algorithm in two different analyses. First, we have provided only the abstract of the decrees as an input to the algorithm. Then, in the second analysis, we have provided to LDA the entire content of the decrees.

Decree’s abstract analysis.

Fig 4 illustrates the results of the abstract analysis with 10 topics. In this case, we opted to show, as an example, the top-30 most relevant terms for Topic 2. For an overall visualization of these results, Table 2 details each of the 10 estimated topics, with the top-5 most relevant terms and a description of each topic.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. LDAvis results for abstract only.

https://doi.org/10.1371/journal.pone.0271741.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Top 5 LDAvis terms for abstract analysis using λ = 0.4.

https://doi.org/10.1371/journal.pone.0271741.t002

Here, we briefly describe the main findings from this topic model. An example of near and correlated topics, with an overlapped area, is demonstrated by topics 1 and 9. They are easily detected in Fig 4. They both refer to terms related to the type of decision made. They discriminated different decisions, Topic 1 being more related to executive directives for the execution of laws and Topic 9 more associated with administrative orders.

Presidents issue this type of decree to put the executive branch into operation. Terms referring to the regulation of the civil service and giving directions for bureaucracies are captured by prevalent terms of topics 2 and 5, with the slightly overlapping circles. Topic 4 refers to the federal level of public administration in charge of managing budgetary resources and national policies. It is sided by Topic 8 which captures the budgetary decisions, which are critical to government tasks. It refers to budget allocation among federal administration units, but also inter-branches transfers or to sub-national governments. Interestingly, it is not correlated to topic 3, which alludes more to state funding; tax regulation, and collection, as depicted in the Map Fig 4.

One of the distinctly different clusters is formed by topics 6 and 7, isolated in the Map. The former comprise terms associated with foreign affairs and diplomatic issues, while the latter refers to commercial agreements and economic cooperation treaties. It is worth noticing that these documents have a very peculiar semantic structure, aligned with international laws, and they write by the diplomatic bureaucracies. Finally, topic 10 is relatively ambiguous, including military-related functional issues, but also nonspecific words.

As in this section we have worked with only the abstracts, which are short samples of text, we expected that the LDA would not be able to detect more specific topics, as well as illustrated by Zhao et al. [21]. Moreover, the abstract data does not precisely identify the theme, but rather a short description of the subject of the decrees. Therefore, we applied another analysis with the full decree.

Full decree analysis.

We present the results of the LDA with the full text of the decrees in Table 3, which discriminates the 25 estimated topics together with the top-5 most relevant terms and a thematic description of them. An overall view of the cluster of these estimated topics is depicted by Fig 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. LDAvis results for full decree.

https://doi.org/10.1371/journal.pone.0271741.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Top 5 LDAvis terms for full decree using λ = 0.4.

https://doi.org/10.1371/journal.pone.0271741.t003

Different from the models with the abstracts, the models with the full text of the decrees discriminate the decisions of the Executive Branch in the same regulatory or administrative area. For example, Fig 5 presents the clusters formed by the topics, detailing for topic 5 the top-30 most relevant terms.

At the top of this figure, we can see the cluster formed by topics 5, 6, and 10, which have overlapping areas. They refer to the commissions ordinarily formed in support of the decision-making process within the Executive Branch and the agencies it regulates. However, the topics discriminate the advisory commissions linked to ministries (top 5) of sectoral collegiate bodies (top 10) and regulatory decisions of these structures (top 6). In the lower-left quadrant of the figure, we have a cluster formed by several topics related to the administrative activities of the Executive Branch. An example are the decisions on taxation and credits, differentiated by decisions on the taxation of imports (topic 3) and taxation of domestic activities and products. Finally, as observed in the analysis of the abstracts of the decrees, the topics related to decisions on foreign policy and international agreements (topics 18 and 20) are distant from the others in the intertopic distance map. As already noted, this distance reflects the boundaries between domestic policy and foreign policy, a recurrent trait of contemporary states.

Furthermore, a brief description of each topic is given in Table 3.

Taxonomies

We present 13 classes for the cabinets-based taxonomy. Those classes focus on the organizational structure of the Executive Branch in Brazil during the time course of the database. A brief description of each is given in Table 4 that shows the main cabinets related to each class.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Cabinets-based taxonomy description.

https://doi.org/10.1371/journal.pone.0271741.t004

In turn, we present 15 classes for the Senate-based taxonomy (Table 5). Those classes focus on the legal effects resulting from each decree.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Senate-based taxonomy description.

https://doi.org/10.1371/journal.pone.0271741.t005

These classes cover policy areas targeted by decrees already identified by the unsupervised analysis, for instance, defense, foreign affairs, and economic regulation. Decrees related to these salient policy areas reveal great text consistency and a tendency to form clusters, as demonstrated by LDA. However, presidential decree-making in Brazil encompasses a variety of policies and administrative decisions. In addition, cross-cutting issues are regulated by different executive agencies, and their relevance reflects new legislation or changes in government agendas and priorities. For instance, culture- and sports-related policies have been implemented by the Minister of Education or by specific portfolios created to manage these policies. Thus, both taxonomies reflect and categorize central issues driving the issuance of administrative decrees across several administrations.

However, these two taxonomies differ in their concepts and purposes. The cabinets-based taxonomy reflects how ministries’ jurisdictions are set up in each administration. I.e., the policy areas and administrative tasks in charge of each ministry. It reveals presidents’ choices regarding the size and design of the cabinet. For example, while one president implements social security and welfare policies through two different ministries, others prefer to merge them into a single portfolio. These choices related to the cabinet size are commonplace in Brazil, where the number of ministerial portfolios has oscillated from 9 up to 39 ministries. The Senate’s classification is a legal taxonomy that classifies and labels decrees by fields of law and their legal effects. Hence, it links decrees, as non-statutory decisions, to the Brazilian legal system. Since legal issues are cross-cutting to different ministries, we expect other clusters to emerge from our topic modeling strategies based on these two taxonomies.

It is also usual to find decrees that affect one or more areas or are co-signed by two or more cabinets. Therefore, those classes are modeled in a multi-label schema. But how often does each area collaborate with another? Fig 6 presents a matrix illustrating how ministerial portfolios collaborate with each other while signing decrees. It is worth noting the central role played by the EOP group, which refers to the decisions made by the units of the Executive Office of the President, in the decree-making process. In support of the president’s decisions and inter-ministerial coordination, the Economy group plays a decisive gate-keeping power in the executive’s decrees. These units evaluate the economic and budgetary impact of the decrees, including those requested by other ministries. In consequence, these groups, Economy and EOP, tend to sign a large number of decrees together.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Co-sponsorship of cabinets’ policy-areas.

The main diagonal stands for the number of decrees with only one label.

https://doi.org/10.1371/journal.pone.0271741.g006

Notably, these patterns of co-authorship among ministries were also observed in the drafting of the Executive’s bill proposals before they were introduced to Congress. Rennó and Wojcik [39] note that the ministers of the Finance Ministry, Planning Ministry, Justice Ministry, Development, Industry and Foreign Trade Ministry, Labor Ministry, and Social Security Ministry co-signed, together with the president, the largest number of Executive Branch bill proposals from 1995 to 2010.

While the pivotal role is notorious for our cabinets-based classification, the same is not observed in the Senate-based groups. Fig 7 illustrates the collaboration across fields of law. In that case, we observe that the class Executive Branch organization is the one that tends to collaborate with every other, while the Economy does not draw so much attention. This illustrates the differences between those two taxonomies and indicates that cabinet acts might go beyond the main areas of their portfolios.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Collaboration between areas from the perspective of the Senate-based taxonomy.

The main diagonal stands for the number of decrees with only one label.

https://doi.org/10.1371/journal.pone.0271741.g007

While the pivotal roles of the EOP and Economy groups impact our cabinet-based classification, a different pattern emerges when we use the Senate-based groups. Fig 7 illustrates decrees labeled in one (main diagonal) or more fields of the law. In this case, we observe that the class Executive Branch organization is the one that tends to overlap with other classes, while the Economy class does not show broad transversality. It points out the differences between the two taxonomies we use in this analysis. It suggests that multi-labeling is more frequent in cabinets-based classification, because the ministries are in charge of different components of public policy (budget, administration, departmentalized bureaucracies) or depend on several executive agencies (sectoral, centralized and decentralized organs).

Also, to explore how these two taxonomies overlap, Fig 8 shows the intersection between the classes of them. We highlight that some classes have similar descriptions and labels in both classifications, however, we have to bear in mind that these two taxonomies are based on conceptually distinct classifications of the contents of the decrees. Indeed some similar classes reveal consistency across taxonomies, e.g. Foreign affairs, Security, and Education (all homonyms in both taxonomies). However, the fields of law affect cross-cutting policies executed by different ministerial portfolios. In particular, Industry (note that this class also has a homonym in the cabinets-based taxonomy), Taxes, and Executive Branch Organization decrees from the perspective of the Senate-based classification tend to map to Cabinets related to the Economy class. This relates to the pivotal role of the ministries of Economy and Finance in Brazil and also illustrates how the characteristics explored by the taxonomies differ. The classification based on ministerial policy areas shows a more heterogeneous set of decisions, since cabinet units are more sensitive to political and administrative dynamics and their actions are hardly constrained to the fields of law.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Overlap between the Senate and cabinets-based taxonomies.

https://doi.org/10.1371/journal.pone.0271741.g008

Finally, it is worth noting that there are decrees unable to match any class: 165 for the Senate-based and 42 for the cabinets-based. We decided to remove those decrees from the dataset while performing the supervised text classification analysis, keeping only decrees classified by both taxonomies, i.e., 6,635 decrees.

Temporal analysis.

Taxonomies can also be used to infer the priorities of the Executive Branch across different administrations and situations. Figs 9 and 10 present the annual number of decrees related to each class respectively for the Senate and for the cabinets-based taxonomies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Number of decrees per year from the perspective of the Senate-based taxonomy focusing on the top 4 areas most issued in 2019.

https://doi.org/10.1371/journal.pone.0271741.g009

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Number of decrees per year from the perspective of the cabinets-based taxonomy focusing on the top 4 areas most issued in 2019.

https://doi.org/10.1371/journal.pone.0271741.g010

From the Senate-based taxonomy perspective, most classes have decrees well distributed all over time. The number of decrees classified as Executive Branch Organization, however, peaked in 2019. This relates to the inauguration of the Bolsonaro’s administration in 2019, when the organization of the Federal Executive agencies has changed considerably.

The inter-temporal comparison of the number of decrees classified by the cabinet-based taxonomy shows interesting trends. First, economic issues concentrated the largest number of decrees issued by the president, despite slight fluctuations at some points in this time series. Second, the number of decrees in some classes has increased considerably since 2016. The number of decrees classified as Economy, EOP, and Security has peaked, reflecting constitutional and legal reforms approved by Congress and changes in public administration prioritized by the Temer (2016–2019) and Bolsonaro (2019-) governments. In particular, the advance in security decrees in Bolsonaro’s presidential term signals changes in the president’s policy priorities implemented administratively, not through new legislation. Therefore, this outcome is not captured by the Senate-based taxonomy. Finally, in the reverse direction, the number of decrees classified as Foreign Affairs has decreased since 2014.

Supervised text classification

Table 6 presents the F1-score metric given by each of the approaches we have tested for the Senate-based classification. Regarding macro and micro averages, HAN is the single classifier that produces the best result on the test set; however, both macro and micro averages are quite similar to the ones of the SVM approach, which is a simpler and computationally cheaper method. It is worth noticing that some classes are accurately classified regardless of the used technique. This is the case for Foreign Affairs (F1-score always > 95%), Taxes (>90%), and Security, Executive Branch organization, Mines and energy, and labor (> 80%). Note that some of those areas relate to topics previously shown by the unsupervised analysis. Also, note that most results got better through majority voting, i.e., by combining the techniques. However, the combination of techniques requires the computation of all techniques over the dataset and, therefore, can be computationally expensive.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. F1-score for the Senate-based taxonomy models.

https://doi.org/10.1371/journal.pone.0271741.t006

We also highlight classes where all the supervised techniques tend to fail more. Health, Industry, and Justice are areas with this tendency when compared with the others in our database. In particular, Justice is the single class whose results were all under 70%. One reason that might explain this behavior for Industry and Justice is that those are classes that aggregate a vast set of different matters, that might also interfere in topics related to other classes.

Finally, results for the Health class can relate to the fact that, in Brazil, the most frequently used regulation to the health area is given through Ministerial Ordinances [40, 41], another type of textual decision issued by ministers on topics related to their respective policy area’s jurisdiction. Therefore, though some decrees are associated with this area, that is unlikely the most accurate tracking of Health decisions. A possible approach to improve results for this area would be enriching our training sets with samples from related Ministerial Ordinances to indicate better the particular terminologies and decisions of this area.

Table 7 presents F1-score for the cabinets-based taxonomy. Here, the Convolutional Neural Network outperforms the other two, indicating that no single approach is best for all taxonomies. Still, HAN and SVM approaches get close to those results on average and also outperform the Convolutional Neural Network in some particular classes. Some tendencies observed for the Senate-based classification can also be observed here. In particular, the performance of Foreign Affairs class is also greater than all other classes. Most of those results can again get better by ensembling the models through the majority voting system. However, we again emphasize the computational costs related to ensembles.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. F1-score for the cabinets-based taxonomy models.

https://doi.org/10.1371/journal.pone.0271741.t007

We reassert that, though the idea of some areas in our two taxonomies is similar, the classification given for some samples can be pretty distinct, as the focus of each taxonomy is distinct. Still, we note that similar results are observed for some classes with similar descriptions to the ones of the Senate, for instance, the results given for Health class, and the already mentioned results for Foreign Affairs. Results for classes such as Labor, however, are notoriously different—in this particular case, worse—given the differences between the methodologies of each taxonomy.

The most notorious result relates to the EOP class, for which there is no similar class in the Senate-based taxonomy. This class is specifically related to the organization of the Presidency and decisions centralized by the Presidency’s units. Hence, it does not fit well in the Senate-based taxonomy. This class features the worst result, meaning all classifiers get results under 51%. This result can relate to the wide variety of topics, contexts, and situations that encompasses EOP. This, in turn, makes EOP classification based on textual features more challenging.

Networks analysis

As stated previously, another question to be answered with the analysis of administrative decrees is: Is it possible to identify priority policy issues or areas in each administration based on thematic clustering, co-authorship, and inter-connectivity of the decrees? To answer that, it is worth noting that new laws or changes in presidents’ priorities lead to new decrees or the review of existing directives. Therefore, decrees commonly link previous directives or different policy areas in case of inter-ministerial decisions. Hence, by mapping these links, we can uncover the dynamics of the decree-making process in various regulatory areas and their impacts on policy outcomes. In this section, we have modeled our collection of decrees as a graph to explore further the intrinsic mechanisms related to Brazilian administrative decrees. Table 8 presents the main characteristics of this graph.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Characteristics of the administrative decrees network.

https://doi.org/10.1371/journal.pone.0271741.t008

We note that 2,352 decrees of our 20-year-old collection (2000–2019) do not link to any other decree. We call them disconnected decrees and depict their proportion organized per year in Fig 11.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Disconnected decrees per year.

Proportion of disconnected decrees given by (number of disconnected decrees)÷(total number of decrees in a year).

https://doi.org/10.1371/journal.pone.0271741.g011

Note that the administration’s first years display a lower proportion of these disconnected decrees: 2003, 2007, 2011, 2016, and 2019. We believe that the number of disconnected decrees is so because the presidents’ first unilateral acts focus on revoking decrees from previous governments and on issuing decrees with decisions closer to their political preferences. The exception to this trend was in 2015 when the president resorts to a large proportion of disconnected decrees to cope with the political crisis and, after, to cope with the impeachment proceedings that paralyzed the administration from day one of the second presidential term of Dilma Roussef.

Last, per our analysis, decrees are disconnected if they do not relate to any other decree of our collection. Therefore, our set of disconnected decrees may include some related to other decrees issued outside the period 2000–2019. This abstraction, in turn, suggests that further analysis encompassing a more extended period is possible.

Next, we explore aspects of the modeled network from the microscopic and mesoscopic levels, like other analyses done for web links (e.g., [42]). S4 File give an indicative of the macroscopic structure of the disposition of administrative decrees online.

Microscopic properties of the administrative decrees’ network.

Microscopic properties relate to individual nodes in a graph. Nevertheless, to present an overview of the most central elements of our network, we show in Table 9 the proportion of those nodes that relate to each of our taxonomies classes. By doing so, we aim to indicate which sorts of decrees carry the structure of the presented network and which areas play central roles. The analysis given by this table is based on a collection that includes only the 100 most central decrees according to each centrality. Varying the number of decrees in this analysis might lead to different perceptions of importance across areas that can be further explored by future works.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Taxonomies’ 100 most central nodes.

https://doi.org/10.1371/journal.pone.0271741.t009

Per the Senate-based taxonomy, the top 100 most central decrees most relate to the industry. By individually analyzing these industry decrees, we observe that 44 of them relate to the PAC cluster (see section “Mesoscopic properties of the administrative decrees’ network”) due to its high density, which implies a high degree of centrality. This area is followed by Executive Branch organization and Economy classes. From the perspective of the cabinets-based taxonomy, the majority of those nodes relate to Economy. For betweenness centrality, based on the Senate, the majority relates to the Executive Branch organization area, while based on cabinets, most of them relates to Economy.

Table 10 gives the top 5 decrees according to the degree centrality, the centrality values, and the labels given by the Senate and cabinets-based taxonomies. Most of them are from 2019 and refer to decrees revoking many other decrees. Thus, those usually relate to the Executive Branch Organization from the perspective of the Senate-based taxonomy. From the perspective of the cabinets-based taxonomy, those relate mostly to EOP.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Top 5 decrees according to degree centrality.

https://doi.org/10.1371/journal.pone.0271741.t010

Table 11 gives the same analysis according to betweenness centrality. In this case, the top decrees usually relate to positions and commissions, therefore those are usually classified as Executive Branch Organization and Economy. We also note that those decrees are older than the ones with greater degree-centrality. That relates to the fact that to increase betweenness centrality a decree should both cite and get cited by other decrees.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 11. Top 5 decrees according to betweenness centrality.

https://doi.org/10.1371/journal.pone.0271741.t011

It is worth noticing that Economic-related ministerial portfolios, such as finance and planning, play an essential role in maintaining the network’s structure. Those are decrees related to the execution of economic plans, budgetary decisions, public investments, and others. This centrality of the economic area points to the internal hierarchies of the Executive structures, demonstrated here by the gatekeeper role of some of them exercised through presidential decrees.

Mesoscopic properties of the administrative decrees network.

In this level of analysis, our network has gotten divided into 409 groups, indicating the presence of small groups with related decisions. We observe 29 communities of size (number of nodes) greater than 1% of the network (44 nodes) that are detailed and illustrated in S5 File.

Most of those large groups of decrees relate to Executive Branch organization, usually due to changes in its organizational units and regulations of the civil service. Still, it is also possible to observe highly homogeneous clusters related to Foreign Affairs, Taxes, and Environment. Most of those large groups relate to specific areas that are properly illustrated by our taxonomies. This observation indicates that the characteristics of this network take influence from the area and authorship of each administrative decree. Yet, some areas generate or influence several clusters, particularly, Executive Branch organization and Economy. Those clusters also have more links to others of similar composition.

One of the most engaging large clusters comprises a little more than 1% of the network’s nodes, but with more than 13% of the network’s edges. By checking the abstracts and texts of this cluster, we observe that this group relates to a major economic plan called Growth Acceleration Program—PAC (or Programa de Aceleração de Crescimento—PAC, in Portuguese). PAC execution took place during the presidents Silva’s and Rousseff’s terms, starting in 2007 and discontinued in 2014. This program required the intensive issuance of decrees to finance and execute public investments and programs in infrastructure, transportation, energy production and connectivity, and public housing throughout this period.

The influence of the PAC cluster has also been observed in the microscopic properties of the network, particularly in the degree-centrality analysis. Table 12 gives its full constitution regarding the Senate and cabinets taxonomies. It is possible to observe that the constitution of this cluster is also highly homogeneous from the perspective of both the taxonomies presented in this work.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 12. PAC cluster: Senate and cabinets taxonomies.

Areas that are not present in this cluster were omitted.

https://doi.org/10.1371/journal.pone.0271741.t012

The presence of the PAC cluster can extend the perception of clusters created by particular areas, including the observation that specific regions of the network model might capture major economic plans handled by the Executive. We cannot verify whether this pattern also matches with older economic plans because of our data period. Yet, an interesting question is whether economic plans tend to create more dense regions in similar decisions networks. We encourage future works to explore this kind of group in the context of administrative decrees in Brazil and other countries. Notably, we believe that similar results can be found with various other methods for community analysis that can help tracking and uncovering groups of related decisions.