Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A user-friendly tool to evaluate the effectiveness of no-take marine reserves

  • Juan Carlos Villaseñor-Derbez ,

    Contributed equally to this work with: Juan Carlos Villaseñor-Derbez, Caio Faro, Melaina Wright, Jael Martínez, Christopher Costello

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Bren School of Environmental Science and Management, University of California Santa Barbara, Santa Barbara, California, United States of America

  • Caio Faro ,

    Contributed equally to this work with: Juan Carlos Villaseñor-Derbez, Caio Faro, Melaina Wright, Jael Martínez, Christopher Costello

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing

    Affiliation Bren School of Environmental Science and Management, University of California Santa Barbara, Santa Barbara, California, United States of America

  • Melaina Wright ,

    Contributed equally to this work with: Juan Carlos Villaseñor-Derbez, Caio Faro, Melaina Wright, Jael Martínez, Christopher Costello

    Roles Conceptualization, Investigation, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

    Affiliation Bren School of Environmental Science and Management, University of California Santa Barbara, Santa Barbara, California, United States of America

  • Jael Martínez ,

    Contributed equally to this work with: Juan Carlos Villaseñor-Derbez, Caio Faro, Melaina Wright, Jael Martínez, Christopher Costello

    Roles Conceptualization, Investigation, Methodology, Project administration, Writing – original draft

    Affiliation Bren School of Environmental Science and Management, University of California Santa Barbara, Santa Barbara, California, United States of America

  • Sean Fitzgerald ,

    Roles Conceptualization, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Bren School of Environmental Science and Management, University of California Santa Barbara, Santa Barbara, California, United States of America

  • Stuart Fulton ,

    Roles Conceptualization, Funding acquisition, Supervision, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Comunidad y Biodiversidad A.C., Calle Isla del Peruano, Guaymas, Sonora, México

  • Maria del Mar Mancha-Cisneros ,

    Roles Conceptualization, Methodology, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation School of Life Sciences, Arizona State University, Tempe, Arizona, United States of America

  • Gavin McDonald ,

    Roles Conceptualization, Methodology, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliations Bren School of Environmental Science and Management, University of California Santa Barbara, Santa Barbara, California, United States of America, Sustainable Fisheries Group, University of California Santa Barbara, Santa Barbara, California, United States of America, Marine Science Institute, University of California Santa Barbara, Santa Barbara, California, United States of America

  • Fiorenza Micheli ,

    Roles Conceptualization, Methodology, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Hopkins Marine Station and Center for Ocean Solutions, Stanford University, Pacific Grove, CA, 93950, United States of America

  • Alvin Suárez ,

    Roles Conceptualization, Funding acquisition, Supervision, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Comunidad y Biodiversidad A.C., Calle Isla del Peruano, Guaymas, Sonora, México

  • Jorge Torre ,

    Roles Conceptualization, Funding acquisition, Supervision, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Comunidad y Biodiversidad A.C., Calle Isla del Peruano, Guaymas, Sonora, México

  • Christopher Costello

    Contributed equally to this work with: Juan Carlos Villaseñor-Derbez, Caio Faro, Melaina Wright, Jael Martínez, Christopher Costello

    Roles Conceptualization, Investigation, Methodology, Supervision, Validation, Writing – review & editing

    Affiliations Bren School of Environmental Science and Management, University of California Santa Barbara, Santa Barbara, California, United States of America, Sustainable Fisheries Group, University of California Santa Barbara, Santa Barbara, California, United States of America, Marine Science Institute, University of California Santa Barbara, Santa Barbara, California, United States of America


Marine reserves are implemented to achieve a variety of objectives, but are seldom rigorously evaluated to determine whether those objectives are met. In the rare cases when evaluations do take place, they typically focus on ecological indicators and ignore other relevant objectives such as socioeconomics and governance. And regardless of the objectives, the diversity of locations, monitoring protocols, and analysis approaches hinder the ability to compare results across case studies. Moreover, analysis and evaluation of reserves is generally conducted by outside researchers, not the reserve managers or users, plausibly thereby hindering effective local management and rapid response to change. We present a framework and tool, called “MAREA”, to overcome these challenges. Its purpose is to evaluate the extent to which any given reserve has achieved its stated objectives. MAREA provides specific guidance on data collection and formatting, and then conducts rigorous causal inference analysis based on data input by the user, providing real-time outputs about the effectiveness of the reserve. MAREA’s ease of use, standardization of state-of-the-art inference methods, and ability to analyze marine reserve effectiveness across ecological, socioeconomic, and governance objectives could dramatically further our understanding and support of effective marine reserve management.


Unsustainable fishing practices threaten biodiversity, conservation, economic and social outcomes [1, 2]. Marine Protected Areas (MPAs; and marine reserves, in which all extractive efforts are prohibited) are frequently proposed to aid in the recovery of fish and invertebrate stocks [36] by limiting or restricting fishing effort and gears.

Empirical evidence shows that MPAs increase biomass [4, 7], enhance resilience to climatic impacts [8, 9], and preserve genetic diversity [10]. Compared to MPAs that grant partial protection, marine reserves have higher levels of biomass, density, richness, and larger organisms [3, 1113]. However, these effects are often measured as biological changes within the reserves through time, and many lack a control site for comparison [14].This approach does not account for other factors (e.g. system-level changes in productivity caused by predatory release [15]; or favorable environmental conditions [16]) for which one must control [17] in order to causally attribute a biological change to the reserve. Other studies have used a control-impact comparison approach that uses control sites but does not address temporal variability [4, 7, 1820].

A smaller fraction of studies have used a before-after-control-impact (i.e. BACI) design comparing reserves to control sites before and after implementation [4, 21, 22], which allows the use of causal inference techniques that estimate the effect of the reserve. For example, in ref [21] authors use a BACI design and observe increases in lobster catches –a proxy for abundances– after reserve implementation for protected and control sites. However, the temporal changes in the reserve were greater than in the control site, suggesting a positive effect of the reserve on lobster catches. But even when proper causal inference can be drawn, results are often different across reserves. Effects of reserves on ecological and economic outcomes are highly heterogeneous, and often depend on the specific ecological, economic, and social context.

Standardization of marine reserve evaluation is not new. The IUCN framework “How is your MPA doing?” [23, 24] provides a comprehensive list of biological, socioeconomic, and governance indicators, and insights into how these may be measured or collected. But this framework stops short of analysis, and provides a user with little guidance about establishing causal inference about the reserve. Recent work [25] integrates these three dimensions via the Social Ecological Systems Framework [26, 27] and suggests the use of causal inference techniques to provide a measure of the effect of conservation interventions. However, neither of these approaches provide a user-friendly tool that ensures replicability and scalability of the analysis, particularly when used by the fishers and decision makers themselves.

An increasingly popular way to make science accessible, reproducible, scalable, and replicable is through Open Science and the development of open-access tools [28]. The Ocean Health Index [29, 30], for example, successfully standardized a way to measure the health and benefits of the oceans. This approach has been implemented at a global scale, but also at country-level [31], and regionally [32, 33]. Open access tools are not limited to conservation, and have also been developed to evaluate fishery performance [34, 35], design territorial use rights for fisheries (TURFs, [36]), and improve decision making in the hydro power industry [37].

The purpose of this paper is to describe a user-friendly tool, called “MAREA”, to rigorously systematize the evaluation of marine reserve effectiveness in terms of fisheries and marine conservation goals. The tool is in the form of an open-source application that uses state-of-the-art methods from program evaluation to compare a reserve to control sites along a number of biological, economic, and governance dimensions. We first provide a list of commonly stated management objectives and match them to appropriate indicators. We then develop a simple approach to analyzing these indicators building on causal inference techniques [21], which help us understand the effect of management interventions [25, 38]. To implement the analytical approach, we introduce the Marine Reserve Evaluation Application (MAREA), an open source, web–based tool that automates the framework described in this paper and enables its broader use. Finally, we present a case study on the evaluation of a marine reserve established by the fishers of Isla Natividad (Mexico) in 2006, to demonstrate the potential of MAREA.

Materials and methods

Here, we describe the proposed framework to evaluate the effectiveness of marine reserves (Fig 1). We explain how management objectives were identified and matched to appropriate indicators that allow the evaluation of the reserves, and provide brief guidelines on data collection. Alongside, methodologies to analyze these indicators are presented. We then describe the development of MAREA and explain how this tool can be used by fishermen, managers, and other stakeholders with little scientific background. Finally, we provide guidelines on how to interpret and use the results and output generated by MAREA to inform management.

Fig 1. Workflow to evaluate the effectiveness of marine reserves.

Marine reserve objectives and indicators

Throughout this study, we will refer to the stated goals for which a marine reserve was designed as “objectives.” This work was motivated by the case of Mexico, where 39 reserves have been implemented over the past five years to achieve objectives such as increasing productivity in nearby waters or recovery of overexploited species; most of these reserves have never been formally evaluated for effectiveness at meeting those objectives. Thus, our focus was on identifying common objectives of marine reserves in Mexico. However, a literature review and discussions with marine reserve researchers suggested that the objectives driving Mexican marine reserve implementation are similar to those in the rest of the world. Thus, we group these objectives into seven major categories that may be applied to marine reserves worldwide. Any given reserve may have been implemented to meet one or more of these. The list includes objectives stated in legislation [39, 40] and official documents such as the Technical Justification Studies (Estudios Técnicos Justificativos), agreements, and decrees associated to these areas:

  1. Avoid overexploitation
  2. Conserve species under a special protection regime
  3. Maintain biological processes (reproduction, recruitment, growth, feeding)
  4. Improve fishery production in adjacent waters
  5. Preserve biological diversity and the ecosystem
  6. Recover overexploited species
  7. Recover species of economic interest

Based on these seven objectives, we determined a set of associated indicators to evaluate reserve effectiveness. These indicators are specific variables on which data could be collected and analyzed, to ultimately determine whether the corresponding objective was causally being achieved by the marine reserve. The list of indicators was compiled through a review of scientific literature in which we identified indicators that were used to measure similar objectives [35, 7, 11, 13, 14, 1821, 23, 24, 4144]. A first filter eliminated indicators for which baseline data do not typically exist in Mexico. The preliminary list of indicators was reviewed at a workshop with participation of members from Mexican fishery management agencies and non-government organizations. Later, these were presented to fishers from the Ensenada Fishing Cooperative (S.C.P.P. Ensenada), in El Rosario, Baja California, who provided input. Our final list of indicators includes those identified in review works [4, 44].

Indicators are divided into three main categories: biological, socioeconomic, and governance (Table 1). The nine biological indicators focus on fish and invertebrate communities that are evaluated using underwater ecological surveys performed inside and outside the reserve (see Data and Analysis section for specific sampling design and methodologies). Five socioeconomic indicators reflect the performance of the fishery in terms of landings, income from landings, and availability of alternative livelihoods. Fifteen governance indicators describe the governance structures under which the community operates (e.g., access rights to the fishery, number of fishers, legal recognition of the reserve). Most biological and socioeconomic indicators are quantitative and require a numerical entry (e.g. Fish biomass) while all governance indicators, one biological indicator, and one socioeconomic indicator are qualitative and rely on a descriptive entry (e.g. Reasoning for reserve location). Many of them specifically measure an outcome of the reserve, though some are designed to further the understanding of the mechanisms driving a reserve’s performance. In that sense, most biological and socioeconomic indicators are outcome variables. On the other hand, governance indicators are viewed as possible explanatory variables of reserve performance. Whenever an indicator is applied to “Target species”, it means that the indicator can be used for all species (e.g. Fish Biomass) and/or for individual species that are either the conservation target of the reserve or are of particular economic or ecological interest (e.g. Grouper Biomass). Finally, indicators B3 and B4 are different in that B3 only looks at the density of organisms above size at first maturity (related to reproductive potential), while B4 measures the density of all fish or of a target species. Each indicator targets different plausible desired outcomes, like increased reproductive potential (i.e. B3; [45]) or having more fish -regardless of their size- to attract tourism (i.e. B4). Table 1 presents the proposed indicators, and Table 2 shows how objectives are matched with biological and socioeconomic indicators. Governance indicators are excluded from Table 2, but should be considered for every objective as each serves as a plausible explanatory variable for reserve performance.

Table 1. List of indicators to evaluate the effectiveness of no-take marine reserves.

Table 2. Management objectives and respective performance indicators.

Data and analyses

In many coastal marine reserves, biological data are often collected via underwater visual censuses as part of a reserve’s monitoring program. Scientific divers record fish and invertebrate richness and abundances, as well as fish total length along belt transects. Ecological surveys are typically performed annually in each reserve and corresponding control site(s), before and after the implementation of the reserve, providing a sampling design that can be used to draw causal inference. Control sites are areas where habitat is similar to that of the reserve, but with presence of fishing activity; in principle these are areas that are otherwise observationally identical to the reserve site, but where, for presumably random reasons, a reserve was not implemented. While transect dimensions (i.e. length and width) and sampling methods might vary from study to study, the general idea remains the same: richness, abundances, and sizes of organisms are recorded in a study–specific standardized way. For this reason, MAREA does not assume specific transect dimensions, and pertinent indicators are calculated per transect (Table 1). More information on data collection and formatting is provided in a guidebook [46], which is available in English and Spanish in MAREA’s interface.

This sampling design for biological data allows us to use causal inference techniques [21, 47] to evaluate the effect of the reserve on biological indicators. The hypothesis that the indicators will respond to implementation of the reserve is tested by analyzing spatial and temporal changes in each numeric biological indicator (all but B5) using generalized linear models [21]. To account for variations in the environment and survey conditions, covariates that are gathered during the underwater ecological surveys are included in the difference-in-differences model with form: (1)

In this model, i, t, and z are indices for transect, time, and zone (control or reserve site), respectively. This model allows us to estimate the change in an indicator (I) based on the year (Y), a dummy variable that indicates treatment (Z; i.e. control or reserve), an interaction between a dummy variable that indicates before or after implementation (P) and treatment (Z), and covariates such as bottom temperature (T; in °C), horizontal visibility during the survey (V; in m), and depth at which survey was performed (D; in m). ϵ represents the error term associated to the regression. Here, years are modeled as factors, using the first year as the reference level. This does not impose a linear structure in the way an indicator changes through time (i.e. the change in biomass between 2006 and 2007 does not have to be the same as the change between 2015 and 2016). The treatment and implementation variables, modeled as dummy variables, are coded as Control = 0 and Reserve = 1; and Before implementation = 0 and After implementation = 1, respectively.

Socioeconomic data are often collected by fishers, natural resource management agencies, or Civil Society Organizations (CSOs) by recording landings, income, and sometimes prices for each species. To control for inflation, income is adjusted with the country’s consumer price index [48]: (2)

Where It represents the adjusted income for year t as the product between the reported income for that year and the ratio between the consumer price index (CPI) in that year to the most recent year’s (T) CPI. Since no control sites are typically available for this data type, numeric socioeconomic indicators (S1 and S2) are evaluated with a simplified version of Eq (1):


This model does not formally allow for causal inference, but we can still measure changes in mean landings and income before and after the implementation of the reserve and provide valuable input. For both models (Eqs (1) and (3)), we estimate the model coefficients with ordinary least squares, and calculate heteroskedastic–robust standard errors [49].

While biological and some economic data are regularly collected, governance data are typically not available nor systematically collected by the community or other organizations. Therefore, we created a survey specifically designed to collect information needed for the proposed indicators (B5, S3, and G1–G15). The survey is included as supplementary material in English (S1 Appendix) and Spanish (S2 Appendix). To analyze governance information, we developed a framework based on a literature review of common governance structures and their relation to effectiveness in managing fisheries or marine reserves (S1 Table). This approach has been proven to successfully evaluate governance structures [50]. Unlike with biological and socioeconomic objectives (see Eqs (1) and (3)), MAREA does not quantitatively analyze governance information. Rather, it is presented along with the biological and socioeconomic indicators to provide managers and users with a more complete description of the reserve.

Marine Reserve Evaluation App (MAREA)

We developed MAREA in R version 3.4.2 and R Studio 1.1.383 [51] using the Shiny package [52], to build an interactive web application hosted on an open server; the MAREA app can be accessed at While the original version was developed in Spanish because it was aimed for Mexico and other Latin-American countries, all of its content can be translated by a translation widget available within the app.

MAREA is designed as a 6-step process, divided in tabs that appear upon launching the app. The first tab introduces the app and summarizes the evaluation process. Then, the user selects management objectives, which MAREA automatically matches to appropriate indicators, based on Table 2. Users can also select and deselect indicators based on their interests and data availability by “clicking” on the check-boxes in MAREA. The user can then load data on one or more reserves, using standard *.csv text files; sample datasets are provided within MAREA. Once data have been loaded, MAREA identifies all reserves in the data, and lets the user select the reserve to be evaluated. At this point, the user can also specify the year of implementation of the reserve, reserve dimensions, and indicate target species that are of particular management interest. MAREA provides the user with a section to confirm that all the decisions made leading up to that point are correct. Once the user has confirmed all input data, objectives, and other information, MAREA performs the formal program evaluation analyses discussed above. For a typical data set, the automated analysis step takes less than one second. Finally, the user is taken to the results tab where all results are presented in a simple format. The user can also download a more comprehensive technical report produced in *.pdf format.

The first output is a color–coded scorecard intended to provide a general overview of the effectiveness of the reserve. The scorecard provides a global score for the reserve, a general score for each category of indicators, and an individual score for each indicator. The global and category–level scores are determined by the percentage of positive indicators, overall and for each category, respectively. For numeric biological indicators (all but B5), the color is defined by the sign of the interaction term coefficient (β2) in Eq (1). For socioeconomic indicators, colors are assigned based on the direction of the slope (β1) in Eq (3). Red, yellow, and green are used for βi < 0, βi = 0, and βi > 0, respectively. The intensity of the color is defined by the significance of the coefficient, testing the null hypothesis of no change (i.e. H0: βi = 0) with a Student’s t-test. Cutoff values are p < 0.05 and p < 0.1. Thus, even in a case where βi > 0, if the coefficient is not significant by standard measures (i.e. p > 0.1), the indicator will be assigned a yellow color. A legend (Fig 2) is provided within the scorecard to aid in the interpretation of these results. Governance indicators are represented simply by red or green. The color is defined based on what literature shows to be a negative (red) or positive (green) factor for a reserve (S1 Table). For example, if the perceived degree of illegal fishing is high, this indicator will be assigned a red color. However, due to the nature of some governance indicators, which require the user to provide a narrative, only some indicators are presented in the scorecard (although all are included in the technical report).

Fig 2. Legend used to interpret the scorecard produced by MAREA.

Colors indicate direction of change (red = negative; green = positive), and color intensity is given by the statistical significance.

The second output from MAREA is a technical report intended to communicate information and statistical results in a more comprehensive and technical way. This report also includes a scorecard as a summary of the results, but provides more information for each indicator. For all numeric biological indicators, the report includes a graph of the value of the indicator in the reserve and control sites through time. It also provides a regression table that summarizes the value of all coefficients in the regression and their respective robust standard errors. The summary table also provides information on model fit (R2) and significance of the regression.

The scorecard is produced with functions from the Shinydashboard package [53]. The technical report is produced by a parameterized Rmarkdown document [54] processed by the knitr package [55]. Another feature of MAREA is that the user can choose to share the data. Once the technical report is downloaded, the information on the reserve, its management objectives, and all uploaded data are saved into a central repository. These data can be accessed at any time by any person interested in acquiring them at

Case study

While MAREA is a general tool that can be easily employed to evaluate the effectiveness of any marine reserve with the required input data, we illustrate its use here by applying it to one marine reserve near Isla Natividad, in Baja California Sur, Mexico. Isla Natividad is located 8 Km off the Pacific Coast of the Baja California Peninsula (Fig 3), where fishers operate under a fishing cooperative (S.C.P.P. Buzos y Pescadores de la Baja California) that promotes co-management of marine resources [56, 57]. Additionally, fishers have Territorial Use Rights for Fisheries (TURFs) that provide them with exclusive access rights to exploit the benthic marine resources within a given perimeter [57].

Fig 3. General location of Isla Natividad (left) and map of the island (right).

The marine reserve polygon is indicated in red, and the approximate location of control sites is indicated by blue squares (B = Babencho, D = La Dulce). Shapefiles for Mexican coastline and the United States were obtained from INEGI [58] and the tmap R package [59], respectively.

In 2006, the Isla Natividad community established a biological baseline following the data collection protocol described in this study. The community then implemented two community-based marine reserves within their TURF [8, 41, 60] after establishing a baseline for the soon-to-be reserves and control sites. Evidence suggest that these reserves have been effective at enhancing resilience to climate variations [8] and preserving genetic diversity of high value commercial species such as abalone [10]. These ecological benefits have been translated into economic benefits, enhancing population persistence and bolstering abalone fisheries [43]. For the purpose of this evaluation, we focused on the “La Plana / Las Cuevas” marine reserve, located at the southern end of the island (Fig 3) and its corresponding control site “La Dulce / Babencho”.

The objective of this reserve was to recover species of economic interest –which were overexploited– and to enhance fishery production in nearby waters. Fishers were also interested in preserving biological diversity and the ecosystem. Thus, objectives 4—7 were selected. Using Table 2 to match these objectives with appropriate management indicators, we selected all biological, socioeconomic, and governance indicators included as options in the framework.

Local fishers (who were trained in scientific diving by the CSO Comunidad y Biodiversidad, A.C. (COBI;, ReefCheck California, and Stanford University) and personnel from these institutions performed SCUBA dives to record fish and invertebrate richness and abundances, as well as fish total length. They recorded information along 30 m transects, with a sampling window of 2 m x 2 m following a standardized ReefCheck protocol [61]. Ecological surveys were performed yearly in each reserve and corresponding control site(s), before and after the implementation of the reserve, providing the requisite time series data inside the reserve and for a suitable control site. Annual surveys (2006–2016) were carried out in late July—early August, performing a total of 242 and 245 transects in the reserve site for fish and invertebrate surveys, respectively. Similar sampling effort was applied to the control site, with 221 fish and 222 invertebrate transects. Between 12 and 27 transects were performed in each site every year.

Socioeconomic data were obtained from the National Commission for Aquaculture and Fisheries (Comisión Nacional de Acuacultura y Pesca; CONAPESCA). The data contains species-level information on monthly landings and income from nine species from 2000 to 2014. Data on landings and income were aggregated by year and species, and adjusted by the Consumer Price Index [48]. From the nine species available, we selected as objective species those that contributed the most (88.27%) income from 2000 to 2014: lobster (Panulirus interruptus; 71.76%), red sea urchin (Mesocentrotus franciscanus; 9.33%), snail (Megastraea undosa; 3.93%), and sea cucumber (Parastichopus parvimensis; 3.23%). Abalone species (Haliotis fulgens; 4.52% and Haliotis corrugata; 6.16%) were excluded because the cooperative implemented an informal closure of these fisheries in 2010 to allow the population to recover. Eliminating all fishing pressure on abalones means that the control site receives (for this species) the same treatment as the reserve.

We constructed the governance data based on local knowledge of the area and the community.

Results from illustrative example

In this section we show the results of the application of MAREA to the La Plana/Las Cuevas marine reserve in Isla Natividad, Mexico. These results are intended to highlight the relevance and utility of the MAREA framework and app, which automate the analysis and make it replicable. While we highlight some of the general observed trends, we focus on the utility of the tool rather than on the specific effectiveness of this case study marine reserve.

The scorecard (Fig 4) shows that this reserve achieves a general score of 64%, suggesting that 64% of all indicators are positive. All category–level scores were also high, with values of 67%, 60%, and 71% positive indicators for biological, socioeconomic and governance, respectively.

Fig 4. Scorecard produced by MAREA for the “La Plana / Las Cuevas” marine reserve in Isla Natividad, Mexico.

Among the biological indicators, the greatest effect of the reserve was observed for snail and sea cucumber densities, with values of β2 = 97.17 (p < 0.05) and β2 = 2.31 (p < 0.05), respectively. Fish indicators showed no significant change (p > 0.1), with negative trends for Shannon’s diversity index and fish species richness and positive trends for density, biomass, and mean trophic level. Changes through time for these indicators are presented in Fig 5, and a summary of β2 coefficients is provided in Table 3.

Fig 5. Plots for values of each biological indicator (y-axis) through time (x-axis).

Red and blue correspond to the reserve and control sites, respectively. Black lines indicate yearly mean values, and ribbons indicate ± 1 standard error. Dots are horizontally jittered to aid visualization. This figure contains information for fish Shannon’s diversity index (a), fish species richness (b), fish density (c), fish trophic level (d), fish biomass (e), invertebrate Shannon’s diversity index (f), invertebrate species richness (g), invertebrate density (h), lobster density (i), urchin density (j), snail density (k), and sea cucumber density (l).

Table 3. Summary of average treatment effect of the reserve on biological indicators.

One of the main objectives of this reserve was to increase landings. Results of the socioeconomic indicators show that total landings were, on average, 64.20 metric tonnes higher (p > 0.1) after the implementation of the reserves, though this cannot necessarily be interpreted as causal, because it relies entirely on a before-after comparison. Total income was $10,344.85 (p < 0.05) thousands of Mexican Pesos (K MXP) higher after the implementation of the reserves. On average, lobster and sea cucumber landings increased, while urchin and snail landings and income decreased. Fig 6 presents the changes in these indicators through time, and Table 4 summarizes these results.

Fig 6. Plots for values of each socioeconomic indicator (y-axis) through time (x-axis).

Red and blue correspond to before and after the implementation of the reserve, respectively. This figure contains information for total landings (a), total income (b), lobster landings (c), urchin landings (d), snail landings (e), sea cucumber landings (f), lobster income (g), urchin income (h), snail income (i), and sea cucumber income (j).

Table 4. Summary of differences in socioeconomic indicators before and after the implementation of the reserve.

Recall that the governance objectives are evaluated based on the institutions present, not on a specific quantitative linkage between governance and biological or economic outcomes. Data for this reserve suggest that the community is strongly organized, which is a likely driver of the successes reported above [62]. The first point of success is the existence of a fishing cooperative that is also affiliated with a regional federation of cooperatives. These polycentric governance structures allow various levels of organization that have been shown to foster communication and cooperation [50, 57]; federations also provide bargain power with governments [50, 63]. Access to fishing resources is managed through a TURF, permits, and fishing quotas (for some species). It has been suggested that TURFs promotes a sense of stewardship of resources and incentivizes sustainable management [56]. Together, these structures enabled a participative, bottom–up process during the reserve design phase; opinions of all fishing members –and often non-fishing community members– were included. Participation of community members in reserve surveillance and yearly monitoring indicate commitment and interest, and allow informal communication of results to uninvolved community members. Furthermore, the reserve is partially isolated from poaching activity, and fishers have internal regulations pertaining to the reserves. The low level of illegal fishing by members of the community and outsiders both inside and outside the reserve is another indication of effectiveness. Governance indicators are summarized in Table 5.


We have developed and presented an automated approach for evaluating the effectiveness of marine reserves in Mexico, and perhaps around the world. Here we highlight MAREA’s utility for evidence-based management, and comment on a few of its shortcomings. The findings from Isla Natividad are used purely to validate the relevance of MAREA rather than to discuss particularities of the marine reserve effectiveness, which has been described before [8, 10, 43]. We use examples from the case study to build on the utility of MAREA and discuss ways in which results can be interpreted to inform management.

The causal inference techniques used by MAREA have been suggested [38, 47] and used [21] before in other ad hoc studies. This approach reduces ambiguity in the interpretation of results. For example, invertebrate density decreased through time inside and outside of the reserve (Fig 5h). In this case, a before–after evaluation of the reserve (i.e. ignoring the control site) would have incorrectly concluded that the reserve failed to protect invertebrates. On the other hand, a control–impact approach (i.e. compare reserve vs. control site only in 2016) would have identified higher densities inside the reserve, concluding that the reserve increases invertebrate density. However, by executing a formal difference-in-differences approach for causal inference, MAREA identifies the changes through time and across sites, and estimates the effect of the reserve on density at β2 = 91.21 (p < 0.05). This approach reveals that invertebrate densities decrease in both sites through time, but the decrease is faster for the control site, thus yielding a positive value for β2.

The approach used by MAREA to estimate the effect of the reserve on biological indicators requires cautious interpretation of the results. The value of the β2 coefficient represents the difference between the temporal trends of the reserve and control sites [21]. As exemplified by the case of invertebrate densities, a positive value (i.e. β2 > 0) does not necessarily indicate an increase in the indicator through time, but rather a positive difference with respect to the temporal trend of the control site. The inverse occurs for negative values of β2.

MAREA provides in-depth analysis and a convenient snapshot overview of the effect of the reserve, allowing users to rapidly identify trends. However, users must interpret multiple indicators at a time to better understand the results. For example, with additional knowledge of local environmental variability (i.e. indicator B5: Natural Disturbance), we can better understand the trends in invertebrate densities. As reported before [8], hypoxic conditions that have occurred in Isla Natividad can cause decreases in invertebrate densities, and reserves buffer the negative effect. While MAREA automates the analysis and makes results replicable, proper interpretation will still depend on the user. Results produced by MAREA can only aid in management and decision making when results have been correctly interpreted.

Socioeconomic and governance indicators typically lack a control site, which impede us from using the causal inference techniques employed to measure biological changes [25]. However, we can still extract useful information from them. Again, by combining results from multiple indicators, MAREA can provide insights into the effect of the reserve. For example, lobster and sea cucumber have shown increases in densities, landings, and income. We cannot conclude that landings and income from these species have increased due to the reserve, but we can at least conclude that landings have not decreased. While further information on market behavior of each fishery is needed, these results provide insights into the state of the reserve and its associated fisheries.

As for the governance information, it is difficult to establish causal links between the state of the reserve and the governance structures present in the community. However, providing a single platform (i.e. scorecard) or document (i.e. technical report) where biological, socioeconomic, and governance information is comprehensively included can aid in management. By using MAREA, this information will be reported across reserves in a standardized way, and can help managers identify overarching patterns across sites.

By making results straightforward to interpret, MAREA may also assist in communication with a broader stakeholder community. While stakeholder involvement in the design and implementation phases of marine reserves is important, that may not be sufficient for ensuring long-term buy-in or success. The scorecard is easily understandable by experts and non-experts, and can be used as an effective tool for communicating the results of annual evaluations. Additionally, the technical report can serve as a tool for managers and scientists to rapidly produce and communicate information at a more technical level.

We recognize that the seven objectives and 29 indicators used by MAREA might not fully describe a reserve in countries other than Mexico. In order to ensure the applicability of the tool to reserves in other countries, further testing in other regions should take place. However, the proposed objectives and indicators provide a starting point to perform the evaluation, to which managers and users can add other indicators (e.g. larval dispersal or connectivity) that are relevant to their reserve. Furthermore, MAREA’s value is that it provides a free, simple, and replicable way to perform rigorous impact analysis. The tool can easily be used by fishers, CSO members, and managers in government agencies, providing transparency of the analysis and results. In addition, it can empower and enable local managers and fishers to respond to local change and adapt by allowing direct and easy access to the information.

An evident limitation of MAREA is its dependence on data obtained through a BACI design, and the amount of samples needed to estimate coefficients in Eq (1). It is not uncommon for control sites or baselines to be absent. Properly designing marine reserves by identifying control sites and establishing a baseline before the implementation of the reserve is enough to overcome this issue; reserves for which there is no control site and baseline cannot be evaluated with MAREA. Typical underwater surveys require that at least 12—16 transects are performed for each site (i.e. reserve and control) each year. This provides at least 48 samples (12 samples per site, per year), enough to avoid overfitting Eq (1). However, these problems can be easily avoided during the design and implementation phases by anticipating what data will be needed in the eventual evaluation.

To the best of our knowledge, MAREA is the first tool designed to evaluate marine reserves. Previous work [23, 25] addressed MPA evaluation and provided the foundation for our contribution. However, these did not intended to create user-friendly tools to aid in the evaluation. Conservation management tools that automatize complex calculations can have an important impact in management [64]. The use of open data science enables the creation of open-access tools that can address technical gaps and inprove management [28].

The effectiveness of marine reserves continues to be a matter of debate [11, 44, 65]. With current targets set to increase ocean protection, it is important that we understand the effects of our interventions [38] so we can better inform management [47]. It is therefore important that academics, managers, fishers, and CSOs have access to open access tools like MAREA. This is particularly relevant for Mexico and other Latin American countries, where management agencies are often understaffed and underfunded [66], or where materials are often not available in their language. In this context, MAREA provides a simple and replicable way to align management objectives with performance indicators. The proposed methodologies, especially the way in which biological indicators are evaluated, provide valuable information for managers. We acknowledge there is room for improvement in the way in which socioeconomic and governance data are analyzed. Despite this, providing a unifying platform where all indicators can be analyzed and comprehensively presented represents a valuable step towards effective evidence-–based management [47].

The first release of MAREA is now available, and it will continue to be developed and maintained to keep up to date with the literature. This process will incorporate new features, and enhance current ones, aiming to improve user experience and expand the scope of the analysis. Other modifications may also include addition of more objectives and indicators to ensure applicability in other regions, full translation into other languages to avoid any ambiguities introduced via the automatic translation, or reporting effects over time in percentages to aid interpretation. Yet, we believe that this first release represents a major step towards effective, replicable evaluation and management of marine reserves.

Supporting information

S1 Appendix. Survey to collect governance information from fishing communities.

English version.


S2 Appendix. Survey to collect governance information from fishing communities.

Spanish version.


S1 Table. Assigned values and reasoning of socioeconomic and governance indicators used to color-code the scorecard in MAREA.



We thank Olivier Deschenes and Andrew Plantinga, who provided valuable input to design the model that evaluates the biological indicators. Special thanks to the fishers from Isla Natividad, who gathered the data used in this study, and the fishers from El Rosario, who helped us validate our survey and framework, and to Arturo Hernández and Alfonso Romero who provided help with the logistics. Finally, we thank the editor and two anonymous reviewers for their suggestions, which significantly improved the quality of this paper.


  1. 1. Pauly D, Watson R, Alder J. Global trends in world fisheries: impacts on marine ecosystems and food security. Philosophical Transactions of the Royal Society B: Biological Sciences. 2005;360(1453):5–12.
  2. 2. Halpern BS, Walbridge S, Selkoe KA, Kappel CV, Micheli F, D’Agrosa C, et al. A global map of human impact on marine ecosystems. Science. 2008;319(5865):948–952. pmid:18276889
  3. 3. Lester S, Halpern B. Biological responses in marine no-take reserves versus partially protected areas. Mar Ecol Prog Ser. 2008;367:49–56.
  4. 4. Lester S, Halpern B, Grorud-Colvert K, Lubchenco J, Ruttenberg B, Gaines S, et al. Biological effects within no-take marine reserves: a global synthesis. Mar Ecol Prog Ser. 2009;384:33–46.
  5. 5. Sala E, Costello C, De Bourbon Parme J, Fiorese M, Heal G, Kelleher K, et al. Fish banks: An economic model to scale marine conservation. Marine Policy. 2016;73:154–161.
  6. 6. Hastings A, Gaines SD, Costello C. Marine reserves solve an important bycatch problem in fisheries. Proc Natl Acad Sci USA. 2017;114(34):8927–8934.
  7. 7. Aburto-Oropeza O, Erisman B, Galland GR, Mascareñas-Osorio I, Sala E, Ezcurra E. Large Recovery of Fish Biomass in a No-Take Marine Reserve. PLoS ONE. 2011;6(8):e23601. pmid:21858183
  8. 8. Micheli F, Saenz-Arroyo A, Greenley A, Vazquez L, Espinoza Montes JA, Rossetto M, et al. Evidence that marine reserves enhance resilience to climatic impacts. PLoS ONE. 2012;7(7):e40832. pmid:22855690
  9. 9. Roberts CM, OLeary BC, McCauley DJ, Cury PM, Duarte CM, Lubchenco J, et al. Marine reserves can mitigate and promote adaptation to climate change. Proc Natl Acad Sci USA. 2017;114(24):6167–6175. pmid:28584096
  10. 10. Munguía-Vega A, Sáenz-Arroyo A, Greenley AP, Espinoza-Montes JA, Palumbi SR, Rossetto M, et al. Marine reserves help preserve genetic diversity after impacts derived from climate variability: Lessons from the pink abalone in Baja California. Global Ecology and Conservation. 2015;4:264–276.
  11. 11. Edgar GJ, Stuart-Smith RD, Willis TJ, Kininmonth S, Baker SC, Banks S, et al. Global conservation outcomes depend on marine protected areas with five key features. Nature. 2014;506(7487):216–220. pmid:24499817
  12. 12. Giakoumi S, Scianna C, Plass-Johnson J, Micheli F, Grorud-Colvert K, Thiriet P, et al. Ecological effects of full and partial protection in the crowded Mediterranean Sea: a regional meta-analysis. Sci Rep. 2017;7(1):8940. pmid:28827603
  13. 13. Sala E, Giakoumi S. No-take marine reserves are the most effective protected areas in the ocean. ICES Journal of Marine Science. 2017;
  14. 14. Betti F, Bavestrello G, Bo M, Asnaghi V, Chiantore M, Bava S, et al. Over 10 years of variation in Mediterranean reef benthic communities. Marine Ecology. 2017;38(3):e12439.
  15. 15. Szuwalski CS, Burgess MG, Costello C, Gaines SD. High fishery catches through trophic cascades in China. Proc Natl Acad Sci USA. 2017;114(4):717–721. pmid:28028218
  16. 16. Chavez FP. From anchovies to sardines and back: multidecadal change in the pacific ocean. Science. 2003;299(5604):217–221. pmid:12522241
  17. 17. Davies TK, Mees CC, Milner-Gulland EJ. Use of a counterfactual approach to evaluate the effect of area closures on fishing location in a tropical tuna fishery. PLoS ONE. 2017;12(3):e0174758. pmid:28355269
  18. 18. Guidetti P, Baiata P, Ballesteros E, Di Franco A, Hereu B, Macpherson E, et al. Large-Scale Assessment of Mediterranean Marine Protected Areas Effects on Fish Assemblages. PLoS ONE. 2014;9(4):e91841. pmid:24740479
  19. 19. Friedlander AM, Golbuu Y, Ballesteros E, Caselle JE, Gouezo M, Olsudong D, et al. Size, age, and habitat determine effectiveness of Palau’s Marine Protected Areas. PLoS ONE. 2017;12(3):e0174787. pmid:28358910
  20. 20. Rodriguez AG, Fanning LM. Assessing Marine Protected Areas Effectiveness: A Case Study with the Tobago Cays Marine Park. OJMS. 2017;07(03):379–408.
  21. 21. Moland E, Olsen EM, Knutsen H, Garrigou P, Espeland SH, Kleiven AR, et al. Lobster and cod benefit from small-scale northern marine protected areas: inference from an empirical before-after control-impact study. Proceedings of the Royal Society B: Biological Sciences. 2013;280(1754):20122679–20122679. pmid:23303544
  22. 22. Soykan CU, Lewison RL. Using community-level metrics to monitor the effects of marine protected areas on biodiversity. Conserv Biol. 2015;29(3):775–783. pmid:25572325
  23. 23. Pomeroy RS, Watson LM, Parks JE, Cid GA. How is your MPA doing? A methodology for evaluating the management effectiveness of marine protected areas. Ocean Coast Manag. 2005;48(7-8):485–502.
  24. 24. Pomeroy RS, Parks JE, Watson LM. How is your MPA doing? A guidebook of natural and social indicators for evaluating marine protected areas management effectiveness. IUCN; 2004. Available from:
  25. 25. Mascia MB, Fox HE, Glew L, Ahmadia GN, Agrawal A, Barnes M, et al. A novel framework for analyzing conservation impacts: evaluation, theory, and marine protected areas. Ann N Y Acad Sci. 2017;1399(1):93–115. pmid:28719737
  26. 26. Ostrom E. A General Framework for Analyzing Sustainability of Social-Ecological Systems. Science. 2009;325(5939):419–422. pmid:19628857
  27. 27. Basurto X, Gelcich S, Ostrom E. The social–ecological system framework as a knowledge classificatory system for benthic small-scale fisheries. Global Environmental Change. 2013;23(6):1366–1380.
  28. 28. Lowndes JSS, Best BD, Scarborough C, Afflerbach JC, Frazier MR, OHara CC, et al. Our path to better science in less time using open data science tools. Nat ecol evol. 2017;1(6):0160.
  29. 29. Halpern BS, Longo C, Hardy D, McLeod KL, Samhouri JF, Katona SK, et al. An index to assess the health and benefits of the global ocean. Nature. 2012;488(7413):615–620. pmid:22895186
  30. 30. Halpern BS, Frazier M, Afflerbach J, OHara C, Katona S, Stewart Lowndes JS, et al. Drivers and implications of change in global ocean health over the past five years. PLoS ONE. 2017;12(7):e0178267. pmid:28678881
  31. 31. Selig ER, Frazier M, O’Leary JK, Jupiter SD, Halpern BS, Longo C, et al. Measuring indicators of ocean health for an island nation: The ocean health index for Fiji. Ecosystem Services. 2015;16:403–412.
  32. 32. Halpern BS, Longo C, Scarborough C, Hardy D, Best BD, Doney SC, et al. Assessing the Health of the U.S. West Coast with a Regional-Scale Application of the Ocean Health Index. PLoS ONE. 2014;9(6):e98995. pmid:24941007
  33. 33. Elfes CT, Longo C, Halpern BS, Hardy D, Scarborough C, Best BD, et al. A Regional-Scale Ocean Health Index for Brazil. PLoS ONE. 2014;9(4):e92589. pmid:24695103
  34. 34. Anderson JL, Anderson CM, Chu J, Meredith J, Asche F, Sylvia G, et al. The fishery performance indicators: A management tool for triple bottom line outcomes. PLoS ONE. 2015;10(5):e0122809. pmid:25946194
  35. 35. Dowling N, Wilson J, Rudd M, Babcock E, Caillaux M, Cope J, et al. FishPath: A Decision Support System for Assessing and Managing Data- and Capacity- Limited Fisheries. In: Quinn T II, Armstrong J, Baker M, Heifetz J, Witherell D, editors. Assessing and Managing Data-Limited Fish Stocks. Alaska Sea Grant, University of Alaska Fairbansk; 2016. Available from:
  36. 36. Oyanedel R, Macy Humberstone J, Shattenkirk K, Rodriguez Van-Dyck S, Joye Moyer K, Poon S, et al. A decision support tool for designing TURF-reserves. BMS. 2017;93(1):155–172.
  37. 37. Vilela T, Reid J. Improving hydropower choices via an online and open access tool. PLoS ONE. 2017;12(6):e0179393. pmid:28650968
  38. 38. Burgess MG, Clemence M, McDermott GR, Costello C, Gaines SD. Five rules for pragmatic blue growth. Marine Policy. 2018;87:331–339.
  39. 39. NOM-049-SAG/PESC. NORMA Oficial Mexicana NOM-049-SAG/PESC-2014, Que determina el procedimiento para establecer zonas de refugio para los recursos pesqueros en aguas de jurisdicción federal de los Estados Unidos Mexicanos. DOF. 2014;.
  40. 40. LGEEPA. Ley General del Equilibrio Ecológico y la Protección al Ambiente. DOF. 2017;.
  41. 41. Lester S, McDonald G, Clemence M, Dougherty D, Szuwalski C. Impacts of TURFs and marine reserves on fisheries and conservation goals: theory, empirical evidence, and modeling. BMS. 2017;93(1):173–198.
  42. 42. Chirico AAD, McClanahan TR, Eklöf JS. Community- and government-managed marine protected areas increase fish size, biomass and potential value. PLoS ONE. 2017;12(8):e0182342. pmid:28806740
  43. 43. Rossetto M, Micheli F, Saenz-Arroyo A, Montes JAE, De Leo GA. No-take marine reserves can enhance population persistence and support the fishery of abalone. Can J Fish Aquat Sci. 2015;72(10):1503–1517.
  44. 44. Woodcock P, O’Leary BC, Kaiser MJ, Pullin AS. Your evidence or mine? Systematic evaluation of reviews of marine protected area effectiveness. Fish Fish. 2017;18(4):668–681.
  45. 45. Carter AB, Davies CR, Emslie MJ, Mapstone BD, Russ GR, Tobin AJ, et al. Reproductive benefits of no-take marine reserves vary with region for an exploited coral reef fish. Sci Rep. 2017;7(1):9693. pmid:28852089
  46. 46. Villaseñor-Derbez JC, Faro C, Wright M, Martínez J. A guide to evaluate the effectiveness of no-take marine reserves in Mexico. TURFeffect; 2017. Available from:
  47. 47. Ferraro PJ, Pattanayak SK. Money for nothing? A call for empirical evaluation of biodiversity conservation investments. PLoS Biol. 2006;4(4):e105. pmid:16602825
  48. 48. OECD. Inflation CPI; 2017. Available from:
  49. 49. Zeileis A. Econometric Computing with HC and HAC Covariance Matrix Estimators. J Stat Softw. 2004;11(10).
  50. 50. Espinosa-Romero MJ, Rodriguez LF, Weaver AH, Villanueva-Aznar C, Torre J. The changing role of NGOs in Mexican small-scale fisheries: From environmental conservation to multi-scale governance. Marine Policy. 2014;50:290–299.
  51. 51. R Core Team. R: A Language and Environment for Statistical Computing; 2017. Available from:
  52. 52. Chang W, Cheng J, Allaire J, Xie Y, McPherson J. shiny: Web Application Framework for R; 2017. Available from:
  53. 53. Chang W, Borges Ribeiro B. shinydashboard: Create Dashboards with’Shiny’; 2017. Available from:
  54. 54. Allaire J, Cheng J, Xie Y, McPherson J, Chang W, Allen J, et al. rmarkdown: Dynamic Documents for R; 2017. Available from:
  55. 55. Xie Y. knitr: A General-Purpose Package for Dynamic Report Generation in R; 2017. Available from:
  56. 56. McCay B. Territorial use rights in fisheries of the northern Pacific coast of Mexico. BMS. 2017;93(1):69–81.
  57. 57. McCay BJ, Micheli F, Ponce-Díaz G, Murray G, Shester G, Ramirez-Sanchez S, et al. Cooperatives, concessions, and co-management on the Pacific coast of Mexico. Marine Policy. 2014;44:49–59.
  58. 58. de Estadistica Geografia e Informatica de Mexico IN. Marco Geoestadistico Nacional; 2017. Available from:
  59. 59. Tennekes M. tmap: Thematic Maps; 2017. Available from:
  60. 60. Afflerbach JC, Lester SE, Dougherty DT, Poon SE. A global survey of TURF-reserves, Territorial Use Rights for Fisheries coupled with marine reserves. Global Ecology and Conservation. 2014;2:97–106.
  61. 61. Suman CS, Saenz-Arroyo A, Dawson C, Luna MC. Manual de Instruccion de Reef Check California: Guia de instruccion para el monitoreo del bosque de sargazo en la Peninsula de Baja California. Pacific Palisades, CA, USA: Reef Check Foundation; 2010.
  62. 62. Gutiérrez NL, Hilborn R, Defeo O. Leadership, social capital and incentives promote successful fisheries. Nature. 2011;470(7334):386–389. pmid:21209616
  63. 63. Finkbeiner EM, Basurto X. Re-defining co-management to facilitate small-scale fisheries reform: An illustration from northwest Mexico. Marine Policy. 2015;51:433–441.
  64. 64. Ball IR, Possingham HP, Watts ME. Marxan and relatives Software for spatial conservation prioritization. In: Moilanen A, Wilson KA, Possingham HP, editors. Spatial conservation prioritization. Quantitative methods and computational tools. United Kingdom: Oxford University Press; 2009. p. 185–195. Available from:
  65. 65. Padleton LH, Aghmadia GN, Browman HI, Thurstand RH, Kaplan DM, Bartolino V. Debating the effectiveness of marine protected areas. ICES Journal of Marine Science. 2017;
  66. 66. Lundquist CJ, Granek EF. Strategies for successful marine conservation: integrating socioeconomic, political, and scientific factors. Conserv Biol. 2005;19(6):1771–1778.