Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Effectiveness of Strict vs. Multiple Use Protected Areas in Reducing Tropical Forest Fires: A Global Analysis Using Matching Methods


Protected areas (PAs) cover a quarter of the tropical forest estate. Yet there is debate over the effectiveness of PAs in reducing deforestation, especially when local people have rights to use the forest. A key analytic problem is the likely placement of PAs on marginal lands with low pressure for deforestation, biasing comparisons between protected and unprotected areas. Using matching techniques to control for this bias, this paper analyzes the global tropical forest biome using forest fires as a high resolution proxy for deforestation; disaggregates impacts by remoteness, a proxy for deforestation pressure; and compares strictly protected vs. multiple use PAs vs indigenous areas. Fire activity was overlaid on a 1 km map of tropical forest extent in 2000; land use change was inferred for any point experiencing one or more fires. Sampled points in pre-2000 PAs were matched with randomly selected never-protected points in the same country. Matching criteria included distance to road network, distance to major cities, elevation and slope, and rainfall. In Latin America and Asia, strict PAs substantially reduced fire incidence, but multi-use PAs were even more effective. In Latin America, where there is data on indigenous areas, these areas reduce forest fire incidence by 16 percentage points, over two and a half times as much as naïve (unmatched) comparison with unprotected areas would suggest. In Africa, more recently established strict PAs appear to be effective, but multi-use tropical forest protected areas yield few sample points, and their impacts are not robustly estimated. These results suggest that forest protection can contribute both to biodiversity conservation and CO2 mitigation goals, with particular relevance to the REDD agenda. Encouragingly, indigenous areas and multi-use protected areas can help to accomplish these goals, suggesting some compatibility between global environmental goals and support for local livelihoods.


Tropical deforestation accounts for between one fifth and one quarter of the total human contribution to greenhouse gases [1], [2], and 80% of emissions from the least developed countries. (Data for 2005, including land-used change and forestry, from CAIT 8.0.) Reduction of deforestation therefore contributes to climate change mitigation and may also provide development benefits [3], [4], [5]. The REDD (Reduced Emissions from Deforestation and Degradation) agenda seeks to integrate deforestation reduction into the global climate regime under the United Nations Framework Convention on Climate Change, rewarding countries that reduce forest emissions [3], [6].

Although the REDD agenda is new, the forest protection agenda is not. Conservation and sustainable management of forests have been motivated by biodiversity and livelihood concerns for decades. Where deforestation is a threat to biodiversity, successful conservation or sustainable management efforts will have a side benefit of reducing forest carbon emissions. This is especially salient in the humid tropical forests, where deforestation rates and carbon densities are both high. So an evaluation of the effectiveness of past conservation efforts can inform the design of interventions to promote REDD.

Among conservation interventions in tropical forests, the establishment of protected areas has been the most prominent and best funded [5]. The Global Environment Facility says that its investments in protected areas include $1.6 billion of its own resources and $4.2 billion in cofinancing; much of this has been implemented through the World Bank. Protected areas have expanded rapidly in recent years [7] and now cover around 27.1 percent of the tropical forest estate. (Authors' calculation. Boundary and area data are not available for a small percentage of protected areas, so this may be a conservative estimate.) In many ways they provide a model for broader classes of intervention, since most efforts to reduce deforestation will involve some kinds of restrictions on land use practices [5].

Yet there is considerable uncertainty and controversy over the impacts and effectiveness of protected areas. Views on their environmental effectiveness have see-sawed. In the 1990s, protected areas were often characterized as largely ineffective ‘paper parks.’ [8], [9] Over the last decade, evidence (reviewed below) has suggested, to the contrary, that protected areas are effective – only to be challenged, more recently, on methodological grounds. Meanwhile, from a social viewpoint, strict protected areas (which allow only conservation-related use) are sometimes viewed as effective in protecting biodiversity at the expense of excluding local inhabitants from access to forest resources [10]. Multiple use protected areas, which allow some sustainable use by local inhabitants, might potentially achieve both social and conservation goals – or fail at both [11],[12] – but quantitative studies on livelihood-conservation interactions are sparse [13].

Over the past decade, the growing availability of remotely sensed data on forest cover has facilitated quantitative studies of the impact of protected areas on tropical deforestation. Review papers include [12], [14], [15]; country-specific examples include Belize [16], Brazil [17], [18], [19], [20], Costa Rica [21], [22], [23], Honduras [24], Indonesia [25], [26], [27], [28], Madagascar [29], Peru [30] and Thailand [31]; global studies include [8], [32], [33]. These studies overwhelmingly report that protected areas are associated with lower deforestation rates.

The challenge for impact analysis is to construct a counterfactual: how much deforestation would have taken place if the forest in question had not been put under protection [34]; (see [15] for a detailed discussion and critical review of many studies). This requires controlling for social, economic and environmental factors that affect deforestation rate, and affect where protected areas are located. It is well-established that deforestation rates are lower in lands that are unattractive to agriculture: those that are remote from markets, have poor soils, high slopes, or heavy rainfall. (See [5] for a review.) But it is precisely these kinds of lands which governments might find it easiest to gazette for protection, where population density is low and powerful rural interests less likely to object [35]. And in fact, protected areas are disproportionately sited on lands characterized by higher slopes, higher elevations, and greater remoteness [36]. A naïve comparison of deforestation rates between these “low-pressure” areas and unprotected lands in general would give an inflated estimate of the effectiveness of the protected areas. On the other hand, [20] argue that some Brazilian forests have been designated as indigenous areas to protect them against very high deforestation pressures. Here, a comparison between protected and unprotected forests would yield an underestimate of the impact of protection.

Studies have approach this problem with different degrees of rigor. Some (e.g. [25], [30]) lack explicit controls, or assess deforestation in a protected area for which there is no comparable unprotected area [24]. Several use multivariate methods that explain the presence of deforestation, at the pixel level, as a function not just of protected area status, but also of slope, remoteness, and other determinants of both deforestation and of protected area placement [16],[26], [29]. A more sophisticated variant takes protected area status as endogenous, jointly modeling protection and deforestation via a bivariate probit model [37]. However, this approach requires nominating a variable which affects protection but not deforestation – a requirement difficult to fulfill.

More recently, some studies have used matching methods that are thought to be less sensitive to specification error than the multivariate econometric models. Matching methods seek to pair protected forest plots with unprotected but otherwise similar “control” plots. “Similarity” is defined on the basis of the control variables (such as slope and remoteness). As in the case of econometrics, the credibility of the models relies on the assumption that all significant confounding variables have been included.

For instance, [23] used matching methods to assess the deforestation-reducing impact of Costa Rica's system of protected areas. They found that protected areas on average did modestly reduce deforestation, but by substantially less than a naïve comparison of mean deforestation rates in protected versus unprotected areas [38]. qualifies this result, showing that Costa Rican parks had a greater protective effect in areas facing greater pressure, such as those close to the capital [28]. used propensity-score matching to assess the impact of protected areas on deforestation in Sumatra over 1990–2000, using high-resolution Landsat imagery. Compared to equivalent forests within protected areas, the study found deforestation rates to be 7.4 percentage points higher in buffer areas, and 24 percentage points higher in the landscape beyond.

At the global level, comprehensive evaluation has been hampered by inadequate data. There is no globally consistent, high-spatial resolution time series data for the entire tropical forest biome. Thus [8] relied on questionnaires aimed at protected area managers and [32] employed very coarse resolution (8 km square) remote sensing imagery to detect change; both covered only a subset of tropical protected areas. The study most comparable to ours is [33], which uses matching methods to assess protected area impacts across all biogeographical domains, not just the tropical forests. It uses two strategies to confront the lack of global deforestation data, each with some shortcomings. First, it uses natural land cover at a single point in time as a proxy for deforestation (i.e., change in land cover). Since it is possible that habitat clearance may have preceded (and motivated) the establishment of the protected area, a sensitivity test restricts the sample to pre-1980 protected areas. Second, it estimates land cover change over the period 2000–2005 by comparing two land cover datasets constructed with different methodologies. Since there is significant uncertainty in land cover classification at the 1 km resolution level, the difference between these two datasets may have a high noise/signal ratio, as the study acknowledges. It finds, overall, that the introduction of controls greatly diminishes but does not entirely nullify the estimate of protected area effectiveness. The global average difference in natural land cover in 2000 was about 2.5 percentage points; the difference in measured change between 2000 and 2005 was about 0.5 percentage points. The study finds protective impacts to be greater in flatter, less remote areas, and in the strictest forms of protected area (IUCN categories I and II).

The current study differs from [33] in several ways. It focuses on the tropical forest biome, a more homogeneous domain (from a biological and socioeconomic viewpoint) than the global set of protected areas. It uses what we will argue is a better proxy for land use change – namely, the occurrence of forest fires. And it partitions the set of protected areas along more policy-relevant lines, distinguishing strictly protected, multiple use, and indigenous areas. The study disaggregates results by continent and by remoteness from cities (a proxy for deforestation pressure). It finds in general that strict protected areas are effective, but less than a naïve assessment would indicate. In contrast, multiple use protected areas are in general more effective in reducing deforestation than strict protected areas, and are more effective than a naïve assessment would suggest.


Study area

The study is limited to developing countries (recipient countries of World Bank loans) and the extent of the tropical forest biome. These countries account for the bulk of deforestation and are potentially eligible for REDD participation. The biome—derived from the World Wildlife Fund's Terrestrial Ecoregions of the World [39]—contains the maximum spatial extent of the world's tropical and subtropical moist broadleaf forests.

Figure 1 shows the spatial intersection of these countries and the biome. The area in green is the maximum extent of the study area, covering 19.73 million km2. The biome is split across three continents; each will be analyzed separately. Papua New Guinea and Micronesia are considered part of Asia for this analysis.

Figure 1. The tropical forest biome with standing forests in 2000.

Gray: Tropical forest biome. Green: Standing forest, 2000.

Within this area, the extent of the remaining tropical forest in 2000 was extracted from two land cover data sources: Global Land Cover for the year 2000 [40] derived from ∼1 km resolution SPOT data and Percentage Forest Canopy Cover for 2000 [41] derived from ∼500 m MODIS data.

All 11 land cover classes from GLC2000 that contain forest or forest mosaics were extracted, along with all ∼1-km pixels where the average percent forest cover was greater than 25 percent [42]. This is a higher threshold than the 10 percent used in the FAO Forest Resource Assessment [43] and in a recent assessment of global forest protection [44]. One justification for using the 10 percent threshold in those global analyses was to capture woodland areas in Africa; however, these are not part of the tropical forest biome. Twenty-five percent was chosen to minimize the risk of including tropical woodlands/savannas and other land that was already largely cleared of forest, that was predominantly used for agriculture, and that could exhibit high fire activity that was not necessarily related to deforestation events.

Although both sources are well-documented research products, there are disagreements between the two datasets. To provide a conservative estimate of forest fire incidence, we use the intersection of the two forest covers within the boundaries of the biome covering 13.15 million km2 of tropical forest area in 2000. For reference, a tropical forest extent based on the MODIS data alone or on GLC2000 alone would amount to 15.13 million km2 or 14.51 million km2, respectively. Agreement between the two across the biome is 83.1 percent.

Estimated deforestation fire activity: Outcome variable

The outcome variable was a binary measure: the presence or absence of at least one fire event on a given forested pixel (see Supplementary Information S1). Like other studies [19], we argue that forest fires are a reasonable proxy for tropical deforestation. In the tropics, non-anthropogenic fire is rare [45]. In Indonesia [46], found that fires were associated with land clearance in 8 of 9 study sites. In Amazônia, fires are associated with initial land clearance and with subsequent land management on the cleared plots [47]. (This motivates our choice of outcome measures not as the count of fires on a forest pixel, but the binary indication of whether one or more fires took place on the pixel during the observation period.) A comparison by Morton et al. [48] of screened ‘high-confidence’ fire detection with a deforestation measure based on high resolution Landsat imagery found that 87% of crop-related deforestation and 73% of pasture-related deforestation is associated with at least one such fire. Thus there is a small to moderate possibility of false negatives when using fires as a proxy for overt deforestation. The chance of false positives is minimized here by using Morton et al.'s high-confidence filter, which considers only fires occurring at night and daytime fires with >330 K brightness temperature in the 4 µm channel. While our indicator is imperfect, deforestation detection via visual interpretation of low-resolution imagery is also fallible, and even high-resolution Landsat imagery presents problems of interpretation, and of censoring due to cloud cover. The fire data and most imagery-based methods will fail to detect ‘cryptic’ forest degradation such as low-intensity logging.

Fire activity was estimated from spatially referenced remote sensing data on forest fires from the MODIS Active Fires dataset [49]. MODIS Active Fire data are provided on two satellite platforms, Terra from October 2000 and Aqua from July 2002, both to present day. Thus, there is partial coverage from October 2000 (two passes per day) and complete coverage from July 2002 (four passes per day), including both day and night passes.

High-confidence fires were extracted from more than 1 million MODIS fires scenes between 2000 and 2008 ( Some 1.21 million 1-km pixels recorded at least one fire between October 2000 and January 2009 in the tropical forest biome and 0.70 million of these occurred in forested areas (Table 1). Of the 13.15 million 1-km tropical forest pixels, 5.31 percent had at least one fire event in that time frame.

Table 1. 1-km forest and fire pixel statistics in the tropical forest biome (2000–08).

The outcome variable is a binary measure of forest fire activity per square km: was there at least one fire event in that pixel during 2000–08? This time period is reflected in the choice of covariables and the definition of the control/treatment groups below. The lack of coverage until October 2000 and then partial coverage until July 2002 implies that the binary measure here is slightly conservative as an estimate of fire-affected area.

Another dataset was considered as a proxy for tropical deforestation events: the recently released MODIS Collection 5 Burned Area Product, which includes global, monthly 500-meter (m) resolution maps of burn dates. A direct comparison between the active fire and burned area data for July 2001–June 2002 found that “many forest fires are detected by the active fire product but not by the burned area product” [50]. This higher detection rate, albeit including both medium- and high-confidence fires, and the fact that the burned area data are still provisional led to a preference for the active fire data over the burned area data as a proxy for tropical deforestation events.

The presence of one or more fires in a 1-km pixel cannot be directly translated into an estimate of deforested area. A fire event may represent anything from a small clearing of a single hectare to complete deforestation of the 1-km pixel. However, it can be assessed whether this fire presence/absence data can be used as a plausible proxy for deforestation activity in the tropical forest biome. We compared the binary measure of forest fire activity to deforestation as measured on a set of 183 Landsat scenes, used by Hansen et al. [42] for global imputation of deforestation. (Hansen et al use this high resolution data to calibrate global imputations based on lower resolution MODIS data. However, they caution against using the imputed data at the pixel level.) We plotted the area of fire activity for 2000–05 as a proportion of forest area against percent forest cover loss for 2000–05 per 18.5-km pixel (Supplementary Information S2). The analysis was repeated for 5 percent (top of figure) and 1 percent (bottom of figure) bins of forest cover loss.

There is a strong trend of increasing fire activity with increased loss of forest cover across the biome from 0 to 30 percent forest cover loss. The trend continues for higher forest cover loss percentages, but there are very few 18.5-km pixels (<0.2% of the tropical forest biome area) in these areas. Latin America and the Caribbean and Asia show the same clear trend as the whole biome, but the case is less clear for Africa. It should be noted that the remote sensing estimate of African deforestation differed drastically from the Forest Resources Assessment 2005 by the FAO [42], [43], so the deviation between the fire measures and the remote sensing measures may not be solely due to misclassification of the fire data.

From this it is reasonably sure that the chosen subset of active fires is a plausible proxy for deforestation events, especially in Latin America and Asia. The case is less convincing for Africa but is still plausible.

Protected areas and IUCN management classes

The World Database on Protected Areas (WDPA) [51] is the source for protected area information. The WDPA is compiled by the United Nations Environment Programme and the IUCN, drawing on member organizations in 140 countries. While its accuracy depends on the reporting process [52], it is recognized as the most comprehensive and authoritative database available on protected areas and is commonly used in global studies of conservation (e.g. [33], [53]) It applies a rigorous, consistent, and detailed set of criteria to the identification and classification of protected areas [54]. Protected areas are defined as: “a clearly defined geographical space, recognized, dedicated and managed, through legal or other effective means, to achieve the long-term conservation of nature with associated ecosystem services and cultural values.” Protected area information, including boundaries (and PA center coordinates and area for PAs with unknown boundaries), designation date, IUCN protected area management classification, and status were extracted from the WDPA database for all protected areas that were inside or that intersected the tropical forest biome.

This list of protected areas includes all nationally (IUCN protected area management classes I through VI as well as unknown) and internationally (UNESCO MAB reserves, Ramsar sites, and World Heritage sites) recognized PAs and amounts to 4.13 million km2 of protected area within the biome, of which 3.62 million km2 is forested. (See Supplementary Information S3 for the IUCN definitions of management classes.)

Two treatment groups were considered, based on protected areas with boundary information. The first group consists of all protected areas that were designated pre-2000. The second group is restricted to protected areas that were designated between 1990 and 2000. Use of the restricted group allows us to examine the impact of more recently created protected areas and provides a check against the possibility of endogeneity in the matching variables.

Based on the IUCN categorization, PAs are classified as follows:

  • Strict protection—IUCN classes I though IV
  • Nonstrict or multi-use protection—IUCN classes V and VI
  • Unknown protection—Nationally recognized but with no IUCN class
  • Indigenous—A subset of the unknown class, but under indigenous stewardship.

Strict protection means areas that are designed specifically for nature protection. Nonstrict protection means that the areas have a multiple use management strategy. Category VI, for instance, comprises areas whose primary objective is “to protect natural ecosystems and use natural resources sustainably, when conservation and sustainable use can be mutually beneficial.” The indigenous group of protected areas occurs in Latin America, predominantly in Brazil, with a few areas in Panama and Colombia. Figure 2 shows the IUCN classified protected areas that were designated before 2000; the dominance of the protected tropical forest area in Latin America and the Caribbean is clear. There were 2,974 IUCN classified (IUCN classes I through VI, plus unknown) protected areas designated before 2000 in the tropical forest biome that contained at least 1 km2 of tropical forest.

Figure 2. IUCN-designated protected areas established by 2000.

Protected area category. Strict (IUCN I–IV) [green]. Multiuse (IUCN V, VI) [yellow]. Indigenous [pink].

The control groups are based on areas that have never been protected, up through 2008. We recognize that some forest areas that do not meet IUCN protected area criteria may benefit from other forms of legal protection – for instance, industrial forest concessions – so our comparisons may understate the effect of protection, broadly construed. Where boundary data was missing (for 22% of protected areas accounting for just 6% of total protected area extent), protected areas were represented by circles, with area equal to that of the protected area, centered around the point reported as the protected area location.

Summary statistics for tropical forest area and protected tropical forest area as of 2000 are shown below in Table 2. The number of observed tropical forest fire pixels and the tropical forest area for each region and protection group (pre-2000 areas only) are shown in Table 3.

Table 2. Total tropical forest protected (km2 and %) by protection class and region.

Table 3. Forest and fire area (km2) and fire rates per region/protection group.

Control variables

Variables describing terrain, climate, and remoteness were used to compare points in protected areas with ‘similar’ nonprotected points.

Accessibility to markets is a strong determinant of deforestation pressure [16]. A measure showing travel time to major cities in 2000 [55], [56] was used. This, the first such global measure, accounts for differential travel speeds on roads of different quality, railways, navigable rivers; and for off-road terrain, land cover, slope and elevation. Major cities are defined as having a population of 50,000 or more in 2000.

Distance to road network is a complementary measure of access to forest resources. A distance measure was created based on a vector road network extracted from the fifth edition of the Vector Smart Map Level 0 (VMap0) dataset. The primary source for the database is the 1∶1 million scale Operational Navigation Chart series. The reference period is 1979–99 [57] (The start date is debatable; the third edition of VMAP0, published 1997, also has a 20-year reference period—1974–94! The fifth edition was published in 2000, but given the minor changes after the first edition in 1992, it is unlikely to have much post-1990 data.)

Distance to major cities is a third proximity measure. A straight-line measure was created based on a point dataset of city centroids [58], using the same set of in the accessibility layer.

Terrain is a factor for land use suitability. Mild slopes and lower elevations are likely to be more accessible, more productive, more valuable, and thus more attractive for conversion to agriculture. As well as having a direct relation to suitability, slope and elevation are proxies for physical soil properties, and elevation is a proxy for temperature.

Elevation and slope were derived from the CSI-CGIAR version [59] of the 90-m resolution SRTM digital elevation model from NASA [60]. The CSI-CGIAR version of the data has filled in the data void areas with auxiliary digital elevation model data and topographically correct interpolation algorithms. The mean and variance of both slope and elevation were extracted at 1-km resolution.

Rainfall is another factor for land use suitability. Areas of extremely high rainfall are unlikely to be converted to agriculture, and the associated cloud cover and humidity preclude the use of fire activity as a reliable measure of deforestation.

Rainfall estimates were extracted from data provided by the Tropical Rainfall Monitoring Mission, specifically from the 3B42-TRMM-Adjusted Merged-Infrared Precipitation product [61]. This dataset provides monthly estimates of rainfall rates at a ¼-degree resolution. These rates were converted to millimeters (mm) per month, then aggregated into annual rainfall estimates and finally into an estimate of the average annual rainfall in mm for 2000–08.

Detailed country boundaries were extracted from the Global Administrative Areas database [62]. This information is used for exact matching to ensure that each control/treatment pair belongs to the same country.

Ecoregions, as defined by WWF, divide the world into 825 ecologically homogeneous areas [63]. Matching within ecoregions provides a more restrictive set of ‘equivalent’ forests than that enforced by matching on country and rainfall alone. For instance, Brazil is partitioned into 33 and Indonesia into 30 ecoregions. Since matching within ecoregions reduces the number of potential matches, we performed the analysis with and without this condition.

Summary statistics for all the above variables in the tropical forest and protected tropical forest areas are shown in Table 4. In general, protected tropical forest areas are more remote, have lower fire incidence rates, and have higher elevation/slope than the tropical forests as a whole.

Table 4. Summary statistics for variables in tropical forest areas.

Data and sampling

All spatial data were projected to equal area sinusoidal projection, with a WGS84 datum and spheroid. Unless otherwise stated, raster resolution is 1 km. The relevant data from each data layer were extracted at 1-km spacing and stored in a PostgreSQL database (version 8.3), amounting to some 19 million records, one record per 1-km pixel. The matching analysis was split into three geographic regions: Latin America and the Caribbean, Africa, and Asia. A list of points that would be used to form the control and treatment groups was extracted from the database for each region. The list of points for the treatment group was based on a 10 percent random sample of points. The treatment points had to meet the following criteria:

  • Were designated as protected pre-2000 based on protected area boundary information from the WDPA
  • Classified as forest cover in 2000, based on the 11 land cover classes in GLC2000 that are forest or forest mosaic
  • Met the 25 percent forest cover threshold from MODIS forest cover for 2000
  • Fell into the relevant protection group (strict, multi-use, unknown, indigenous) for the cohort.

The two forest criteria reflect the conservative estimate of tropical forest area in 2000.

The corresponding control group was based on another random sample that was five times as large. The control points had to meet the following criteria:

  • Had never been protected up to the end of 2008
  • Classified as forest cover in 2000, based on the 11 land cover classes in GLC2000 that are forest or forest mosaic
  • Met the 25 percent forest cover threshold from the MODIS forest cover for 2000.

The never protected area takes into account any form of recognized protection from the WDPA through the end of 2008 and including protected areas with information on their designation date. Those protected areas with boundary information are simply masked out. As noted, protected areas with a reported point location but no boundary information are treated as circles of the given area centered on their latitude/longitude coordinate, and those areas are also masked out.

Analytic methodology

The analysis is on 1-km resolution data. The outcome variable is a binary measure of fire presence/absence from 2000–08 as a proxy for deforestation events. The treatment variable is protected/nonprotected. Randomly selected protected points (treatment group) are matched with similar control points, and the difference in deforestation (forest fire) rates is statistically evaluated. (For reviews and applications of matching methods, see [64], [65], [66]).

We use a combination of exact matching and nearest neighbor matching. Treatment and control points are matched exactly on country and on accessibility (travel time to nearest city, segmented by 15 minute increments). An additional five variables described earlier – average elevation, average slope, average rainfall (2000–08), distance to roads, and distance to cities – were used to select comparison points via nearest neighbor matching. We also conducted all analyses with and without the use of ecoregion as an exact matching variable. We use the commonly-employed Mahalanobis distance metric, a scale invariant measure of the multidimensional distance between two points. The algorithm randomly orders the treatment cases and for each one in turn selects the control case with the smallest distance. We matched with and without a 0.5 SD caliper. Use of a caliper (i.e. maximum acceptable distance) increases the quality of the matches but results in some unmatched points. Matching was performed with replacement and bias adjustment. The matching package [66] (version 4.7–6) running in the open source statistical program R (version 2.8.1) on MS Windows XP SP3 was used.


Aggregate results

Table 5 shows the results of the matching analyses for all pre-2000 protected areas, alongside the crude (unmatched) estimates from Table 3. (In all cases the crude –comparing all protected pixels against all never protected pixels – and prematch rates – comparing an unmatched 10 percent sample of protected pixels against a similar proportion of never protected pixels – were very similar or identical, implying that the random sample was representative of the population.) Table 6 repeats, but uses the 1990–2000 protected areas as the treatment group. These tables and the subsequent discussion reflect results without the use of ecoregions as a matching variable. The use of ecoregions reduced the number of matching pairs but had little impact on the estimates. The ecoregion results are appended in Supplementary Information S4.

Table 5. Estimated impact on fire incidence (cumulative over 2000–08) comparing all pre-2000 protected areas against never protected areas.

Table 6. Estimated impact on fire incidence (cumulative over 2000–08) comparing 1990–2000 protected areas against never protected areas.

Looking at Table 5, in the Latin America and the Caribbean region, the matched results for strict protection suggest a much lower level of avoided fire activity than the crude estimates. Nonetheless, protected areas reduced the incidence of forest fires by 2.7–4.3 percentage points against a mean loss of 5.8 percent (Table 3) over 2000–08. Multi-use protected areas appear to be more effective than strictly protected areas by approximately 2 percentage points, and this also translates into a larger area. “Unknown” is less effective, but the area involved is quite small. Indigenous areas are shown to reduce forest fire incidence by 16.3–16.5 percentage points, over two and a half times as much as the crude estimates (5.9 percent) and twice as effective as any other group in the matched results, with a greater estimated avoided fire-affected area than strict, multi-use, and unknown combined. Strictly protected areas in Africa are only one-quarter as effective (about a 1 percentage point impact) as the uncorrected estimates would suggest. The estimated impacts for multi-use areas are not robust: a significant 3 percent for the estimate without calipers, but 0 percent (with wide error bands) for the estimate with calipers. In Asia, strictly protected areas perform better than in the crude estimates, but multi-use protection is twice as effective as strict protection.

Table 6 estimates suggest that, with the exception of indigenous areas, protected areas designated between 1990 and 2000 offer better protection than pre-2000 protected areas as a whole, with improvements ranging from 1 to 3.5 percentage points, disregarding results with few matched pairs. In Latin America and the Caribbean, multi-use protected areas appear to be as effective or more effective than strictly protected areas, but indigenous areas are almost twice as effective as any other form of protection. In Asia, strictly protected areas perform better than in the crude estimates, but multi-use is twice as effective. In Africa, these recently established protected areas appear much more effective than the larger set considered in Table 5, with a robustly estimated impact of about 4.5 percentage points. There are too few points to estimate an impact of multi-use areas.

Table 7 summarizes the results. The range of estimates represents a robustness test—use of two kinds of matching procedures, and a more or less broad scope of protected areas, each with advantages and disadvantages. The conclusion that protected areas are effective at reducing fire incidence on forest is seen to be robust.

Table 7. Summary of estimated protected area reductions of fire incidence (percentage points).

Note that indigenous areas in Latin America are estimated to reduce fire incidence by more than 16 percentage points – yet the mean fire rate in never protected areas is just 7.4%. This suggests that indigenous areas tend to be located in areas of much-higher-than-average deforestation pressure. And indeed [36] show that multi-use protected areas are less prone to be located in low-pressure areas than are strictly protected areas.

To assess the importance of location when estimating the effectiveness of protection, the fire rate in the matched treatment and control groups is disaggregated by travel time. This is done only for the pre-2000 treatment group, as the 1990–2000 group often has too few points to allow disaggregation.

The fire rate per travel time band was plotted and a loess curve was fitted through them using cross validation and Akaike's information criterion to determine the best fitting smoothing factor or bandwidth. Furthermore, the loess estimator (1,000 repetitions) was bootstrapped to determine 95 percent confidence intervals around the curve. (Given the small sample, these confidence intervals may be underestimated [67].) This was done for the fire rates from the matched control (never protected, red), and treatment data (protected pre-2000, green) and for the difference between the two (gray). This difference is essentially a disaggregated version of the estimates in Table 7 and provides an unbiased estimate of the avoided deforestation fires due to protection for different degrees of remoteness. The following figures show these confidence intervals around the loess curve as shaded polygons, as well as the points that they are fitted though. The results are reported for strict, multi-use, and indigenous areas for Latin America and the Caribbean (Figures 3 and 4), strict for Africa (there are insufficient pairs for multiuse to permit disaggregation) (Figure 5), and strict and multiuse for Asia (Figure 6), although the number of pairs for multiuse in Asia is just acceptable.

Figure 3. Unbiased estimated fire rates for tropical forests in Latin America and the Caribbean (with matching).

Top – Strict protection in Latin America and the Caribbean, with Mahalanobis matching (A) and Mahalanobis matching with calipers (B). Bottom – Multi-use protection in Africa, with Mahalanobis matching (C) and Mahalanobis matching with calipers (D).

Figure 4. Unbiased estimated fire rates (red - never protected, green - protected, and grey – difference) for indigenous protection in Latin America and the Caribbean, with Mahalanobis matching (A) and Mahalanobis matching with calipers (B).

Note the change in scale on y axes.

Figure 5. Unbiased estimated fire rates (red - never protected, green - protected, and grey – difference) for tropical forests in Africa with Mahalanobis matching (A) and Mahalanobis matching with calipers (B).

Figure 6. Unbiased estimated fire rates (red - never protected, green - protected, and grey – difference) for tropical forests in Asia (with matching).

Top – Strict protection, with Mahalanobis matching (A) and Mahalanobis matching with calipers (B). Bottom – Multi-use protection, with Mahalanobis matching (C) and Mahalanobis matching with calipers (D).

Some strong regularities emerge. First, in almost all cases, fire activity inside protected areas declines with increasing remoteness. Although the same is generally true for areas outside protected areas, in some cases (strict and multiuse in Latin America and the Caribbean and strict in Asia) the outside rate and hence effectiveness of protection increases with remoteness reaching a maximum at around 9–12 hours. Second, except for strict protection in Africa, protected areas generally have significantly lower fire rates than comparable nonprotected areas. However, this differential declines as remoteness increases. In other words, natural protection is as effective as formal protection in remote areas—at least for the moment. Third, in both Latin America and the Caribbean and Asia, nonremote multi-use areas are located in areas of higher deforestation pressure than strict areas. For instance, at one hour from cities in Latin America and the Caribbean, the controls for multi-use areas experience fire rates of about 16 percent whereas the controls for strict areas had fire rates of about 6 percent. Fourth, in Latin America and the Caribbean, fire rates are generally higher in multi-use than in strict protected areas, controlling for remoteness. Yet the impact of multi-use areas is greater than that of strict areas. At 1–12 hours from cities, for instance, multi-use protected areas reduce fire rates by about 6–12 percentage points, and strict protected areas reduce rates by only about 5 or 8 percentage points. Indigenous areas also have a very high absolute impact.

In Asia, the pattern is different. Controlling for distance, fire rates are higher in strict than in multi-use protected areas. Strict protected areas appear to be ineffective at deterring fires in nonremote areas. Their effectiveness increases with remoteness, peaking at about 12 hours distance from the city and declining thereafter. In contrast, multi-use protected areas are most effective in regions proximate to population centers.

In Africa, strict protected areas appear to have a modest impact. Estimates of the impact of multi-use areas are limited by a small sample and are not robust. The African estimates may be affected by outdated measurements of road proximity and remoteness from cities, since road conditions in parts of the Congo Basin have deteriorated since the reference period of the road maps used.


This paper uses forest fires as a proxy for deforestation and associated carbon release. Using global data for the tropical forest biome, it is apparent that protected areas have a substantially and statistically significantly lower incidence of forest fires than nonprotected areas, even after controlling for terrain, climate, and remoteness. The protective effect is greatest in nonremote areas (for Latin America and Africa) and areas of intermediate remoteness (Asia). Very remote areas have low fire rates even if unprotected—at least for the moment.

Importantly, it is clear that mixed-use protected areas—where some degree of productive use is allowed—are generally as effective or more effective than strict protected areas, especially in less remote areas with greater pressure for agricultural conversion and timber extraction. In Latin America, where indigenous areas can be identified, they are found to have extremely large impacts on reducing deforestation—much larger than a naïve, uncontrolled comparison would suggest. These results suggest that mixed-use and indigenous areas are disproportionately located in areas of higher deforestation pressure. This is noteworthy, given increasing attention to indigenous land rights.

From a policy viewpoint, these findings suggest that some kinds of land use restrictions—variations of protection—can be effective contributors to biodiversity conservation and climate change mitigation goals. The results suggest that indigenous areas and multi-use protected areas can help accomplish these goals, also suggesting some compatibility between environmental goals (carbon storage and biodiversity conservation) and support for local livelihoods. Zoning for sustainable use may be more politically feasible and socially acceptable than designation of strict protection in areas of higher population density and less remoteness. The results also reinforce findings from a field-survey based study of 84 protected areas in Asia and Africa, which found a positive association among biodiversity richness, forest support for livelihoods, and local people's participation in forest governance [13].

This analysis does not however attempt to measure “leakage” —the degree to which protection of one forest plot merely displaces conversion to another, unprotected plot. This is a more significant issue for carbon emissions than for biodiversity conservation, because the latter might be preferentially concerned with certain unique biodiversity locations whereas the former cares only about the density of carbon [68]. reviews theoretical and empirical studies of leakage and concludes that on both grounds leakage is far less than the 100 percent feared by critics. It points out that complementary policies (such as sponsoring crop intensification) could neutralize any leakage thought to arise from forest protection.

In addition, this analysis is unable to detect some kinds of forest degradation. Surreptitious removal of timber can result in biodiversity damage and lower carbon densities, but may not be detected through fire data.

Extension of this line of evaluation will be facilitated as better data become available. Improvements in remote sensing techniques and interpretation offer the prospect of more direct and precise measurement of deforestation and of forest carbon emissions. There is also a need to assemble, harmonize, and make public assessments of protected area management resources and practices in order to better understand the specific interventions that can contribute to reduced carbon emissions. Finally, there is a great need to complement land cover and land management measures with monitoring of human welfare and conditions in protected and unprotected forest areas.

It is important to stress that protected areas may be effective along other dimensions, even where there is little impact on current deforestation rates. This is especially true for protected areas established in remote regions with little current pressure for agricultural conversion. Such areas may already be effective in mitigating other threats, such as poaching of mammals and selective logging. Equally important, it is easier to reach consensus on the necessity and approach to protecting a forest before there are large economic pressures for conversion, often by people from outside the forest itself. A well-established protection regime may be better able to withstand pressures for unsustainable exploitation when the frontier arrives, as it eventually will in many currently remote places.

Supporting Information

Supplementary Information S1.

Fire activity 2000–2008 based on ‘high-confidence’ MODIS fires.


Supplementary Information S2.

Fire area as a proportion of tropical forest area vs. tropical forest cover loss (2000–2005) A: 5% bins B: 1% bins.


Supplementary Information S4.

Estimates using ecoregion as a matching variable.



The authors thank Lucas Joppa and Alex Pfaff for useful discussions on the spatial data and the matching methods employed here and for access to unpublished manuscripts. They thank Andrew Warner for helpful comments and Lucy Fish at UNEP-WCMC for advice on the World Database on Protected Areas. Two anonymous reviewers provided helpful comments and suggestions. The authors are grateful for the assistance of Dinara Akhmetova, Diana Salvemini, Cheikh M'Backe Falle, Heather Ditbrenner, and Janice Joshi.

The findings, interpretations, and conclusions expressed here do not necessarily reflect the views of the Executive Directors of the World Bank or the governments they represent, or of IRRI. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank or IEG concerning the legal status of any territory or the endorsement or acceptance of such boundaries.

Author Contributions

Conceived and designed the experiments: KMC AN. Performed the experiments: AN. Analyzed the data: KMC AN. Wrote the paper: KMC AN.


  1. 1. Gullison RE, Frumhoff PC, Canadell JG, Field CB, Nepstad DC, et al. (2007) ENVIRONMENT: Tropical Forests and Climate Policy. Science 316: 985–986.
  2. 2. Kindermann G, Obersteiner M, Sohngen B, Sathaye J, Andrasko K, et al. (2008) Global cost estimates of reducing carbon emissions through avoided deforestation. Proceedings of the National Academy of Sciences 105: 10302–10307.
  3. 3. Canadell JG, Raupach MR (2008) Managing Forests for Climate Change Mitigation. Science 320: 1456–1457.
  4. 4. Miles L, Kapos V (2008) Reducing Greenhouse Gas Emissions from Deforestation and Forest Degradation: Global Land-Use Implications. Science 320: 1454–1455.
  5. 5. Chomitz KM, Buys P, De Luca G, Thomas TS, Wertz-Kanounnikoff (2007) At Loggerheads? Agricultural Expansion, Poverty Reduction, and Environment in the Tropical Forests. Washington, DC: World Bank.
  6. 6. FAOUNDPUNEP (2008) UN Collaborative Programme on Reducing Emissions from Deforestation and Forest Degradation in Developing Countries (UN-REDD). 29 p.
  7. 7. Coad L, Campbell A, Granziera A, Burgess N, Fish L, et al. (2008) State of the world's protected areas 2007 - An annual review of global conservation progress. Cambridge, UK: UNEP-WCMC.
  8. 8. Bruner AG, Gullison RE, Rice RE, da Fonseca GAB (2001) Effectiveness of Parks in Protecting Tropical Biodiversity. Science 291: 125–128.
  9. 9. Bonham C, Sacayon E, Tzi E (2008) Protecting imperiled “paper parks”: potential lessons from the Sierra Chinajá, Guatemala. Biodiversity and Conservation 17: 1581–1593.
  10. 10. West P, Igoe J, Brockington D (2006) Parks and Peoples: The Social Impact of Protected Areas. Annual Review of Anthropology 35: 251–277.
  11. 11. Southworth J, Nagendra H, Munroe DK (2006) Introduction to the special issue: Are parks working? Exploring human-environment tradeoffs in protected area conservation. Applied Geography 26: 87–95.
  12. 12. Nagendra H (2008) Do Parks Work? Impact of Protected Areas on Land Cover Clearing. AMBIO: A Journal of the Human Environment 37: 330–337.
  13. 13. Persha L, Agrawal A, Chhatre A (2011) Social and Ecological Synergy: Local Rulemaking, Forest Livelihoods, and Biodiversity Conservation. Science 331: 1606–1608.
  14. 14. Naughton-Treves L, Holland MB, Brandon K (2005) The Role of Protected Areas in Conserving Biodiversity and Sustaining Local Livelihoods. Annual Review of Environment and Resources 30: 219–252.
  15. 15. Joppa L, Pfaff A (2010) Reassessing the forest impacts of protection. Annals of the New York Academy of Sciences 1185: 135–149.
  16. 16. Chomitz KM, Gray DA (1996) Roads, Land Use, and Deforestation: A Spatial Model Applied to Belize. The World Bank Economic Review 10: 487–512.
  17. 17. Chomitz KM, Thomas TS (2003) Determinants of Land Use in Amazônia: A Fine-Scale Spatial Analysis. American Journal of Agricultural Economics 85: 1016–1028.
  18. 18. Soares-Filho B, Moutinho P, Nepstad D, Anderson A, Rodrigues H, et al. (2010) Role of Brazilian Amazon protected areas in climate change mitigation. Proceedings of the National Academy of Sciences 107: 10821–10826.
  19. 19. Adeney JM, Christensen NL Jr, Pimm SL (2009) Reserves Protect against Deforestation Fires in the Amazon. PLoS ONE 4: e5014.
  20. 20. Nepstad D, Schwartzman S, Bamberger B, Santilli M, Ray D, et al. (2006) Inhibition of Amazon Deforestation and Fire by Parks and Indigenous Lands. Conservation Biology 20: 65–73.
  21. 21. Arturo Sánchez-Azofeifa G, Daily GC, Pfaff ASP, Busch C (2003) Integrity and isolation of Costa Rica's national parks and biological reserves: examining the dynamics of land-cover change. Biological Conservation 109: 123–135.
  22. 22. Pfaff A, Robalino J, Sanchez-Azofeifa GA, Andam K, Ferraro PJ (2009) Park Location Affects Forest Protection: Land Characteristics Cause Differences in Park Impacts across Costa Rica. The BE Journal of Economic Analysis & Policy 9: 26.
  23. 23. Andam KS, Ferraro PJ, Pfaff A, Sanchez-Azofeifa GA, Robalino JA (2008) Measuring the effectiveness of protected area networks in reducing deforestation. Proceedings of the National Academy of Sciences 105: 16089–16094.
  24. 24. Southworth J, Nagendra H, Carlson LA, Tucker C (2004) Assessing the impact of Celaque National Park on forest fragmentation in western Honduras. Applied Geography 24: 303–322.
  25. 25. Curran LM, Trigg SN, McDonald AK, Astiani D, Hardiono YM, et al. (2004) Lowland Forest Loss in Protected Areas of Indonesian Borneo. Science 303: 1000–1003.
  26. 26. Gaveau DLA, Epting J, Juhn D, Kanninen M, Leader-Williams N (2009) The future of forests and orangutans (Pongo abelii) in Sumatra: predicting impacts of oil palm plantations, road construction, and mechanisms for reducing carbon emissions from deforestation. Environmental Research Letters 4: 1–11.
  27. 27. Gaveau DLA, Linkie M, Suyadi , Levang P, Leader-Williams N (2009) Three decades of deforestation in southwest Sumatra: Effects of coffee prices, law enforcement and rural poverty. Biological Conservation 142: 597–605.
  28. 28. Gaveau DLA, Epting J, Lyne O, Linkie M, Kumara I, et al. (2009) Evaluating whether protected areas reduce tropical deforestation in Sumatra. Journal of Biogeography 36: 2165–2175.
  29. 29. Gorenflo LJ, Corson C, Chomitz KM, Harper G, Honzák M, et al. (2011) Exploring the Association Between People and Deforestation in Madagascar. In: Cincotta RP, Gorenflo LJ, editors. Human Population: Its Influences on Biological Diversity. Berlin: Springer. pp. 197–221.
  30. 30. Oliveira PJC, Asner GP, Knapp DE, Almeyda A, Galvan-Gildemeister R, et al. (2007) Land-Use Allocation Protects the Peruvian Amazon. Science 317: 1233–1236.
  31. 31. Cropper M, Puri J, Griffiths C (2001) Predicting the Location of Deforestation: The Role of Roads and Protected Areas in North Thailand. Land Economics 77: 172–186.
  32. 32. DeFries R, Hansen A, Newton AC, Hansen MC (2005) Increasing isolation of protected areas in tropical forests over the past twenty years. Ecological Applications 15: 19–26.
  33. 33. Joppa LN, Pfaff A (2010) Global protected area impacts. Proceedings of the Royal Society B: Biological Sciences.
  34. 34. PaulFerraro J, SubhrenduPattanayak K (2006) Money for Nothing? 7 p. A Call for Empirical Evaluation of Biodiversity Conservation Investments. PLoS Biology April 11, 2006 ed.
  35. 35. Leader-Williams N, Harrison J (1990) Designing protected areas to conserve natural resources. Science Progress 74: 189–204.
  36. 36. Joppa LN, Pfaff A (2009) High and Far: Biases in the Location of Protected Areas. PLoS ONE 4: e8273.
  37. 37. Cropper M, Puri J, Griffiths C (2001) Predicting the location of deforestation: The role of roads and protected areas in North Thailand. Land Economics 77: 172–186.
  38. 38. Pfaff A, Robalino J, Sanchez-Azofeifa A, Andam K, Ferraro P (2009) Location affects protection: Observable characteristics drive park impacts in Costa Rica. The BE Journal of Economic Analysis & Policy 9:
  39. 39. Olson DM, Dinerstein E, Wikramanayake E, Burgess N, Powell G, et al. (2001) Terrestrial Ecoregions of the World: A New Map of Life on Earth. BioScience 51: 933–938.
  40. 40. Bartholome E, Belward AS (2005) GLC2000: a new approach to global land cover mapping from Earth observation data. International Journal of Remote Sensing 26: 1959–1977.
  41. 41. Hansen M, DeFries R, Townshend JR, Carroll M, Dimiceli C, et al. (2003) Global Percent Tree Cover at a Spatial Resolution of 500 Meters: First Results of the MODIS Vegetation Continuous Fields Algorithm. Earth Interactions 7: 1–15.
  42. 42. Hansen MC, Stehman SV, Potapov PV, Loveland TR, Townshend JRG, et al. (2008) Humid tropical forest clearing from 2000 to 2005 quantified by using multitemporal and multiresolution remotely sensed data. Proceedings of the National Academy of Sciences 105: 9439–9444.
  43. 43. FAO (2006) Global Forest Resources Assessment 2005. Rome, Italy: Food and Agriculture Organization of the United Nations. 348 p.
  44. 44. Schmitt CB, Burgess ND, Coad L, Belokurov A, Besançon C, et al. (2009) Global analysis of the protection status of the world's forests. Biological Conservation 142: 2122–2130.
  45. 45. Stott P (2000) Combustion in tropical biomass fires: a critical review. Progress in Physical Geography 24: 355–377.
  46. 46. Applegate G, Chokkalingam U, Suyanto (2001) The Underlying Causes and Impacts of Fires in South-east Asia. Jakarta: CIFOR.
  47. 47. Aragao LEOC, Shimabukuro YE (2010) The Incidence of Fire in Amazonian Forests with Implications for REDD. Science 328: 1275–1278.
  48. 48. Morton DC, Defries RS, Randerson JT, Giglio L, Schroeder W, et al. (2008) Agricultural intensification increases deforestation fire activity in Amazonia. Global Change Biology 14: 2262–2275.
  49. 49. Giglio L, Descloitres J, Justice CO, Kaufman Y (2003) An enhanced contextual fire detection algorithm for MODIS. Remote Sensing of Environment 87: 273–282.
  50. 50. Roy DP, Boschetti L, Justice CO, Ju J (2008) The Collection 5 MODIS Burned Area Product - Global Evaluation by Comparison with the MODIS Active Fire Product. Remote Sensing of Environment 112: 3690–3707.
  51. 51. UNEP/IUCN (2009) Protected areas extracted from the 2009 World Database on Protected Areas (WDPA). World Conservation Union (IUCN) and UNEP-World Conservation Monitoring Center (UNEP-WCMC).
  52. 52. Gaston KJ, Jackson SF, Cantu-Salazar L, Cruz-Pinon G (2008) The Ecological Performance of Protected Areas. Annual Review of Ecology, Evolution, and Systematics 39: 93–113.
  53. 53. Tang Z, Fang J, Sun J, Gaston KJ (2011) Effectiveness of Protected Areas in Maintaining Plant Production. PLoS ONE 6: e19116.
  54. 54. Dudley N, editor. (2008) Guidelines for Applying Protected Area Management Categories. Gland, Switzerland: IUCN.
  55. 55. Nelson A (2008) Travel time to major cities: A global map of Accessibility. Ispra: European Commission.
  56. 56. (2009) Getting There. Science 323: 19.
  57. 57. Nelson A, de Sherbinin A, Pozzi F (2006) Towards development of a high quality public domain global roads database. Data Science Journal 5: 223–265.
  58. 58. CIESIN (2004) Global Rural-Urban Mapping Project (GRUMP), Alpha Version: Population Density Grids. Socioeconomic Data and Applications Center (SEDAC), Columbia University.
  59. 59. Reuter HI, Nelson A, Jarvis A (2007) An evaluation of void filling interpolation methods for SRTM data. International Journal of Geographic Information Science 21: 983–1008.
  60. 60. Farr TG, Kobrick M (2000) Shuttle Radar Topography Mission produces a wealth of data. American Geophysical Union Earth - Oceans - Atmosphere 81: 583–585.
  61. 61. Huffman GJ, Adler RF, Arkin P, Chang A, Ferraro R, et al. (1997) The Global Precipitation Climatology Project (GPCP) Combined Precipitation Dataset. Bulletin of the American Meteorological Society 78: 5–20.
  62. 62. Hijmans R, Garcia N, Kapoor J, Rala A, Maunahan A, et al. (2008) Global Administrative Areas (version 0.9).
  63. 63. Olson DM, Dinerstein E, Wikramanayake ED, Burgess ND, G.V.N. Powell GVN, et al. (2001) Terrestrial Ecoregions of the World: A New Map of Life on Earth. BioScience 51: 933–938.
  64. 64. Ho DE, Imai K, King G, Stuart EA (2007) Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference. Political Analysis mpl013.
  65. 65. Morgan SL, Harding DJ (2006) Matching Estimators of Causal Effects: Prospects and Pitfalls in Theory and Practice. Sociological Methods Research 35: 3–60.
  66. 66. Sekhon JS (2007) Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching package for R. Journal of Statistical Software.
  67. 67. Schenker N (1985) Qualms About Bootstrap Confidence Intervals. Journal of the American Statistical Association 80: 360–361.
  68. 68. Chomitz KM (2002) Baseline, leakage and measurement issues: how do forestry and energy projects compare? Climate Policy 2: 35–49.