Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Using Qualitative and Quantitative Methods to Choose a Habitat Quality Metric for Air Pollution Policy Evaluation

Using Qualitative and Quantitative Methods to Choose a Habitat Quality Metric for Air Pollution Policy Evaluation

  • Edwin C. Rowe, 
  • Adriana E. S. Ford, 
  • Simon M. Smart, 
  • Peter A. Henrys, 
  • Mike R. Ashmore


Atmospheric nitrogen (N) deposition has had detrimental effects on species composition in a range of sensitive habitats, although N deposition can also increase agricultural productivity and carbon storage, and favours a few species considered of importance for conservation. Conservation targets are multiple, and increasingly incorporate services derived from nature as well as concepts of intrinsic value. Priorities vary. How then should changes in a set of species caused by drivers such as N deposition be assessed? We used a novel combination of qualitative semi-structured interviews and quantitative ranking to elucidate the views of conservation professionals specialising in grasslands, heathlands and mires. Although conservation management goals are varied, terrestrial habitat quality is mainly assessed by these specialists on the basis of plant species, since these are readily observed. The presence and abundance of plant species that are scarce, or have important functional roles, emerged as important criteria for judging overall habitat quality. However, species defined as ‘positive indicator-species’ (not particularly scarce, but distinctive for the habitat) were considered particularly important. Scarce species are by definition not always found, and the presence of functionally important species is not a sufficient indicator of site quality. Habitat quality as assessed by the key informants was rank-correlated with the number of positive indicator-species present at a site for seven of the nine habitat classes assessed. Other metrics such as species-richness or a metric of scarcity were inconsistently or not correlated with the specialists’ assessments. We recommend that metrics of habitat quality used to assess N pollution impacts are based on the occurrence of, or habitat-suitability for, distinctive species. Metrics of this type are likely to be widely applicable for assessing habitat change in response to different drivers. The novel combined qualitative and quantitative approach taken to elucidate the priorities of conservation professionals could be usefully applied in other contexts.


Atmospheric nitrogen (N) pollution is having severe impacts on human health [1] and is a worldwide threat to biodiversity [2,3]. Nitrogen changes the competitive balance among plant species, and those that are negatively affected by N tend to have greater threat status [4]. Nitrogen pollution also influences a variety of other taxa (e.g. [5]). However, N pollution can favour species that indicate good site condition such as certain Sphagnum mosses [6], and species which are important components of protected habitats [7,8]. Species richness may increase despite long-term N application when species that are adapted for nutrient-rich habitats invade [9]. Despite these ostensibly beneficial effects it is widely recognised that atmospheric N pollution has mainly negative impacts on biodiversity [1012]. Policy development in this area makes extensive use of simulations of the ecological impacts of N pollution scenarios [13,14]. Models are available which simulate aspects such as habitat-suitability for phyto-sociological communities [15], proportional cover of plant species [16], or habitat-suitability for plant and lichen species [1719], and these models are often used to explore air pollution scenarios, e.g. at the annual workshop of the Co-ordination Centre for Effects (CCE) of the Convention on Long Range Transboundary Air Pollution. Progress made in terms of predicting ecological responses has however outstripped the capacity to interpret these changes in terms of biodiversity targets. In the current study we consulted habitat specialists using qualitative and quantitative methods to determine an appropriate basis for a habitat quality metric for use in this context.

Biodiversity is a complex and much-debated concept, and it is often argued that it needs to be represented using many dimensions (e.g. [2022]). A project designed to streamline biodiversity indicators still resulted in a short-list of 26 indicators [23]. However, air pollution policy makes extensive use of the concept of critical load [24], which relies on assessment of whether a habitat is damaged or not, and thus requires definition of a single summary metric of habitat quality (HQ). A one-dimensional HQ metric is also needed if biodiversity impacts are to be included in integrated assessments of air pollution effects across sectors [25]. In this study the term ‘habitat’ is used in the sense of a biotope, i.e. an area with environmental conditions that support a particular biotic assemblage, rather than in the sense of the environment of a particular species. The term ‘quality’ is used in the sense of ‘degree of excellence’ [26].

The chance of achieving agreement on a biodiversity metric may be increased by limiting its scope to terrestrial ecosystems. Also, since transitions between major habitat types are more usually mediated by management than pollution, it is often possible to disregard β- and γ-diversity in the context of pollution impacts and to consider the quality of the habitat rather than site-scale or landscape-scale biodiversity. Nevertheless, even for a single habitat many different approaches can be taken to defining biodiversity value, such as species richness, or the presence or abundance of particular taxa. Taxa considered important may be scarce or declining species; those important to the structure, function or integrity of the habitat; and/or those that provide important ecosystem services [27]. Species scarcity can be defined with reference to global, national or local populations. It may be argued that preventing extinction at global scale is of critical importance [28], and this is the basis of the “Red List” [29,30]. However, a local perspective can also be important, in that encountering species that are unusual or rare in a particular area foments an interest in nature conservation and ensures that nature conservation values persist in human society [31]. The presence of competitive, invasive or ubiquitous taxa is often considered to reduce biodiversity, although whether non-native species should necessarily be considered negative is debated [3234]. Biodiversity can be defined as similarity to a pre-industrial or other reference state, although there is some conflict between the ideal of “naturalness” and the appreciation that some human interventions can increase biodiversity [35,36].

Scientific justifications can be constructed for many of these criteria, but ultimately the balance of considerations used to define HQ depends on the value-system and priorities of whoever is making the judgement [3740]. This conclusion is rejected by many biodiversity scientists, and earlier versions of this article were indeed rejected by several journals for reasons best summarised by the comment of one reviewer: “the scientific basis of conservation is a rule”. However, it is difficult to see how biodiversity can be assessed in isolation from value judgements [41], and appropriate policy responses are best developed using insights and methods from both natural and social science [42]. The necessity for value judgments cannot be avoided by combining multiple criteria into a single metric, since the weighting of components is inevitably subjective and the combined metric is likely to be rather opaque [43]. The Biodiversity Indicators Partnership recommends discussion of key questions, and the use to which a potential biodiversity indicator will be put, in collaboration with stakeholders [44], and this was the guiding principle for our study.

The aim of the study was to assess quantitative indicators of HQ for their potential for evaluating N pollution impacts over a range of habitats. Our intention was to identify a general metric of habitat quality, not to assess whether this metric explicitly related to N deposition. Although the context of the study was an assessment of air pollution impacts, sensitivity to pollution is not widely seen as a valued component of biodiversity in itself. To ensure that the metric was chosen according to a robust evidential basis, we used an innovative combination of qualitative and quantitative techniques to canvass and analyse the views of a set of key informants. To our knowledge, this is the first time that such an approach has been applied to select a biodiversity indicator. We focused on national conservation policy experts to elucidate the views of those most actively involved in policy formulation and implementation, as outlined below. The study was based in the UK, where N pollution has had severe effects on biodiversity [45,46], but the approach and results are widely relevant to studies of the impacts of environmental change on biodiversity.

Materials and Methods

Interviews with key informants

The interviews, including preparation and data-analysis stages, conformed to the “consolidated criteria for reporting qualitative studies” [47]. We used a purposive or ‘judgement’ sampling approach [48], identifying the 16 key individuals (13 male, 3 female) from governmental organisations in England, Scotland, Wales and Northern Ireland that have statutory responsibilities for making decisions about the strategy for managing and monitoring particular habitats. These habitat specialists had expertise in one or more of the three major habitat types that were targeted in the study. Our aim was to develop a widely-applicable metric, so we selected specialists across a range of quite different habitat types, as defined at EUNIS Level 1: D: mires, bogs and fens; E: grasslands and lands dominated by forbs, mosses or lichens; and F: heathland, scrub and tundra. These habitats are widespread and clearly affected by N pollution [46], in contrast to woodlands, for which the evidence of N impacts is less consistent [49]. The specialists were contacted initially via e-mail, which included a briefing note to provide background to the project, and invited to participate in a semi-structured interview lasting around one hour, and a subsequent ranking exercise of habitat examples to be carried out in their own time. Of 16 specialists contacted, two declined citing time pressures, but 14 were able to participate, representing the majority of the key individuals identified. Interviews were conducted between 28th August and 5th September 2013 by Rowe (PhD, male) and Ford (PhD, female) in person at the participants’ places of work, apart from one telephone interview. Rowe has had training in and experience of participatory methods [50], and Ford has considerable experience of qualitative research in the context of nature conservation, including conducting in-depth and semi-structured interviews, surveys, focus groups, and analyses of stakeholder attitudes, values and participation [51]. Rowe had participated in the CCE process for around 10 years prior to the study, and was fully aware of the need to define a one-dimensional HQ metric, but did not have prior assumptions about the basis for such a metric. The participants were made aware that the purpose of the research was not to focus on aspects of the habitat that are sensitive to N, but to understand how overall HQ is assessed in practice. The study did not include vulnerable participants, so we did not seek consent from an institutional review board, but we took several steps to ensure that ethical standards were upheld including: i) prior to the interviews, we provided participants with a letter explaining the purpose of the study and how it would be conducted; ii) prior to the interviews, we provided participants with a letter of consent, explaining that the data from the interviews would be confidential and how the data would be used, which the participants read and signed; iii) we stated in the letter of consent and at the start of the interview that participants could withdraw from the study at any time; iv) the data are held securely and only accessible to the researchers who undertook the interviews; and v) publications resulting from the study maintain the anonymity and confidentiality of the participants.

A set of six topics that may be related to the assessment of HQ was defined in advance, and acted as our interview guide (Table 1). Although using pre-defined topics was inevitably somewhat normative, it was thought useful to remind the specialists of a wide range of potential considerations, and to ensure some consistency of coverage. The interviews were conducted in an informal style and did not cover the topics in a prescribed sequence, to allow for variation in questioning including additional questions and two-way dialogue. This approach retained the benefits of having a clear structure to aid the analysis process and keep the research focused, whilst also providing the flexibility needed to collect rich qualitative data. Habitat specialists were asked to describe the main features that are looked for when assessing HQ (Topic 1). For Topic 2, they were asked whether some species should be valued more than others, and to explore the basis for such evaluations. Within this topic, the specialists were also asked whether invasive species (whether native or non-native) should be considered negative per se or only when they supplant existing species. For Topic 3, the habitat specialists were asked what plant and lichen species would lead them to rank a site as having high or low HQ. The specialists were asked whether HQ can be assessed on the basis of presence or abundance of just plants and lichens, and if not, what other taxonomic groups are important (Topic 4); and whether species-groups (e.g. forbs, subshrubs, graminoids, grasses, mosses, Sphagna) are useful for assessing HQ (Topic 5). The specialists were also asked their opinion on a reference community approach to assessing HQ, where a site is measured against an ideal or target example of the community (Topic 6).

Table 1. Pre-defined topics covered by semi-structured interviews, and themes that emerged within these topics.

The habitat specialists’ responses were analysed under predefined topic headings (Table 1). The interviews were recorded digitally and transcribed the same day. Field notes were taken for reflection towards the end of the interview, to ensure that all topics were covered, but these notes were not used for analysis. Interview transcriptions were analysed by Ford using Atlas Ti [52]. This software facilitates marking-up of the transcripts under different topics, and subsequent collation and analysis of related text sections. Data were first coded according to each of the pre-defined six topics and by habitat (the ‘a priori codes’). Within each of the six topics, themes were then identified using an approach based on grounded theory, in which themes are allowed to emerge from the data, i.e. are not pre-determined by the researcher [53]. The pre-defined topics and the emergent themes are presented in Table 1. Participants were not asked to check transcriptions, but were supplied with a more extensive summary of responses than is presented here, and given a chance to respond and make corrections to interpretations and emphases.

Ranking exercise

Habitat examples.

Following the semi-structured interview, each habitat specialist was given a set of 12 examples for each habitat or habitats that they were responsible for. Specialists ranked a mean of 1.91 +/- 0.94 (standard deviation) sets each. The examples were taken from a database of 31,261 relevés originally used to develop the British National Vegetation Classification (NVC, [54]), and for the habitats considered in the current study were usually derived from 2×2 m relevés. All relevés were automatically assigned to the nearest NVC subcommunity using the MAVIS program [55] which uses Czekanowski’s quantitative index of similarity, i.e. taking into account the abundance as well as presence of species [56]. These NVC subcommunities were mapped onto EUNIS [57] Level 3 and Level 2 classes using established correspondences [58]. The NVC dataset was collected with the aim of sampling the full range of British habitats, including the best examples of particular habitat types, as well as from more modified sites. To ensure that this range was adequately represented by the examples, we calculated a preliminary metric for each example as the sum of its rank-scores for two simple measures: species-richness, and the inverse of the prevalence (proportion of UK 10×10 km gridsquares containing the species) of the most scarce species present. The habitat examples were ordered according to this preliminary metric, and one example was chosen at random from each of 12 strata. Each example was presented to the specialists as a list of all the plant and lichen species present in a defined area, with associated cover-score (Domin) values. Species were listed in descending order of abundance without distinguishing vascular plants, bryophytes and lichens.

Ranking by specialists.

The specialists were asked to rank the examples in order of “overall habitat quality”, by weighing different considerations and using their overall judgement, in the same way that they would when assessing whether to give a site a conservation designation or assessing a site using the concept of “favourable habitat condition” [59]. The specialists were asked to consider this ranking carefully and return the results at a later date (all rankings were returned within two weeks of the interview). It was explained that the examples should be assessed in relation to the definition of the habitat class in question. For example, a relatively species-rich relevé with very low sub-shrub cover would not be considered a high-quality example of a heathland, since sub-shrubs define heathland. The habitat specialists were given a free choice as to whether to rank examples at EUNIS Level 2 (e.g. D1 Raised and blanket bogs) or Level 3 (e.g. D1.1 Raised bogs). In one case the specialist first classified the set into two different types and then ranked each type separately, so we assigned the top-ranked and bottom-ranked example of each type an equal overall ranking and gave the other examples an overall ranking based on their ranking within the type, i.e. assuming that both types were of equal value. A total of 21 rankings was obtained, from nine habitat-classes (Table 2).

Table 2. Types of habitat considered in the study, and numbers of rankings obtained.

Comparisons with algorithmic metrics.

The specialist’s ranking of the examples was viewed as a definitive assessment of their overall HQ. When more than one specialist ranked a set of examples, the mean rank-order was calculated. Methods used to calculate different algorithmic metrics from the same examples are summarised in Table 3, with additional information below. All metrics were designed to be positively correlated with habitat quality. Each metric was assessed according to how well they correlated with the specialist’s rank order for the same examples, using Kendall’s Tau test.

All cover-based metrics were calculated by conversion of Domin scores [61] to the midpoint cover percentage of each class, i.e. Domin 1 = 1%, 2 = 2%, 3 = 3%, 4 = 7%, 5 = 18%, 6 = 29.5%, 7 = 42%, 8 = 63%, 9 = 83%, 10 = 95.5%. For each example, the value of Simpson’s Diversity Index Ds was calculated as (1)

Where ai is the cover value for species i and A is the total cover for all species.

Reference communities, with high nature conservation value, were derived from the NVC [54] by considering all NVC subcommunities considered to 'overlap with', be 'equal to' or be 'contained in' the EUNIS class, using a pre-existing correspondence table [58]. Similarities of each habitat example to corresponding NVC types were calculated using the Czekanowski quantitative index [62,63], i.e. (2) where yij and yik denote the cover of species i in the habitat example (j) and the reference community (k), respectively and p is the number of species. This index lies in the range between 0 and 1, and is 1 if the example and reference are identical. Since NVC tables include species from a large number of relevés they do not indicate the likely composition of a single relevé, so pseudo-relevés were generated for each NVC class for the similarity calculation, as explained in Tipping et al. [64]. For most EUNIS classes, several corresponding NVC types are listed, and it would be hard to justify selecting one of these as more valuable than another. We therefore applied two methods for calculating a similarity metric: the maximum similarity of the example to any corresponding NVC class; and the mean similarity to all corresponding NVC classes.

Positive and negative indicator species for each habitat (see S1 File) were obtained from the Common Standards Monitoring (CSM) guidance documents [6568], which are used to assess the condition of designated sites in the UK. The guidance includes lists of indicator-species, grouped into those whose presence (and sometimes abundance or prevalence) indicates favourable condition, referred to as positive indicator-species, and species that indicate unfavourable condition, i.e. negative indicator-species. Indicator-species were originally selected on the basis that they are typical or distinctive for the habitat; are useful for determining site condition; are not so scarce that they will rarely be observed; and occur across a wide geographic range (Richard Jefferson, pers. com.). There is not an exact match between the habitat classes used in CSM and the EUNIS classes for which metrics need to be derived, so CSM guidance for the most relevant class was selected for each EUNIS Level 3 habitat. For EUNIS Level 2 habitats the lists of indicator-species for component EUNIS Level 3 habitats were combined, although some species were excluded since they appeared as both positive and negative indicators for different sub-types of the habitat in question.

The cover proportions of species-groups such as grasses, forbs or subshrubs are an important aspect of habitat structure. Different species-groups are considered important for the structure and function of different habitats. Sphagnum moss species have important roles in water retention and peat formation in bogs. Heathlands are defined by their subshrub component, so we calculated total cover of subshrubs, including Ericaceae, Myrica, Ulex minor, U. gallii and all Genista, but excluding Ulex europaeus, all Rubus and all Salix. Forb cover is considered a useful condition measure for grasslands [69]. The metric we calculated was forb cover / total cover, which is more mathematically robust than grass / forb cover ratio and increases with greater forb cover. Forbs were assumed to be all herbs apart from Graminae, and included sedges and allies (e.g. Eriophorum and Luzula), rushes, horsetails and ferns, and partially woody genera such as Helianthemum and Hypericum. Taxa not included as forbs were grasses, trees, shrubs and subshrubs (including Rubus and Myrica), mosses and lichens.

Vascular plant and bryophyte species have been scored according to where they are likely to occur in relation to different environmental axes. These scores were originally assigned by Ellenberg [70] and here we refer to them as ‘Ellenberg’ scores, but in fact we used scores derived algorithmically from UK vascular plant and bryophyte occurrence data by Mark Hill and colleagues [71,72]. Mean values of the Ellenberg ‘N’ (EN) score correspond to plant productivity [73,74], and in survey data have been shown to be correlated with N deposition rate [6,75].


Qualitative analysis of interview responses

The habitat specialists’ interview responses were analysed based on six pre-defined topic headings and 22 themes (see Table 1). The key messages relating to each of these themes are summarised in Table 4, along with example supporting quotations. A more detailed summary, including further quotations, is provided in the S2 File.

The interview data highlight the complexity surrounding specialists’ assessment of habitat quality. Various features of habitats have importance in assessing quality, including habitat functioning and ecosystem services delivery, vegetation composition and structure, cover of species-groups, and presence or abundance of individual species. Habitat type, geographical context (e.g. scale, altitude and location) and historical context and management may all also affect judgements of quality. As expected, some species are valued more highly than others, such as those with a significant structural or functional role for the habitat, or scarce species–although scarce species have a limited role in overall habitat quality assessment, principally because they are found on few sites. Species considered to be indicators of low habitat quality are typically those that are invasive (native or non-native) and have detrimental impacts on valued species; or species that directly indicate poor environmental conditions such as inappropriate management. Conversely, the presence of species that indicate good environmental conditions and/or are distinctive for the habitat is seen as indicating high habitat quality. Plants and lichen species are typically considered more useful than other taxa for assessing habitat quality; although other taxa can be useful, there are practical constraints in using them widely. Our analysis also revealed that varied and distinct examples may be considered of high quality within a particular habitat type, for example because of local variation, so it would be challenging to identify a suitable reference community for a given habitat.

Comparison of specialists’ and algorithmic rankings

The correlations between the mean ranks assigned by specialists and the ranks of the same set of examples according to different metrics are illustrated for the habitat for which the most rankings were obtained (F4, Temperate shrub heathland) in Fig 1, and summarised for all habitats in Table 5. For example, assessing examples of heathland using the number of positive indicator-species resulted in a closer correlation with specialists’ rankings than did ranking by species-richness (Fig 1).

Fig 1. Correlations of habitat specialists’ rankings with algorithmic rankings, for heathland.

Correlations between rank-scores given by habitat specialists to 12 examples of EUNIS class F4 (Temperate shrub heathland) and rank scores for metrics based on algorithms applied to the same examples: Species richness; Simpson diversity; Scarcity, −1 × UK prevalence of least-prevalent species present; number of positive indicator-species; −1 × number of negative indicator-species; number of positive indicator-species minus number of negative indicator-species; subshrub cover; greatest Czekanowski similarity to corresponding National Vegetation Classification (NVC) subcommunities; Infertility, −1 × mean Ellenberg N score. Tau, Kendall’s Tau statistic.

Table 5. Coefficients for correlations between habitat specialists’ rankings of examples of different habitats and algorithmic rankings.

The number of positive indicator-species was the metric most consistently associated with specialists’ rankings with correlations in seven of the nine classes, and was the most closely correlated metric in six of the nine cases. Subtracting the number of negative indicator-species mainly did not improve the correlation, with the exception of D2 (valley mires, poor fens and transition mires). Species-richness was correlated with specialists’ rankings in grassland habitats and was the most closely correlated of all metrics for the two heathland subclasses, although there was no correlation for the overall heathlands class. Simpson’s diversity index was correlated for five of the nine habitat classes, although only at marginal significance levels. Values for the scarcity metric were not correlated. Both maximum and mean similarity to corresponding NVC communities were significantly correlated with specialists’ rankings for five classes, and mean similarity was the most closely correlated of all metrics for D1.2 (blanket bog).

The cover of functional groups was only significantly correlated with specialists’ rankings in the case of Sphagnum in the bog and mire classes. The cover of subshrubs in heathlands was not correlated with heathland HQ. The lack of correlation with cover ratio of forbs in grasslands can be explained by examples with large cover values for species typical of fertile and/or disturbed swards.

Similarity to reference assemblages was significantly correlated with HQ for most sets of grassland examples and for dry heathland, although not for bogs. In some cases the correlation was closer when using similarity to the most-similar reference assemblage, but in some cases the mean similarity to all reference assemblages for the habitat class better reflected HQ. The Infertility index was strongly correlated with HQ in E2 ‘Mesic grasslands’, and was significantly correlated in four of the other habitat classes, but the association was not as consistent and clear as it was for positive indicator-species.


Priorities as to what aspects of biodiversity should be protected are inevitably subjective, despite considerable efforts to evaluate biodiversity metrics objectively (e.g. [76,77]). Drawing upon social research methods to explore the opinions of conservation professionals allows a more meaningful and justified analysis than can be obtained using only natural science approaches [78]. The use of semi-structured interviews necessitated a small sample size, but was appropriate for the research objectives as it provided an insight into the value judgements of key individuals [48], in this case those responsible for designating, managing and monitoring habitats. Although our sample was not extensive enough to ensure data saturation, due to logistical constraints, it nevertheless captured views of a range of key specialists with decision-making roles. It proved very useful to discuss biodiversity concepts and issues before the ranking exercise, and in particular to explain that the study aimed to explore the basis for assessing HQ in general rather than assessing sensitivity to N in particular. The overall pattern of correlations with alternative metrics (Table 5) corroborated the qualitative analysis, and provided additional information which allows metrics to be assessed. Rank-correlation has been used previously to assess different biodiversity metrics in relation to the first axis of a plant community ordination [79], but to our knowledge this is the first study to apply qualitative and quantitative methods to assess the relationship between different biodiversity metrics and the priorities of conservation professionals.

The interview responses revealed that vegetation composition is very important for assessing habitat condition. Taxa other than plants and lichens are clearly important, but practical constraints mean that vegetation composition is the most commonly used basis for discriminating between sites (Table 4). This implies that presenting data on floristic composition did not misrepresent the examples. It is advantageous when assessing biotic integrity to have data for different taxon groups [80], but where such data are lacking it is necessary to consider surrogate taxa. Plants are important for habitat structure and energy inputs, and provide a diversity of chemical substrates and toxins likely to influence biodiversity at other trophic levels. Effects of environmental drivers on other taxa are often mediated by vegetation–for example, declines in butterfly abundance driven by N pollution are likely due to the loss of suitable micro-sites in more rank vegetation [81]. Plants are also suitable surrogate taxa for assessing overall biodiversity since they are easily observed and expertise in their identification is comparatively widespread [82], as also noted by the respondents.

The procedure for assigning examples to habitats was related to the algorithmic metrics based on Czekanowski similarity, with the difference that the assignment was binary (in or out of the habitat class) whereas the metric was continuous. Also, a combination of species-richness and species scarcity was used for the preliminary stratification, to ensure a broad range of examples. Other metrics were not directly related to the methods used to select examples. It seems unlikely that the methods used to assign and stratify the examples could have led to circularity in choosing one of the related metrics, even if that had been the outcome of the study. Species scarcity was not significantly correlated with the specialists’ assessments for any habitat, and species richness was the most strongly correlated for only two of the nine habitat classes. The method for presenting the examples was limited, in that a visual impression was not presented. Not all of the topics and themes that emerged in discussions were represented by the examples–in particular biophysical aspects, the size of the habitat stand, and characteristics of the site rather than the individual habitat. However, the main aim of the study was to assess metrics for within-habitat biodiversity, for which species lists with abundances provide adequate information. Photographs would have provided additional insight into the habitat, but were not available for the example relevés, and in any case are difficult to standardise since the appearance of a stand depends on the species in flower, the light conditions, and the skills of the photographer. Another consideration was that ultimate aim of the study was to allow interpretation of model outputs, and the current generation of pollution impacts models is unable to generate visual impressions of different plant assemblages.

Species richness is clearly related to public perceptions of biodiversity value, although its limitations as a biodiversity metric are recognised in the scientific literature [83]. Species richness may not reflect phylogenetic richness, nor the presence of rare species [84]. An increase in species richness may reflect the invasion of atypical species [9], and this concern emerged frequently in the interviews, particularly when discussing bogs and heaths (see S2 File).

Scarce species were an important priority for the specialists, but were considered impractical to use for site assessment. Reflecting this, the national prevalence of the most-scarce plant species present was not related to overall assessments. Schemes for assessing sites by combining species-weights based on scarcity (e.g. [85]) have proved difficult to implement, principally due to disagreements over the weighting that should be given to different species [86]. Metrics based on similarity to a reference assemblage were also generally rejected by the specialists, mostly on the basis that there was too much variation in what is considered an ideal or target species composition for a particular habitat class. Rankings based on similarity to a reference were well-correlated with specialists’ rankings for some of the habitats, but this correlation not consistent, e.g. there was no correlation for F4.2 ‘Dry temperate shrub heathland’. Mean fertility score (Ellenberg N) has been shown to be correlated with N deposition rate [45], and rankings based on mean Ellenberg N were correlated with specialists’ rankings for five of the nine habitat classes, but was the most highly correlated of the metrics only for one class, F4 ‘Temperate shrub heathland’.

The cover of functionally important groups was highlighted as an important factor by many of the specialists. However, rankings based on the total cover of these functionally important groups did not correspond to the specialists’ rankings, with the exception of Sphagnum cover for bogs. Sphagnum is important for the maintenance of bog function, and Sphagnum cover was correlated with HQ, albeit weakly, for all three of the bog / mire classes assessed. Functionally important groups did not appear to be important for assessment of the other two classes studied, even though total forb cover is frequently cited as indicating better-quality grasslands, and subshrub cover is definitive for heathland. Functional-group abundance may be important for specific habitats, but this metric was only superior to that based on positive indicator-species in two cases, and may not be as widely applicable.

The specialists referred frequently and favourably to positive indicator-species, and a metric based on the number of positive indicator-species gave the most consistent correlation with the specialists’ rankings. Methods used to select indicator species are often insufficiently clear [87,88], but indicator-species are commonly used in ecological assessment (e.g. [89]) and the current study suggests that this use is appropriate. The presence of a few key taxa has been shown to be a good predictor of total species richness across many taxa [90]. The positive indicator-species used in the study were originally selected on the basis that they are typical of and distinctive for the habitat, and indicate favourable site conditions [91]. These aspects help explain why correlations with overall habitat quality were relatively strong for this metric. In particular, the aspect of distinctiveness–species that are not necessarily scarce, but are restricted to good-quality examples of the habitat–emerged as critical. Similarly, in a study of Brazilian sites, Trindade et al. [92] found that the best indicators of mammal diversity were species with restricted ranges.

Negative indicator-species were also discussed in many of the interviews, although they were commonly seen as indicating poor site conditions rather than being damaging in their own right. Although some negative indicator species indicate that the abiotic environment is changing adversely for the habitat, non-native invasive species in particular can invade sites even when environmental conditions are unchanged. The distinct responses of these two groups may explain the lack of correlation between the number of negative indicator-species in the examples and the specialists’ HQ rankings.

The study demonstrated that despite the diverse criteria used to assess the quality of a range of habitats across the varied conditions of the UK, it is possible to find a single metric that gives an indication of other aspects of habitat quality. The results from the study were instrumental in the adoption by a recent CCE Workshop of habitat suitability for characteristic species as a common indicator for use in air pollution impacts modelling by all Signatory Parties [93]. The specific metric selected in this study is likely to be applicable to only some ecosystems, policy contexts, and environmental threats. However, the approach taken proved capable of providing sufficient evidence to support a course of action in a complex and much-debated policy area. The combination of qualitative and quantitative methods used to summarise expert judgements is likely to be effective at other interfaces between science and policy.


The evidence provided by analysis of the habitat specialists’ responses suggests that the presence of positive indicator-species is the most suitable basis for a biodiversity metric for use in the context of evaluating habitat damage and recovery from air pollution. The metric most consistently related to overall habitat quality was the number of positive indicator-species present. Models are available which predict the habitat-suitabilities for a large number of plant and lichen species under different pollution scenarios [13], and predicting suitability for positive indicator-species is a useful focus for this modelling work. The success of positive-indicator species in indicating overall quality suggests that identifying species that discriminate good-quality examples of a habitat should be a priority for conservation assessment. The combined quantitative and qualitative approach taken to determining an appropriate metric is likely to be widely applicable in other fields where it is necessary to summarise expert judgements.

Supporting Information

S2 File. Extensive summary of interviews.



The work was funded by the UK Department for the Environment, Food and Rural Affairs (DEFRA) under project AQ0828. A version of this study was presented as a DEFRA report in 2013 [94]. We are very grateful for the time and enthusiasm of the habitat specialists who contributed substantially to this report through participation in the interviews and the ranking exercise: Isabel Alonso, Paul Corbett, Andrew Coupar, Alistair Crowle, Iain Diack, Richard Jefferson, Peter Jones, Jane Mackintosh, Andrew McBride, Michael Rebane, Jan Sherry, Stuart Smith, Graham Sullivan and Alex Turner. We would also like to thank the project steering committee (Clare Whitfield, Zoe Russell and Lynette Clapp) who helped considerably with defining the study.

Author Contributions

  1. Conceived and designed the experiments: ECR AESF.
  2. Performed the experiments: ECR AESF.
  3. Analyzed the data: ECR AESF.
  4. Contributed reagents/materials/analysis tools: SMS PAH.
  5. Wrote the paper: ECR AESF MRA.


  1. 1. Kampa M, Castanas E (2008) Human health effects of air pollution. Environmental Pollution 151: 362–367. pmid:17646040
  2. 2. Sala OE, Chapin FS, Armesto JJ, Berlow E, Bloomfield J, Dirzo R, et al. (2000) Biodiversity—Global biodiversity scenarios for the year 2100. Science 287: 1770–1774. pmid:10710299
  3. 3. Bleeker A, Hicks WK, Dentener E, Galloway J, Erisman JW (2011) N deposition as a threat to the World's protected areas under the Convention on Biological Diversity. Environmental Pollution 159: 2280–2288. pmid:21122958
  4. 4. Hodgson JG, Tallowin J, Dennis RLH, Thompson K, Poschlod P, Dhanoa MS, et al. (2014) Leaf nitrogen and canopy height identify processes leading to plant and butterfly diversity loss in agricultural landscapes. Functional Ecology 28: 1284–1291.
  5. 5. Feest A, van Swaay C, van Hinsberg A (2014) Nitrogen deposition and the reduction of butterfly biodiversity quality in the Netherlands. Ecological Indicators 39: 115–119.
  6. 6. Emmett BA, Rowe EC, Stevens CJ, Gowing DJ, Henrys PA, Maskell LC, et al. (2011) Interpretation of evidence of nitrogen impacts on vegetation in relation to UK biodiversity objectives.: JNCC Report 449. JNCC, Peterborough, UK. 105 p.
  7. 7. Henrys PA, Stevens CJ, Smart SM, Maskell LC, Walker KJ, Preston CD, et al. (2011) Impacts of nitrogen deposition on vascular plants in Britain: An analysis of two national observation networks. Biogeosciences 8: 3501–3518.
  8. 8. Stevens CJ, Smart SM, Henrys PA, Maskell LC, Walker KJ, Preston CD, et al. (2011) Collation of evidence of nitrogen impacts on vegetation in relation to UK biodiversity objectives.: JNCC Report 447. JNCC, Peterborough, UK.
  9. 9. Pierik M, van Ruijven J, Bezemer TM, Geerts R, Berendse F (2011) Recovery of plant species richness during long-term fertilization of a species-rich grassland. Ecology 92: 1393–1398. pmid:21870612
  10. 10. Bobbink R, Hicks K, Galloway J, Spranger T, Alkemade R, Ashmore M, et al. (2010) Global assessment of nitrogen deposition effects on terrestrial plant diversity: a synthesis. Ecological Applications 20: 30–59. pmid:20349829
  11. 11. Erisman JW, Galloway JN, Seitzinger S, Bleeker A, Dise NB, Petrescu AMR, et al. (2013) Consequences of human modification of the global nitrogen cycle. Philosophical Transactions of the Royal Society B-Biological Sciences 368.
  12. 12. Butchart SHM, Walpole M, Collen B, van Strien A, Scharlemann JPW, Almond REA, et al. (2010) Global Biodiversity: Indicators of Recent Declines. Science 328: 1164–1168. pmid:20430971
  13. 13. de Vries W, Wamelink W, van Dobben H, Kros H, Reinds G- J, Mol-Dijkstra J, et al. (2010) Use of dynamic soil-vegetation models to assess impacts of nitrogen deposition on plant species composition and to estimate critical loads: an overview. Ecological Applications 20: 60–79.
  14. 14. Rowe EC, Wamelink GWW, Smart SM, Butler A, Henrys PA, van Dobben HF, et al. (2015) Field survey based models for exploring nitrogen and acidity effects on plant diversity and assessing long-term critical loads. In: de Vries W, Hettelingh J-P, Posch M, editors. Critical loads for nitrogen, acidity and metals for terrestrial and aquatic ecosystems: Springer, Dordrecht, Netherlands. pp. 297–326.
  15. 15. Schlutow A, Dirnböck T, T. P , Scheuschner T (2015) Use of an empirical model approach for modelling trends of ecological sustainability. In: de Vries W, Hettelingh J-P, editors. Critical loads and dynamic risk assessments: nitrogen, acidity and metals in terrestrial and aquatic ecosystems: Springer, Dordrecht, Netherlands. pp. 381–402.
  16. 16. Belyazid S, Kurz D, Braun S, Sverdrup H, Rihm B, Hettelingh JP (2011) A dynamic modelling approach for estimating critical loads of nitrogen based on plant community changes under a changing climate. Environmental Pollution 159: 789–801. pmid:21145634
  17. 17. Smart SM, Scott WA, Whitaker J, Hill MO, Roy DB, Critchley CN, et al. (2010) Empirical realised niche models for British higher and lower plants—development and preliminary testing. Journal of Vegetation Science 21: 643–656.
  18. 18. Reinds GJ, Bonten L, Mol-Dijkstra JP, Wamelink GWW, Goedhart P (2012) Combined effects of air pollution and climate change on species diversity in Europe: First assessments with VSD+ linked to vegetation models. In: Posch M, Hettelingh JP, editors. CCE Status Report 2012: Co-ordination Centre for Effects. pp. 49–61.
  19. 19. Henrys PA, Smart SM, Rowe EC, Jarvis SG, Fang Z, Evans CD, et al. (2015) Niche models for British plants and lichens obtained using an ensemble approach. New Journal of Botany 5: 89–100.
  20. 20. Margules C, Usher MB (1981) Criteria used in assessing wildlife conservation potential—a review. Biological Conservation 21: 79–109.
  21. 21. Feest A (2013) The utility of the Streamlining European Biodiversity Indicators 2010 (SEBI 2010). Ecological Indicators 28: 16–21.
  22. 22. Leroy B, Le Viol I, Petillon J (2014) Complementarity of rarity, specialisation and functional diversity metrics to assess community responses to environmental changes, using an example of spider communities in salt marshes. Ecological Indicators 46: 351–357.
  23. 23. EEA (2007) Halting the loss of biodiversity by 2010: proposal for a first set of indicators to monitor progress in Europe. European Environment Agency, Technical report No 11/2007.
  24. 24. Nilsson J, Grennfelt P (1988) Critical loads for sulphur and nitrogen. Report 188:15. Copenhagen, Denmark: UNECE/Nordic Council of Ministers.
  25. 25. Amann M, Bertok I, Borken-Kleefeld J, Cofala J, Heyes C, Hoglund-Isaksson L, et al. (2011) Cost-effective control of air quality and greenhouse gases in Europe: Modeling and policy applications. Environmental Modelling & Software 26: 1489–1501.
  26. 26. Pearsall J, editor (1998) The New Oxford Dictionary of English.: Oxford University Press. 2152 p.
  27. 27. Isbell F, Calcagno V, Hector A, Connolly J, Harpole WS, Reich PB, et al. (2011) High plant diversity is needed to maintain ecosystem services. Nature 477: 199–U196. pmid:21832994
  28. 28. Nicholson E, Keith DA, Wilcove DS (2009) Assessing the threat status of ecological communities. Conservation Biology 23: 259–274. pmid:19245533
  29. 29. Mace GM, Stuart SN (1994) Draft IUCN Red List Categories, Version 2.2. Species 21–22: 13–24.
  30. 30. IUCN (2009) The IUCN Red List of threatened species.
  31. 31. Miller JR (2005) Biodiversity conservation and the extinction of experience. Trends in Ecology & Evolution 20: 430–434.
  32. 32. Davis MA, Chew MK, Hobbs RJ, Lugo AE, Ewel JJ, Vermeij GJ, et al. (2011) Don't judge species by their origins. Nature 474: 153–154. pmid:21654782
  33. 33. Simberloff D, Alexander J, Allendorf F, Aronson J, Antunes PM, Bacher S, et al. (2011) Non-natives: 141 scientists object. Nature 475: 36–36.
  34. 34. Van Der Wal R, Fischer A, Selge S, Larson BMH (2015) Neither the public nor experts judge species primarily on their origins. Environmental Conservation 42: 349–355.
  35. 35. Ridder B (2007) An exploration of the value of naturalness and wild nature. Journal of Agricultural & Environmental Ethics 20: 195–213.
  36. 36. Heckenberger MJ, Russell JC, Toney JR, Schmidt MJ (2007) The legacy of cultural landscapes in the Brazilian Amazon: implications for biodiversity. Philosophical Transactions of the Royal Society B-Biological Sciences 362: 197–208.
  37. 37. Failing L, Gregory R (2003) Ten common mistakes in designing biodiversity indicators for forest policy. Journal of Environmental Management 68: 121–132. pmid:12781752
  38. 38. Hull RB, Richert D, Seekamp E, Robertson D, Buhyoff GJ (2003) Understandings of environmental quality: Ambiguities and values held by environmental professionals. Environmental Management 31: 1–13. pmid:12447571
  39. 39. Fischer A, Young JC (2007) Understanding mental constructs of biodiversity: Implications for biodiversity management and conservation. Biological Conservation 136: 271–282.
  40. 40. Suckall N, Fraser E, Quinn C (2009) How class shapes perceptions of nature: Implications for managing visitor perceptions in upland UK. In: Bonn A, Hubacek K, Allott T, Stewart J, editors. Drivers of environmental change in uplands: Routledge. pp. 393–403.
  41. 41. Goncalves B, Marques A, Soares A, Pereira HM (2015) Biodiversity offsets: from current challenges to harmonized metrics. Current Opinion in Environmental Sustainability 14: 61–67.
  42. 42. Mooney HA, Duraiappah A, Larigauderie A (2013) Evolution of natural and social science interactions in global change research programs. Proceedings of the National Academy of Sciences of the United States of America 110: 3665–3672. pmid:23297237
  43. 43. Suter GW (1993) A critique of ecosystem health concepts and indexes. Environmental Toxicology and Chemistry 12: 1533–1539.
  44. 44. UNEP (2014) Developing indicators for national targets as part of NBSAP updating: Examples of the Biodiversity Indicator Development Framework in practice.: United Nations Environment Programme / World Conservation Monitoring Centre. Available:
  45. 45. Maskell LC, Smart SM, Bullock JM, Thompson K, Stevens CJ (2010) Nitrogen deposition causes widespread loss of species richness in British habitats. Global Change Biology 16: 671–679.
  46. 46. Field CD, Dise NB, Payne RJ, Britton AJ, Emmett BA, Helliwell RC, et al. (2014) The role of nitrogen deposition in widespread plant community change across semi-natural habitats. Ecosystems 17: 864–877.
  47. 47. Tong A, Sainsbury P, Craig J (2007) Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care 19: 349–357. pmid:17872937
  48. 48. Marshall MN (1996) Sampling for qualitative research. Family Practice 13: 522–525. pmid:9023528
  49. 49. Verheyen K, Baeten L, De Frenne P, Bernhardt-Romermann M, Brunet J, Cornelis J, et al. (2012) Driving factors behind the eutrophication signal in understorey plant communities of deciduous temperate forests. Journal of Ecology 100: 352–365.
  50. 50. Brock K, Rodriguez G, Rowe EC, Zaragoza L (1995) Reflecciones sobre una experiencia de trabajo rural participativa: problemas y temas de discussion. [Reflections on an experience of participatory rural work: problems and discussion themes.]. Memorias del Segundo Congreso Internacional sobre Sistemas Agropecuarias. Universidad Autonoma del Estado de Mexico.
  51. 51. Ford-Thompson AES, Snell CJ, Saunders G, White PCL (2015) Dimensions of local public attitudes towards invasive species management in protected areas. Wildlife Research 42: 60–74
  52. 52. Scientific Software Development (2013) ATLAS.ti Version 6.2. Scientific Software Development GmbH., Berlin
  53. 53. Strauss A, Corbin J (1998) Basics of qualitative research: Grounded theory procedures and techniques. 2nd edition.: Sage Publications Inc., Newbury Park, CA, USA.
  54. 54. Rodwell JS, editor (1991–2000) British Plant Communities. Volumes 1–5.: Cambridge University Press, Cambridge.
  55. 55. Smart SM (2000) MAVIS: Modular Analysis of Vegetation Information System.: Centre for Ecology and Hydrology, Lancaster, UK. Available:
  56. 56. Magurran AE (1988) Ecological diversity and its measurement. London: Croon Helm Ltd. 179 p.
  57. 57. EEA (2014) European Nature Information System. EEA (European Environment Agency), Copenhagen, Denmark. Available:
  58. 58. NBN (2013) National Biodiversity Network Habitats Dictionary. Available:
  59. 59. JNCC (2007) Assessing conservation status: the UK approach. Second Report by the UK under Article 17 on the implementation of the Habitats Directive from January 2001 to December 2006.: Joint Nature Conservation Committee, Peterborough.
  60. 60. Preston CD, Pearman DA, Dines TD (2002) New Atlas of the British and Irish Flora.: Oxford University Press. 910 p.
  61. 61. Goldsmith FB, editor (1991) Monitoring for conservation and ecology. London: Chapman and Hall. 275 p.
  62. 62. Czekanowski J (1909) Zur differential Diagnose der Neandertalgruppe. Korrespbl dt Ges Anthrop 40: 44–47.
  63. 63. Yoshioka PM (2008) Misidentification of the Bray-Curtis similarity index. Marine Ecology Progress Series 368: 309–310.
  64. 64. Tipping E, Henrys PA, Maskell LC, Smart SM (2013) Nitrogen deposition effects on plant species diversity: threshold loads from field data. Environmental Pollution 179: 218–223. pmid:23688734
  65. 65. JNCC (2004) Common Standards Monitoring Guidance for Lowland Grassland. Version February 2004.
  66. 66. JNCC (2004) Common Standards Monitoring Guidance for Lowland Wetland Habitats. Version February 2004.: 61.
  67. 67. JNCC (2009) Common Standards Monitoring Guidance for Lowland Heathland. Version February 2009.: 47.
  68. 68. JNCC (2006) Common Standards Monitoring Guidance for Upland habitats. Version October 2006.
  69. 69. Stevens CJ, Dise NB, Gowing DJG, Mountford JO (2006) Loss of forb diversity in relation to nitrogen deposition in the UK: regional trends and potential controls. Global Change Biology 12: 1823–1833.
  70. 70. Ellenberg H, Weber HE, Dull R, Wirth V, Werner W, Paulissen D (1992) Zeigerwerte von pflanzen in mitteleuropa: 2nd ed. Scripta Geobotanica 18: 1–258.
  71. 71. Hill MO, Preston CD, Roy DB (2004) PLANTATT: Attributes of British and Irish plants: Status, size, life history, geography and habitats.: Centre for Ecology and Hydrology. Available:
  72. 72. Hill MO, Preston CD, Bosanquet SDS, Roy DB (2007) BRYOATT—attributes of British and Irish mosses, liverworts and hornworts with information on native status, size, life form, life history, geography and habitat.: Centre for Ecology and Hydrology. 88 p.
  73. 73. Hill MO, Carey PD (1997) Prediction of yield in the Rothamsted Park Grass Experiment by Ellenberg indicator values. Journal of Vegetation Science 8: 579–586.
  74. 74. Wagner M, Kahmen A, Schlumprecht H, Audorff V, Perner J, Buchmann N, et al. (2007) Prediction of herbage yield in grassland: How well do Ellenberg N-values perform? Applied Vegetation Science 10: 15–24.
  75. 75. van den Berg LJL, Jones L, Sheppard L, Smart SM, Bobbink R, Dise NB, et al. (in press) Evidence for differential effects of reduced and oxidised nitrogen deposition on vegetation independent of nitrogen load. Environmental Pollution.
  76. 76. Stanton JC, Semmens BX, McKann PC, Will T, Thogmartin WE (2016) Flexible risk metrics for identifying and monitoring conservation-priority species. Ecological Indicators 61: 683–692.
  77. 77. van Strien AJ, Soldaat LL, Gregory RD (2012) Desirable mathematical properties of indicators for biodiversity change. Ecological Indicators 14: 202–208.
  78. 78. Mascia MB, Brosius JP, Dobson TA, Forbes BC, Horowitz L, McKean MA, et al. (2003) Conservation and the social sciences. Conservation Biology 17: 649–650.
  79. 79. Bowers K, Boutin C (2008) Evaluating the relationship between floristic quality and measures of plant biodiversity along stream bank habitats. Ecological Indicators 8: 466–475.
  80. 80. Medeiros HR, Bochio GM, Ribeiro MC, Torezan JM, dos Anjos L (2015) Combining plant and bird data increases the accuracy of an Index of Biotic Integrity to assess conservation levels of tropical forest fragments. Journal for Nature Conservation 25: 1–7.
  81. 81. Wallis de Vries MF, Van Swaay CAM (2006) Global warming and excess nitrogen may induce butterfly decline by microclimatic cooling. Global Change Biology 12: 1620–1626.
  82. 82. Lewandowski AS, Noss RF, Parsons DR (2010) The Effectiveness of Surrogate Taxa for the Representation of Biodiversity. Conservation Biology 24: 1367–1377. pmid:20455907
  83. 83. Fleishman E, Noss RF, Noon BR (2006) Utility and limitations of species richness metrics for conservation planning. Ecological Indicators 6: 543–553.
  84. 84. Tucker CM, Cadotte MW, Davies TJ, Rebelo TG (2012) Incorporating Geographical and Evolutionary Rarity into Conservation Prioritization. Conservation Biology 26: 593–601. pmid:22624595
  85. 85. Arponen A, Heikkinen RK, Thomas CD, Moilanen A (2005) The value of biodiversity in reserve selection: Representation, species weighting, and benefit functions. Conservation Biology 19: 2009–2014.
  86. 86. Landi S, Chiarucci A (2010) Is floristic quality assessment reliable in human-managed ecosystems? Systematics and Biodiversity 8: 269–280.
  87. 87. Carignan V, Villard MA (2002) Selecting indicator species to monitor ecological integrity: A review. Environmental Monitoring and Assessment 78: 45–61. pmid:12197640
  88. 88. Siddig AAH, Ellison AM, Ochs A, Villar-Leeman C, Lau MK (2016) How do ecologists select and use indicator species to monitor ecological change? Insights from 14 years of publication in Ecological Indicators. Ecological Indicators 60: 223–230.
  89. 89. Munoz JM, Alcantara J (2013) The value of indicator species for landscape characterization. Plant Biosystems 147: 418–428.
  90. 90. Hayes M, Boyle P, Moran J, Gormally M (2015) Assessing the biodiversity value of wet grasslands: can selected plant and insect taxa be used as rapid indicators of species richness at a local scale? Biodiversity and Conservation 24: 2535–2549.
  91. 91. JNCC (2004) Common Standards Monitoring: Introduction to the guidance manual.
  92. 92. Trindade J, Loyola RD (2011) Performance and Consistency of Indicator Groups in Two Biodiversity Hotspots. PLoS ONE 6.
  93. 93. ICPMM (2014) Final Chair's report of 24th CCE Workshop and the 30th meeting of the Programme Task Force, 7th-10th April 2014 in Rome, Italy.: International Cooperative Programme on Modelling and Mapping of Critical Loads and Levels and Air Pollution Effects, Risks and Trends. Available: 40 p.
  94. 94. Rowe EC, Ford-Thompson A, Monteith D, van Hinsberg A, Smart S, Henrys P, et al. (2013) A biodiversity metric for interpreting outputs of models of atmospheric nitrogen pollution impacts on habitats.: Final report on Defra project AQ0828, “Research to define a biodiversity metric to inform the UK’s response to the CCE ‘Call for Data 2012–2014’. CEH project NEC04988. Centre for Ecology and Hydrology, ECW, Deiniol Road, Bangor, LL57 2UW, UK. Available: 90 p.