Multi-stakeholder environmental management and governance processes are essential to realize social and ecological outcomes. Participation, collaboration, and learning are emphasized in these processes; to gain insights into how they influence stakeholders’ evaluations of outcomes in relation to management and governance interventions we use a path analysis approach to examine their relationships in individuals in four UNESCO Biosphere Reserves. We confirm a model showing that participation in more activities leads to greater ratings of process, and in turn, better evaluations of outcomes. We show the effects of participation in activities on evaluation of outcomes appear to be driven by learning more than collaboration. Original insights are offered as to how the evaluations of outcomes by stakeholders are shaped by their participation in activities and their experiences in management and governance processes. Understanding stakeholder perceptions about the processes in which they are involved and their evaluation of outcomes is imperative, and influences current and future levels of engagement. As such, the evaluation of outcomes themselves are an important tangible product from initiatives. Our research contributes to a future research agenda aimed at better understanding these pathways and their implications for engagement in stewardship and ultimately social and ecological outcomes, and to developing recommendations for practitioners engaged in environmental management and governance.
Citation: Plummer R, Dzyundzyak A, Baird J, Bodin Ö, Armitage D, Schultz L (2017) How do environmental governance processes shape evaluation of outcomes by stakeholders? A causal pathways approach. PLoS ONE 12(9): e0185375. https://doi.org/10.1371/journal.pone.0185375
Editor: Andrea Belgrano, Sveriges lantbruksuniversitet, SWEDEN
Received: April 18, 2017; Accepted: September 12, 2017; Published: September 25, 2017
Copyright: © 2017 Plummer et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data necessary to replicate the findings of this study are available within the paper and its Supporting Information files. Additional data is available on request from authors, however, all identifying characteristics (e.g., names of individuals and affiliations) shall be removed to protect the identities of those individuals involved.
Funding: This work was supported by Vetenskapsrådet, the Swedish Research Council (grant 2012-5498; http://www.vr.se/) (RP, DA, OB, LS). Further support for researchers at the Stockholm Resilience Centre (OB, LS) comes from a core grant from Mistra.
Competing interests: The authors have declared that no competing interests exist.
We are confronted with the challenge of stewardship in the Anthropocene where humans are a primary driver of environmental change [1,2]. In this era, stakeholder participation, collaboration and learning are important features of management and governance approaches needed to enhance social and ecological outcomes [3,4,5]. Yet, to the best of our knowledge it has not been empirically proven that these features lead to enhanced evaluation of outcomes nor are their interactions well understood. Empirically linking management and governance interventions to outcomes is a pressing need in natural resource settings , evidence-based conservation , and environmental decision-making [8,9].
The importance of forging connections among participation, learning and collaboration is reflected in diverse resource management and governance approaches [2, 10–13]. For example, adaptive co-management or adaptive governance approaches emphasize the importance of stakeholder participation in decision making activities, as well as a need to harness the power of collaboration, draw upon diverse knowledge types, and facilitate the institutional flexibility to respond to ecosystem change [12–14]. However, empirically validating the relationships among these features and in combination with evaluating outcomes is complicated and contested. Gaining insights on these variables, their connections and contributions to evaluating outcomes in adaptive and collaborative management processes in a systematic way is an identified knowledge gap and an ongoing need [15–18] to advance scholarly research and to support decision making in environmental stewardship practice. Thus, our research focuses on establishing and understanding the causal relationships among participation, social processes that occur in governance, and evaluations of outcomes, using the individual stakeholder perspective.
While greater stakeholder participation is not appropriate under all circumstances [19,20], arguments persist that participation is linked to advancing deliberative decision making, increasing efficiency, and enhancing effectiveness [20–23]. Improving stakeholder involvement in making decisions about the environment is now embedded in policy discourses across scales to enhance governance [23, 19]. However, claims about stakeholder participation and social and ecological outcomes are often presumed, but rarely proven ([23, 24], see  for an exception). A synthesis of studies that do evaluate participation  (e.g., [25–27]) suggests improvements in quality of decisions are strongly contingent upon the qualities of the associated process.
Multi-party (i.e., collaboration-based) solutions have provided an avenue to overcome impasses and tackle complex problems. Engaging diverse individuals in an interactive and iterative problem-solving process may lead to better decisions, increase implementation and build capacity for the future [3,4]. Extensive research over the past decades has documented a wide range of cases of multi-party collaboration relating to the environment. This research has offered valuable insights into qualities of collaboration which influence success, such as transparency , social capital , willingness to compromise  and leadership [6, 30]. However, meta-analyses and reviews of collaborative environmental governance have revealed a mix of positive and negative outcomes [31, 32].
The shift towards ecosystem-based and adaptive management has stressed the need to learn from large-scale experiments and ‘learn by doing’ [33, 34]. Learning has become a normative goal in environmental management [35, 36] and it is considered an essential element in promoting desirable changes in behaviours in pursuit of sustainability . Social learning, or learning that extends beyond the individual through social interaction to become situated within larger social units , is a widely adopted perspective within this scholarship. Social learning is manifested by collective action: forging learning partnerships, creating learning platforms, and instilling learning ethics . There is very limited empirical evidence linking the promotion or enhancement of social learning to stakeholder participation [23, 36], and assumptions about the effectiveness of social learning in collaborative processes are largely untested .
Understanding the impacts of governance and management is essential for learning and adapting decisions and actions . These impacts can be identified as the results and effects of governance and management , collectively considered ‘outcomes’. Results are the tangible and intangible outputs of governance and management within and beyond the boundaries of the system [28, 39]. Effects are the ecological and social consequences of management and governance, including ecosystem services and enhanced livelihoods [28, 39]. Evaluating outcomes is complicated and contested. Perceptions of stakeholders have been recognized and used as an indirect measure of environmental outcomes and their potential as a source of bias is an identified concern ([40–42], see  for a summary). The potential presence of other variables of influence adds difficulty to correlating environmental outcomes and governance processes  and the potential for confounding variables and attribution issues are similarly present for social outcomes. Bennett  has recently challenged notions of what counts as evidence, clarifies the contributions of perceptions research, and calls for increased incorporation of evidence from social and natural sciences to enhance decisions.
We validate the relationships among participation, collaboration and learning in relation to evaluating outcomes as well as address critical gaps in knowledge about their interrelationships in environmental management and governance. using a path analysis modeling approach. Specifically, we concentrate on how participation and features of learning and collaboration influence evaluations of outcomes by the stakeholders involved. Our approach assumes that individuals’ long-term engagement in an environmental management and governance process is dependent upon their beliefs in the ability of that process to deliver desirable outcomes and that individuals engaged in the range of activities related to environmental management and governance are best-positioned to evaluate outcomes from it. We conducted the research at the individual level, collecting responses from four biosphere reserves, two in Sweden and two in Canada, for the analysis. Biosphere reserves (BRs) represent an appropriate context in which to conduct this research as members engage in governance and management activities in support of the dual aims of biodiversity conservation and sustainable development ; thus, they explicitly aim to achieve ecological and social outcomes.
We investigated three potential causal pathways among participation, process (learning and collaborative qualities) and evaluating outcomes (Fig 1). The first model combines learning and collaboration into a single ‘process’ variable. In the second and third models, learning and collaborative qualities, respectively, were examined for their individual roles in the causal pathway.
The first model (a) test the effect of the process generally (model 1). The process variable is then unpacked and model 2 corresponds to (b), testing the effect of learning and model 3 corresponds to (c), testing the effect of collaborative qualities. Straight arrows represent indirect effects between activities and outcomes, while curved arrows represent direct effects between them.
This section details the methods used for this study, including the cases, respondents, measures and analysis. The instruments used to collect data, additional detail on data preparation and the model data are available in S1–S3 Appendices. Brock University Research Ethics Board approved this research (file number 13–026).
Four BRs were selected for study: Frontenac Arch and Georgian Bay in Canada and Kristianstads Vattenrike and Ӧstra Vätterbranterna in Sweden. These BRs were chosen based on their relative similarity in terms of size, location in high-income countries, and active governance processes. The individual was selected as the unit of analysis, and the decision to focus on these four BRs helped to minimize external variability as much as possible while providing a large enough sample of individuals engaged in management and governance interventions from the cases to conduct the analysis.
BR managers identified individuals in their respective BRs who were engaged in the management and governance process. These individuals were invited to participate in the study by attending a workshop and completing a questionnaire prior to participating in the workshop. Written, informed consent was provided in all cases. The final sample included 66 individuals who provided responses to all sections of the questionnaire. Respondents’ affiliations ranged from local and regional/provincial government to environmental non-governmental organizations to private business and landowners. Respondents engaged in range of activities related to management and governance of the BRs and thus their perceptions reflect their unique experiences in relation to the processes and multiple outcomes measured in this study.
A questionnaire was administered that included questions about the specific activities in which individuals participated, quality of collaboration, learning, and evaluation of outcomes (results and effects) of the governance and management process. The items corresponding to each of these sections are provided in S1 Appendix. For activities, respondents were asked to identify specific activities in which they have participated based on seven categories: preparation of biosphere related materials for UNESCO, practical actions in the landscape, projects that involve monitoring, social events, mapping of the biosphere reserve, activities related to management and planning, and activities related to governance or decision making. The total number of identified activities was calculated for each individual.
In all other sections of the questionnaire respondents were asked to rate their agreement with provided statements on a five-point Likert response scale (1 = strongly disagree and 5 = strongly agree). ‘Collaborative qualities’ items reflected those of established importance in the literature, such as transparency and deliberation [28, 39, 46, 47]. ‘Learning’ was measured using a typology of cognitive, normative and relational learning [39, 48]. ‘Results’ comprise evaluation of tangible and intangible products that immediately (first order) or indirectly (second order) come about from the management and governance process (see  for the original typology, [28, 39]). ‘Effects’ are evaluations of their consequences, consisting of ecological sustainability and enhanced livelihoods [28, 39]. Total scores were obtained for each of the five sections of the questionnaire by summing the responses to items, resulting in summary scores for each variable for each respondent. For descriptive statistics of the means and distributions of responses to each section please see S4 Appendix.
Path analysis is a structural modeling approach that estimates direct and indirect effects (i.e., effects that are mediated by an intervening variable) among a set of variables . Three models were examined using MPlus version 7 for Windows  (Fig 1). Evaluations of outcomes were specified to be the dependent variable in each model. The total number of activities each individual engaged in was used as the predictor at the start of the path analysis, and a dummy variable representing the BRs was used as a covariate to eliminate any group effects, that is, to eliminate any possible impacts membership in a particular BR might have on the analysis. Details regarding the analysis including the correlation matrix and model results are provided in S2 and S3 Appendices. The process variable was used as potential mediating variable in the first model. Thereafter, the process variable was unpacked into its component parts: learning and collaborative qualities. Learning was used as a potential mediating variable in the second model, and collaborative qualities in the third model (Fig 1b and 1c, respectively). Thus, models 2 and 3 were used to explore the relative contributions of learning and collaborative qualities to the relationship between activities and outcomes. The goal of the analysis was to understand the indirect effects (i.e., effects of activities on outcomes through process/learning/collaborative qualities) and direct effects (i.e., effects of activities on outcomes). The significance of indirect effects was examined through 95% confidence intervals (CIs) using maximum likelihood estimators with 1000 bootstrap samples, such that any effects that did not include 0 in the CI were considered to be significant.
The first model examined the relationship between activities, process (i.e., composite of learning and collaborative qualities) and evaluations of outcomes (Fig 2a). Activities and process together explained 56.3% (p < .001) of variance in evaluations of outcomes and activities alone explained 22.6% (p = .002) of variability in process. The direct effects of activities on evaluations of outcomes was not significant but the indirect effect of activities to outcomes through process was, such a higher level of participation was related to higher scores on Process measures and in turn, greater scores for outcomes.
Solid arrows indicate significant effects; dashed arrows represent non-significant relationships. *Significant indirect effect.
In the alternative models examined, the process variable was substituted with either learning (model 2) or collaborative qualities (model 3) (Fig 2b and 2c). Activities explained 19.5% (p = .007) of variance in learning and 41.2% (p < .001) of variance in evaluations of outcomes. Similar to the results of the first model, only the indirect effect of activities on evaluations of outcomes (i.e., through learning) was significant. In the third model, activities explained 23.4% of variance in collaborative qualities (p = .019) and 48.3% of variance in evaluations of outcomes (p < .001). However, neither the direct nor indirect effects were significant. Thus, it appears that effects of activities on evaluations of outcomes is influenced more by the path through learning compared to quality of collaborations. It is worth noting that no group effects related to membership in a particular BR were found in any model.
Evidence to support the importance of a quality management and governance process
Insights derived from the analysis unveil the dynamics ‘on the ground’ as perceived by stakeholders engaged in management and governance in BRs. As captured in the first model (Fig 2a), participation in more activities led to higher rating of process variables and, in turn, more positive evaluation of social and ecological outcomes. Our finding gives insight into previous studies where participation in more management and governance activities is assumed to lead to improved decisions and strongly depends on the quality of the process . This finding regarding the importance on the quality of the process itself is not surprising. However, the question of what constitutes a quality process has given rise to syntheses of principles and attributes in resource and environmental management (e.g., [23, 51–53]), as well as in environmental governance and resilience scholarship (e.g., [12, 54]). Our findings complement these insights from qualitative studies and confirm, empirically, the combination of collaboration and learning as foundational to better perceptions among stakeholders about outcomes from environmental management and governance. We were able to account for a substantial amount of the variance of evaluated outcomes (56.3%) in our first model using the combined ‘process’ variable.
The findings resonate with governance and management efforts more broadly. The BRs can be considered as “early movers and motivators” in addressing the stewardship challenge [2: 7373] and thus provide an excellent context in which to conduct this research. Commenting on management and governance effectiveness oversteps our results, but our findings contribute two valuable insights. First, we broaden the basis for evidence about outcomes beyond a single source, such as a BR manager, to include stakeholders. Second, we are able to enhance knowledge regarding the extent of variance of outcomes explained by participation and process variables. Insights into these approaches almost exclusively come from qualitative case studies and reviews of the literature (e.g., [11, 16]), but see Stoll-Kleeman et al.  and Schultz et al.  for exceptions that revealed a belief by managers about the importance of community participation, especially in association with acceptance of the organization/designation and conservation program success. Although tremendous value is derived from the richness of studying cases, and context is clearly important, several limitations have been observed in the literature to date (e.g., inconsistent conceptualization, imprecise definitions, lack of measurements), that hamper the comparability of research and the ability to draw causal inferences [16, 39]. Our study has directly addressed these limitations. The modeling and statistical analysis employed here offer tools for further application in adaptive governance as well as a broader suite of approaches to environmental collective action.
Learning contributes more than collaboration to perceived outcomes
Deconstructing the first model and examining the effects of learning and collaborative qualities separately (models 2 and 3, Fig 1) showed that participation in more activities contributes to greater learning scores but not to better perceptions of collaborative qualities. Therefore, the effects of activities on evaluating outcomes appear to be driven by learning rather than quality of collaboration. However, since the overall model (Fig 2a) where the process variable included both learning and collaborative qualities is significant, these analyses highlight the relative importance of each process sub-variable (i.e., learning seems more important when explaining effects of activities). These findings enrich the dialogue among scholars about how a group of actors come to address collective action problems when managing linked systems of people and nature. First, the frequency of engagement in activities is important: those who engaged in more activities tended to learn more and provide more positive evaluations of outcomes. Second, and related specifically to the group processes, collaborating in a ‘quality’ process is seen as the means to addressing collective action problems (e.g., [31, 41, 56]). But also, addressing collective action problems is reliant upon learning, and learning comes about through a ‘quality process’ (e.g. [5, 37, 57, 58]). The pathways revealed through our analysis augment these existing causal presumptions and align with recent assertions by Lubell [59, 60] that solving collective-action problems requires complex institutional systems that facilitate key processes of learning, cooperation and distribution of benefits and costs.
Shedding light on conceptual and analytical considerations
Conceptual tensions and analytical complications surround research about approaches to environmental management and governance in relation to outcomes. We elected to use self-reported information by the individuals who participated in the activities of the BRs. Our choice is predicated upon the logic that the individuals involved in the initiative are best positioned to reflect upon the process as well as evaluate the outcomes of interest (see S1 Appendix), and this choice is supported by research that shows that stakeholders with different experiences and knowledge bring different perceptions of the same resources . We statistically control for BR level effects with regards to processes and outcomes so that analysis throughout is at the individual level. Moreover, we utilize a multi-item approach for rating outcomes as opposed to a single-item effectiveness measure, to hedge against inflated perceptions by stakeholders as found by Selin et al. . These decisions in our analytical approach were made to enable a focus on the features influencing evaluation of outcomes by the stakeholders involved.
Utilizing independent measures of environmental outcomes is complementary to our approach and a laudable ideal (see  for a recent example), as is pursuing innovative research designs that present opportunities to limit confounding variables (e.g., ). Enthusiasm for independent measures must be tempered with awareness of the inherent difficulties in understanding and attributing environmental  as well as social impacts. The indirect path in our first model is consistent with findings in collaborative environmental management of positive evaluations of an initiative by active participants . Although a multi-item approach was utilized to enhance the validity of the measures and potentially counter inflation of evaluations, it is not possible to rule out the ‘halo effect’, which occurs when there are high levels of trust among individuals, inflating perceptions of their collective impact . The indirect path is also consistent with the broader phenomena of cognitive dissonance where respondents engaging in more intensive endeavours exaggerate the associated positive outcomes . The direct path from participation to evaluation of outcomes was not statistically significant. This opens interesting opportunities to expand the model and probe additional features which may be exerting influence on the evaluation of outcomes, such as personal attributes or specific institutional and political conditions within which governance occurs. Finally, we utilized one portion of a more extensive dataset in which individuals are nested within BRs. Attaining and utilizing a larger and longitudinal dataset opens a host of possibilities for conceptual and analytical advancement. Specifically, aggregated variables such as outcomes could be tested as separate variables (i.e., results and effects); an understanding of feedbacks could be developed; and, analysis could be conducted at a level (individual, BR) and/or at multiple levels, capturing multi-level interactions.
Implications for practice and policy
Our findings suggest that stakeholder engagement in multiple activities influences their experiences in environmental management and governance and, in turn, their evaluation of the outcomes. It highlights the importance of a positive experience in environmental stewardship efforts for continued engagement  and specifically identifies the critical role of learning . Participation by volunteers in particular has been considered essential to the functioning of environmental stewardship programs [64,66], with learning highlighted as an important motivator for engagement. Accordingly, practices that create opportunities for engagement of stakeholders in multiple activities and for learning to occur are likely to improve outcomes as evaluated by those stakeholders. As Bennett [7: 584] reminds us, “it is positive perceptions, not just objective scientific evidence of effectiveness, that ultimately ensure the support of local constituents thus enabling the long-term success of conservation”.
Our research offers first steps to in a larger effort to identify important practical guidelines that enhance engagement in, and outcomes from, environmental stewardship. However, it is important to note that the relatively small sample size, while certainly adequate for the models tested here would benefit from a large sample to improve both the power of the analysis and the ability to test relationships among more variables. Accordingly, further research to build a nuanced understanding of participation in different kinds of activities, types of learning and specific collaborative qualities that are most influential on perceived outcomes, and how virtuous cycles of engagement are created, is needed. Research of this kind, when undertaken with a very large sample of environmental stewardship stakeholders in a range of contexts, can provide guidance for practitioners in terms of how to encourage participation and nurture a positive management and governance process to enhance social and ecological outcomes.
Participation, collaboration and learning are advanced as important features of approaches to contemporary environmental management and governance. The findings from our research afford evidence that their importance can be empirically determinable in relation to the evaluation of outcomes. Our work to untangle the dynamics and relative contributions of these essential features to evaluation of outcomes advances the understanding and importance of stakeholder perceptions , empirical connections between interventions with outcomes , and inner-workings of social-ecological governance approaches taking up the stewardship challenge . We show here that the scope of inquiry should be extended beyond collaboration [60, 67] to a broader suite of approaches for managing and governing social-ecological systems. Insights about pathways to address collective action problems appear somewhat fragmented in the literature, and further research is required to close this gap and/or discern multiple processes facilitating management and governance arrangements [59, 60]. Efforts to advance our understanding of these pathways will greatly benefit from an expanded acceptance of what constitutes evidence (i.e., perception as we use it in this analysis), as well as more effective use of qualitative and quantitative measures from natural and social sciences . Ultimately, such a research agenda may enable us to empirically connect management and governance interventions to outcomes, and therefore, offer 1) an evidence-based argument for necessary changes [8, 9]; and 2) a richer understanding of the features influencing the stakeholder views that are critical for long-term success .
S2 Appendix. Additional detail on data preparation.
S3 Appendix. Additional detail on data analysis and preparation.
We gratefully acknowledge participation in this research by the managers and others from the Frontenac Arch, Georgian Bay, Kristianstads Vattenrike, and Östra Vätterbranterna Biosphere Reserves. Our appreciation is also extended to our colleagues—Beatrice Crona, for her constructive insights with conceptualizing the research project and Danielle Molnar and Gordon Hodson for their feedback on the analytical approach. Finally, we wish to thank Malena Heinrup, Katrina Krievins, Flor de Luna Estrada, Disa Hansson and Kerrie Pickering for research assistance.
- 1. Steffen W, Persson A, Deutsch L, Zalasiewicz J, Williams M, Richardson K, et al. The Anthropocene: from global change to planetary stewardship. Ambio. 2011 Nov; 40 (7): 739–761. pmid:22338713
- 2. Schultz L, Folke C, Österblom H and Olsson P. Adaptive governance, ecosystem management, and natural capital Proc. Natl Acad. Sci. USA 2015 Jun 16;112 (24): 7369–7374.
- 3. Innes J E and Booher D E. Consensus building and complex adaptive systems: a framework for evaluating collaborative planning. Journal of the American Planning Association. 1999; 65 (4): 412–423.
- 4. Wondolleck J M and Yaffee S L. Making Collaboration Work: Lessons from Innovation in Natural Resource Management. Washington, DC: Island Press; 2000.
- 5. Muro M and Jeffrey P. A critical review of the theory and application of social learning in participatory natural resource management processes. Journal of Environmental Planning and Management. 2008 May 8; 51 (3): 325–344.
- 6. Selin S W, Schuett M A and Carr D. Modeling stakeholder perceptions of collaborative initiative effectiveness. Society & Natural Resources. 2000 Dec;13 (8): 735–745.
- 7. Bennett N J. Using perceptions as evidence to improve conservation and environmental management. Conserv. Biol. 2016 Jun; 30 (3): 582–592. pmid:26801337
- 8. Carpenter S R, Mooney H A, Agard J, Capistrano D, DeFries R S, Diaz S, et al. Science for managing ecosystem services: beyond the Millennium Ecosystem Assessment. Proc. Natl Acad. Sci. USA. 2009 Feb 3; 106 (5): 1305–1312. pmid:19179280
- 9. Guerry A D, Polasky S, Lubchenco J, Chaplin-Kramer R, Daily G C, Griffin R, et al. Natural capital and ecosystem services informing decisions: from promise to practice. Proc. Natl Acad. Sci. USA. 2015 Jun 16; 112 (24): 7348–7355. pmid:26082539
- 10. Berkes F, Colding J and Folke C, editors. Navigating Social-Ecological Systems: Building Resilience for Complexity and Change. Cambridge: Cambridge University Press; 2003. p 1–30.
- 11. Folke C, Hahn T, Olsson P and Norberg J. Adaptive governance of social-ecological systems. Annual Review of Environment and Resources. 2005 Nov 21; 30: 441–473.
- 12. Armitage D, Plummer R, Berkes F, Arthur R I, Charles A T, Davidson-Hint I J, et al. 2009 Adaptive co-management for social-ecological complexity. Frontiers in Ecology and the Environment. 2009 Mar; 7(2): 95–102.
- 13. Biggs R, Schlüter M, Biggs D, Bohensky E L, Burnsilver S, Cundill G, et al. Toward principles for enhancing the resilience of ecosystem services. Annual Review of Environment and Resources. 2012 Aug 7; 37: 421–448.
- 14. Folke C, Carpenter S, Elmqvist T, Gunderson L, Holling C S and Walker B. Resilience and sustainable development: building adaptive capacity in a world of transformations. AMBIO: A Journal of the Human Environment. 2002 Aug; 31(5): 437–440.
- 15. Plummer R. The adaptive co-management process: an initial synthesis of representative models and influential variables. Ecol. Soc. 2009; 14 (2):24.
- 16. Plummer R, Crona B, Armitage D R, Olsson P, Tengö M and Yudina O. Adaptive comanagement: a systematic review and analysis. Ecol. Soc. 2012; 17 (3): 11.
- 17. Butler J R A, Young J C, McMyn I A G, Leyshon B, Graham I M, Walker I et al. Evaluating adaptive co-management as conservation conflict resolution: learning from seals and salmon. J. Environ. Manage. 2015 Sept; 160 (1): 212–225.
- 18. Fabricius C and Currie B. Adaptive Management of Social-Ecological Systems. Allen CR and Garmestani AS, editors. Dordrecht: Springer Science+Business Media; 2015.pp. 147–179.
- 19. Rydin Y and Pennington M. Public participation and local environmental planning: the collective action problem and the potential of social capital. Local Environment. 2000; 5 (2): 153–169.
- 20. Hulbert M and Gupta J. The split ladder of participation: a diagnostic, strategic, and evaluation tool to assess when participation is necessary. Environmental Science & Policy. 2015 Jun; 50: 100–113.
- 21. Schultz L, Duit A and Folke C. Participation, adaptive co-management, and management performance in the world network of biosphere reserves. World Dev. 2011 Apr; 39(4): 662–671.
- 22. Dietz T. Bringing values and deliberation to science communication. Proc. Natl Acad. Sci. USA. 2013 Aug; 110: 14081–14087. pmid:23940350
- 23. Reed M S. Stakeholder participation for environmental management: a literature review. Biological Conservation. 2008 Oct; 141 (10): 2417–2431.
- 24. Brody S D. Measuring the effects of stakeholder participation on the quality of local plans based on the principles of collaborative ecosystem management. Journal of Planning Education and Research. 2003 Jun; 22: 407–419.
- 25. Beierle T C. The quality of stakeholder-based decisions. Risk Anal. 2002 Aug; 22 (4): 739–749. pmid:12224747
- 26. Koontz T M. We finished the plan, so now what? Impacts of collaborative stakeholder participation on land use policy. Policy Studies Journal. 2005 Aug; 33 (3): 459–481.
- 27. Fritsch O and Newig J. Reflexive Governance for Global Public Goods. Brousseau E, Dedeurwaerdere T, and Siebenhüner B, editors. Cambridge: The MIT Press; 2012. pp 181–204.
- 28. Plummer R and Armitage D. A resilience-based framework for evaluating adaptive co-management: linking ecology, economics and society in a complex world. Ecol. Econ. 2007 Feb 15; 61 (1): 62–74.
- 29. Pretty J. Social capital and the collective management of resources Science. 2003 Dec 12; 302 (5652): 1912–1914. pmid:14671287
- 30. Gutiérrez N L, Hilborn R and Defeo O. Leadership, social capital and incentives promote successful fisheries. Nature. 2011 Feb 17; 470: 386–389. pmid:21209616
- 31. Ansell C and Gash A. Collaborative governance in theory and practice. Journal of Public Administration Research and Theory. 2008 Nov;18 (4): 543–571.
- 32. Newig J and Fritsch O. Environmental governance: participatory, multi-level—and effective? Environmental Policy and Governance. 2009 May 26;19(3): 197–214.
- 33. Holling CS, editor. Adaptive Environmental Assessment and Management. Chichester: John Wiley & Sons;1978.
- 34. Walters C J and Holling C S. Large-scale management experiments and learning by doing. Ecology. 1990 Dec; 71(6): 2060–2068.
- 35. Armitage D, Marschke M and Plummer R. Adaptive co-management and the paradox of learning. Glob Environ Change. 2008 Feb; 18(1): 86–98.
- 36. Reed M, Evely A C, Cundill G, Fazey I, Glass J, Laing A, et al. What is social learning? Ecol. Soc.2010; 15(4): r1.
- 37. Keen M, Brown VA and Dyball R. Social Learning in Environmental Management: Towards a Sustainable Future. London: Earthscan; 2005.
- 38. Rodela R, Cundill G and Wals A E. An analysis of the methodological underpinnings of social learning research in natural resource management. Ecol. Econ. 2012 May; 77:16–26.
- 39. Plummer R, Schultz L, Armitage D, Bodin O, Crona J, and Baird J. Developing a diagnostic approach for adaptive co-management and considering its implementation in biosphere reserves. Beijer Discussion Paper Series no. 245. Stockholm: Beijer Institute of Ecological Economics; 2014.
- 40. Lubell M. Collaborative environmental institutions: all talk and no action? J. Policy Anal. Manage. 2004 Jun 9; 23 (3): 549–573.
- 41. Koontz T M and Thomas C W. What do we know and need to know about the environmental outcomes of collaborative management? Public Administration Review. 2006 Nov 9; 66: 111–121.
- 42. Mandarano L A. Evaluating collaborative environmental planning outputs and outcomes: restoring and protecting habitat and the New York-New Jersey harbor estuary program. Journal of Planning Education and Research. 2008 Jan; 27: 456–468.
- 43. Scott T. Does collaboration make any difference? Linking collaborative governance to environmental outcomes J. Policy Anal. Manage. 2015 Apr 8; 34 (3): 537–566.
- 44. Biddle J C and Koontz T M. Goal specificity: a proxy measure for improvements in environmental outcomes in collaborative governance. J. Environ. Manage. 2014 Dec; 145: 268–276. pmid:25083592
- 45. UNESCO 1996 Biosphere Reserves: The Seville Strategy and the Statutory Framework of the World Network. Paris: UNESCO; 2014.
- 46. Sandström A, Crona B and Bodin Ö. Legitimacy in co-management: the impact of preexisting structures, social networks and governance strategies. Environmental Policy and Governance. 2014 Jan/Feb; 24: 60–76.
- 47. Birnbaum S, Bodin Ö and Sandstöm A. Tracing the sources of legitimacy: the impact of deliberation in participatory natural resource management. Policy Sciences. 2015 Sept 11; 48 (4): 443–461.
- 48. Baird J, Plummer R, Haug C and Huitema D. Learning effects of interactive decision-making processes for climate change adaptation. Glob. Environ. Change. 2014 Jul; 27: 51–63.
- 49. Kline R B. Principles and Practice of Structural Equation Modeling. New York: The Guilford Press; 2016.
- 50. Muthén L K and Muthén B O. MPlus User’s Guide 7th Ed. Los Angeles: Muthén & Muthén; 2013.
- 51. Webler T, Tuler S and Krueger R O B. What is a good public participation process? Five perspectives from the public. Environ. Manage. 2001; 27: 435–450. pmid:11148768
- 52. Lockwood M, Davidson J, Curtis A, Stratford E and Griffith R. Governance principles for natural resource management. Society & Natural Resources. 2010; 23 (10): 986–1001.
- 53. Luyet V, Schlaepfer R, Parlange M B and Buttler A. A framework to implement stakeholder participation in environmental projects. J. Environ. Manage. 2012 Nov; 111: 213–219. pmid:22926750
- 54. Lebel L, Anderies J M, Campbell B, Folke C, Hatfield-Dodds S, Hughes T P et al. Governance and the capacity to manage resilience in regional social-ecological systems. Ecol. Soc. 2006; 11(1):19.
- 55. Stoll-Kleemann S, De la Vega-Leinert A C and Schultz L. The role of community participation in the effectiveness of UNESCO Biosphere Reserve management: evidence and reflections from two parallel global surveys. Environmental Conservation. 2010 Jun 2; 37 (3): 227–238.
- 56. Sabatier P A, Focht W, Lubell M, Trachtenberg Z, Vedlitz A and Matlock M. Swimming Upstream: Collaborative Approaches to Watershed Management. Sabatier P A, Focht W, Lubell M, Trachtenberg Z, Vedlitz A, and Matlock M, editors. Cambridge: The MIT Press; 2005. pp 3–21.
- 57. Cundill G and Rodela . A review of assertions about the processes and outcomes of social learning in natural resource management. J. Environ. Manage. 2012 Dec 30; 113: 7–14. pmid:22985710
- 58. Armitage D, Plummer R, Baird J, Dzyundzyak A, Schultz L, and Bodin Ö. Learning and adaptive co-management. Environmental Policy and Governance. In review.
- 59. Lubell M. Governing institutional complexity: the ecology of games framework. Policy Studies Journal. 2013 Aug; 41 (3): 537–559.
- 60. Lubell M. Collaborative partnerships in complex institutional systems. Current Opinion in Environmental Sustainability. 2015 Feb; 12: 41–47.
- 61. Cebrián-Piqueras M A, Karrasch L and Kleyer M. Coupling stakeholder assessments of ecosystems services with biophysical ecosystem properties reveals importance of social contexts. Ecosystem Services. 2017 Feb; 23: 108–115.
- 62. Leach W D and Sabatier P A. Swimming Upstream: Collaborative Approaches to Watershed Management. Sabatier P A, Focht W, Lubell M, Trachtenberg Z, Vedlitz A, and Matlock A, editors. Cambridge: The MIT Press; 2005. pp 233–258
- 63. Coglianese C. The Promise and Performance of Environmental Conflict Resolution. O’Leary R and Bingham LB, editors. Washington: Resources for the Future Press; 2003. pp 69–86.
- 64. Ryan R., Kaplan R. & Grese R. Predicting volunteer commitment in environmental stewardship programmes. Journal of Environmental Planning and Management. 2001 Aug 3; 44: 629–648.
- 65. Göhler D, Cashore B and Blom B. Forests and Rural Development. Pretzsch J, Darr D, Uibrig H and Auch E, editors. Berlin Heidelberg: Springer-Verlag; 2014. pp 333–373.
- 66. Grese R.E., Kaplan R., Ryan R.L. & Buxton J. Psychological benefits of volunteering in stewardship programs. Gobster PH & Hull RB, editors. Restoring nature: Perspectives from the social sciences and humanities. Washington DC: Island Press; 2000.
- 67. McAllister R R and Taylor B M. Partnerships for sustainability governance: a synthesis of key themes. Current Opinion in Environmental Sustainability. 2015 Feb; 12: 86–90.