Decision framework.

Decision frameworks are required to characterise the interaction between the people, processes and systems for decision-making about a complex asset. Decision frameworks existing in multiple settings and can be described as the principles, processes, and practices to proceed from information and desires to choices that inform actions and outcomes [31]. The visualisation of a comprehensive decision framework in Fig 2, developed by the authors, draws on this literature, as well as our view of best-practice from our experience in asset-intensive industries. This is the theoretical framework through which the RRAP complications were proposed to be addressed. Each of the core elements of this decision framework are described in more detail below. The definition used in this scope for a decision framework is:

“Decision Framework: The architecture of people, processes and systems used to make decisions”.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Visualisation of a comprehensive decision framework to be applied to the RRAP objectives and decision space.

https://doi.org/10.1371/journal.pone.0240460.g002

Decision hierarchy.

For RRAP we first established the decision hierarchy which is a key element of a coherent decision framework in large, complex assets. They are used in the early phase of large developments and in the management of operating large assets to characterise and organise problems and opportunities according to the potential value proposition of successful solutions and the complexity and uncertainty that impacts identification and selection of potential solutions. Decision hierarchies are used in asset management contexts [36], business case development [37], software development for decision-makers [38], and in theory of decision-making [39]. The definition of decision hierarchy used by the authors draws on this literature, as well as our view of best-practice definition from our experience in asset-intensive industries, and is defined as below:

“Decision Hierarchy: The organisation of the decisions that need to be made, supported by who makes those decisions and how they relate to objectives, value drivers and boundaries across the decision landscape”.

Objectives hierarchy.

The setting of a coherent objectives hierarchy is critical to the success of any project or program that would benefit from implementing a decision framework. They are used in sustainability contexts [40], in contexts where asset developers are trying to balance financial and broader organisational aspirations [41], where public agencies are seeking to develop social infrastructure [42], and numerous applications are presented in the earliest decision science literature [25]. The definition of objectives hierarchy used by the authors draws on this literature, as well as our view of best-practice definition from our experience in asset-intensive industries, and is defined as below:

“Objectives Hierarchy: The arrangement of organisational, program and / or project objectives into relevant hierarchical levels”.

The objectives hierarchy should be derived directly from the strategy for the program as set by the appropriate governance body. The decision support experts serve the dual role of providing subject matter expertise to support relevant processes and activities that inform decision making, and, of providing assurance for the governance group ensuring there is an effective link between decision-making and strategy.

Decision processes.

The decision processes used by the program are instrumental to successful alignment of stakeholders and thus to executing quality decisions. The importance of effective business processes is well understood in management circles [43] and in the decision sciences [19,28,44]. Clearly understanding and planning decision processes, as well as identifying the relevant processes for each decision in the decision hierarchy will maximise the chance that decision-making is smooth and aligned. The definition of decision processes used by the authors is as below:

“Decision Processes: The processes by which decisions are framed, choices are identified, developed and logically analysed for their consequences and trade-offs, and commitments to action are made, across the decision landscape”.

Stakeholder engagement.

The importance of stakeholder engagement in a decision framework is paramount as alignment among the relevant people, using the right processes and tools, and the right knowledge, is necessary for quality decisions. A key part to this is ensuring the relevant stakeholders are prepared to undertake the discussions required to commit to action. There is strong evidence throughout decision science and management literature that stakeholder engagement and participation is key to decision-making success [45].

Knowledge base, management information systems and decision support systems.

Decision processes require an appropriate knowledge base upon which to make the decisions. This includes accessing the right modelling, data and information, as well as the use of relevant management information systems and decision support systems where appropriate. Ensuring that the best possible knowledge is accessible to decision-makers is an important aspect in the development and management of large asset bases [12,13,46,47] and is described variously in dedicated journal publications (e.g., Decision Support Systems Journal). The availability of knowledge through management information systems and the utility of decision support systems are characterised widely in operations research, decision science and information management literature [48–52]. The definition of knowledge base, management information system and decision support system used by the authors are as below:

“Knowledge Base: The data, modelling and information available pertaining to the asset, including understanding of provenance and uncertainty, used to inform decision-making”.

“Management Information System: The computerised system that gathers data from multiple sources and makes it available to users (including synthesis) to support quality decision-making”.

“Decision Support System: The computerised system that gathers data from identified sources, synthesises it, and makes it available to users in accordance with specified decision processes to support quality decision-making on specific semi-structured and unstructured decision problems”.

Quality decisions.

The comprehensive decision framework is brought together by the requirements for quality decisions, which include effectiveness, transparency, defensibility, consistency, efficiency and an understanding of certainty. Each of these requirements maps to a function within the decision framework for research or development programs as shown below in Table 1; for example, effectiveness maps to the objectives hierarchy through outcomes, and consistency maps to the decision processes through process. These requirements are described variously in theoretical texts for decision sciences [53,54].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Requirements of a comprehensive decision framework for a research or development program, including how they map to key decision framework functions.

https://doi.org/10.1371/journal.pone.0240460.t001

Value of information and work breakdown.

There are two decision framework functions that serve as key planning interfaces with the wider research and development program in which it sits. The decision framework requirement for efficiency is made functional through value of information (VoI) method that will help with prioritising where effort could be spent in the rest of the program to decrease uncertainty in the knowledge base–this method is detailed later in this paper. The decision framework requirement for certainty is made functional through a work breakdown of the rest of the program that focuses effort on decreasing uncertainty in the knowledge base as prioritised by the decision framework. As such, a comprehensive decision framework should help the program managers to effectively prioritise effort to efficiently increase certainty to inform higher quality decisions.

The cost-benefit analysis method.

This section summarises the cost-benefit analysis (CBA) method, with a more detailed method available in Reef Restoration and Adaptation Program: Cost-Benefit Analysis [56].

CBA was identified as a potentially relevant analysis method to establish and communicate the value of intervention. CBA attempts to systematically estimate and compare the total benefits and costs of a proposal by calculating the dollar value of the gains and losses for all people affected. CBA is typically used by governments (including the Australian Government who is the ultimate target funder of the RRAP) as (1) it provides decision makers with quantitative and qualitative information about the likely effects of a proposal, (2) encourages decision-makers to take account of all the positive and negative effects of a proposal, and discourages them from making decisions based only on the impacts on a single group, (3) is typically expressed in a single unit ($) which promotes comparability and assists in the assessment of relative priorities across proposals and (4) captures linkages between the proposal and other sectors of the economy, especially when proposals are large in scale such that wider economic effects are material.

The CBA method that was proposed for the scope of work involved producing an economic model for assessing the benefits of environmental and social protection against the costs of the restoration and adaptation required to do so. The method is based on those presented in The Economics of Groundwater Remediation and Protection [57], and Environmental and Economic Sustainability [47], and is aligned to standard cost-benefit analysis processes used by governments globally [58–60]. The approach describes and measures the case for investment in economic terms, by explicitly monetising costs and benefits to society to the extent possible and appropriate, and adding these to private costs and benefits of a proposed project or action.

In economics, the overall objective of any decision is assumed to be the maximisation of human welfare over time. To compare the different benefit and cost streams over time, the process of discounting is used and amounts over time are expressed as “present values”. Economic analysis recommends the decision with the maximum net present value (NPV present value of net benefits, or benefits minus costs, over time) or the highest benefit cost ratio (ratio of the present value of benefits to the present value of costs).

In practice, only some of the benefits identified can be readily quantified and monetised, but those considered in the cost-benefit analysis must be sufficient to determine the case for investment. Those monetised are likely be several of the key private benefits (such as benefits to industry, benefits to livelihoods). External benefits (such as benefits to communities, benefits to ecosystems) are less readily monetised as there is often no market data that could be directly used for their estimation.

The value of information method.

This section summarises the value of information (VoI) method, with a more detailed method available in Appendix C of Reef Restoration and Adaptation Program: Cost-Benefit Analysis [56].

Value of information (VoI) is a field of decision analysis used to estimate the value of acquiring additional data to reduce decision uncertainty. The origins of VoI lie in the work carried out by Raiffa and Schlaifer on applied statistical decision theory at Harvard University in 1961 [61]. It has had many applications in different decision-making domains including, medical, economic, environmental, agricultural and ecological research [62–64]. Whereas basic decision analysis enables decision-makers to identify the best course of action when faced with a situation of uncertainty, VoI provides guidance on how decision-makers might invest in reducing that uncertainty before selecting a course of action [62].

The starting point is some objective function to be maximised, and a choice between courses of action leading to uncertain outcomes with respect to the objective function [63]. In the case of RRAP the objective function is the intervention strategy, the course of action is implementation with or without additional research into these interventions, and the outcome is the resulting cost and efficacy of the intervention.

VoI can be described as the increase in expected value (and certainty) in the outcome of a course of action with the benefit of additional information relating to this action, compared to a course of action without the benefit of the same information. Alternatively, positive VoI occurs when the value in the outcome of a decision benefiting from additional information, minus the cost of obtaining that information, exceeds the value in the outcome of the same decision without the information.

On this basis, VoI can be used as a quantitative method to estimate the return on investment of carrying out additional research into an objective function (intervention), through some form of data collection exercise, with a view on increasing expected value (and certainty) in the outcome of the course of action chosen (implementation). The value of effort to increase certainty in current knowledge (e.g., research, expert elicitation, technology trials, field surveys, etc) can be assessed through this methodology and is referred to as pre-posterior analysis [62,63]. A common way to reduce uncertainty is to gain new information, however, not all uncertainty is equally important to reduce. The most important uncertainties are those that, when reduced, will enable a more effective solution and potentially lower cost and higher benefit. New information is of little value if it does not change the efficacy or cost of the proposed action in the absence of said new information [64].

VOI was identified as a potentially relevant analysis method to establish and communicate the value of research and development given (1) level of uncertainty in RRAP, (2) availability of the proposed Program budget and potential Program benefits from the CBA to inform a VOI analysis and (3) stated Program desire to prioritise effort in reducing uncertainty in the following stage of RRAP.

Key insights from the CBA.

The CBA was effective in being able to establish that RRAP is an investable proposition. By taking into account the balance of costs and benefits, it was shown that there is significant potential economic upside from RRAP interventions of up to AUD 4.1B net present value (NPV) (2016 dollars, 3.5% discount rate) under base case assumptions, which is equivalent to AUD 28B undiscounted over 60 years. Taking a 10% exceedance probability for 1,000 iterations of sensitivity parameters, the potential upside net benefit from RRAP was shown to be up to AUD 14.5B NPV (2016, 3.5%). Thus, RRAP is likely an investable proposition for further R&D more often than not across a broad range of conditions and potential uncertainties. For more detailed presentation and discussion of these findings, please refer to the reports at the top of this section.

From a cost-benefit perspective, restoration and adaptation intervention is a valid new management strategy for the GBR and further R&D should be invested in. The CBA for RRAP showed that, within the high degree of uncertainty inherent in the Program, there are a strong set of conditions where active restoration and adaptation interventions are likely to deliver positive net benefits. As a result, it was recommended that the Program receive investment to progress to the next stage of R&D.

Key insights and recommendations for using CBA as a decision method for RRAP.

The high degree of uncertainty within RRAP at many levels, at least one order of magnitude greater than what would typically be observed during a R&D program concept feasibility phase, means that this CBA should be considered an early attempt to examine the performance of potential program elements. That is, the assessment considered the high degrees of uncertainty within RRAP and examined, within this range, the potential futures for RRAP performance. It did not assume that engineering and scientific action will be taken that result in poor performance; it instead flags where further research will need to be undertaken to reduce uncertainty, better define potential performance, and thus ensure that only well-performing program elements are progressed.

The CBA process should be iterated as uncertainty decreases. It is one tool within RRAP that can be used to ensure only well performing program elements are advanced to the next stage. This informed high-level implications for the next stages of RRAP investment. However, it must be noted that the high degree of uncertainty within the source data means they are necessarily high-level. The SDM process that provided the framework for the CBA, however, is repeatable in subsequent RRAP stages to inform intervention selection and ensure optimised implementation of the Program.

Key insights from the value of information analysis.

The VoI analysis was able to establish that there is a strong case for the investment in at least one phase of R&D, despite the conservative assumptions made in recognition of the significant uncertainty in the Program. The major insights that were established through the VoI analysis were:

• Doing nothing (no R&D or implementation) is an option, but not one recommended by RRAP—the CBA established that action on the GBR has significant potential upside of between AUD 4.1B and AUD 144B net present value (NPV) (2016, 3.5 percent discount rate), and conducting the R&D phase of the Program inherently retains this as an option value.
• Attempting to deploy interventions without an R&D phase is not recommended—the limited number of interventions that might be possible (i.e., technically feasible and able to gain regulatory approval without R&D) are small-scale, likely to be expensive per unit area, poorly guided, with high ensuing ecological risk and with no guarantee they would deliver net benefits, so R&D is essential to improve these factors.
• Investing AUD 100m in an initial phase of R&D is highly likely to be worthwhile–only a 10 percent improvement in the certainty of a small-scale intervention implementation being successful is required in order to justify the costs of the phase of R&D.
• Securing an additional AUD 50m for an initial phase of R&D is highly likely to be worthwhile–if success certainty increases by more than an additional 1.8 percent, the additional investment is worthwhile.
• If implementation does not proceed after five years’ R&D, there remain positive benefits of the investment in R&D–the benefits include expenditure multiplier, and research and knowledge benefits in the Australian economy. Quantification of the expenditure multiplier demonstrates that these benefits alone exceeds the cost of investment under reasonable assumptions.

Key insights and recommendations for using value of information as a decision method for RRAP.

The successful application of VoI to establish and communicate the value proposition of investment in R&D provides a template for guiding both later phases of R&D investment, and, prioritising investment in program areas to resolve uncertainty during each phase of R&D. For the former, while a high likelihood of success of small-scale intervention after a second round of R&D is required to deem the second round R&D investment worthwhile, this decision point is four years away, allowing time for better data and information to generate more accurate insights prior to decision-making. For the latter, the ability of this approach to characterise the impact of uncertainty resolution on the ultimate net benefits of the program, enables prioritisation on uncertainty reducing activities in the program, and screening of those interventions where the cost of uncertainty reduction is incommensurate with the potential benefits.