Identification of differences in health impact modelling of salt reduction

We examined whether specific input data and assumptions explain outcome differences in otherwise comparable health impact assessment models. Seven population health models estimating the impact of salt reduction on morbidity and mortality in western populations were compared on four sets of key features, their underlying assumptions and input data. Next, assumptions and input data were varied one by one in a default approach (the DYNAMO-HIA model) to examine how it influences the estimated health impact. Major differences in outcome were related to the size and shape of the dose-response relation between salt and blood pressure and blood pressure and disease. Modifying the effect sizes in the salt to health association resulted in the largest change in health impact estimates (33% lower), whereas other changes had less influence. Differences in health impact assessment model structure and input data may affect the health impact estimate. Therefore, clearly defined assumptions and transparent reporting for different models is crucial. However, the estimated impact of salt reduction was substantial in all of the models used, emphasizing the need for public health actions.


Introduction
The World Health Organization (WHO) recently estimated that dietary risk factors accounted for 11.3 million deaths and 241.4 million disability-adjusted life years (DALYs) [1], with high salt intake being a major contributor [2]. The WHO has set a target to reduce population salt intake by 30%, aiming at an average of 5 gram per day, by 2025 [3]. Sodium reduction in processed foods, and raising awareness of consumers on salt reduction and monitoring salt consumption in populations and food reformulations are the primary interventions to reduce the level of salt intake [4]. For several countries, the expected health gain (e.g. averted morbidity or a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 DALYs) related to salt reduction have been calculated [5][6][7][8][9]. These studies used different approaches to quantify the health impact of salt reduction. Differences in modelling approaches can affect the estimated number of incident cases of disease averted or the number of deaths postponed. To illustrate, Coxson et al demonstrated that a 3-gram lower salt intake could avert 280,000 deaths in the USA using a dynamic-state transition model that estimated salt reduction on blood pressure and subsequently on mortality, but this number almost doubled (500,000 deaths) when a direct effect on mortality was estimated using relative risks from a post-hoc observational analysis of an randomized controlled trial of sodium reduction [8]. In otherwise comparable quantitative health impact assessment (HIA) models, these estimates may also lead to heterogeneity in estimated health outcomes. For example, Scarborough et al observed that applying the assumptions of the CHD policy model leads to a calculated 8 to 16% of CVD deaths being postponed, while a similar analysis using the PRIME model (previously the DIETRON model) suggested a postponement of 4 to 6% [10]. Such heterogeneities may be due to variation in underlying assumptions on the salt intake to health effect association or due to intrinsic factors of the models. Therefore, insight in the underlying model structures, assumptions and (demographic) input data used is essential to interpret and compare the outcomes of population health models.
The objective of the present study is to gain insight in how differences in various HIA models may result in heterogeneity in health impact estimates. We first identified eight models used to calculate the health impact of salt reduction, and describe their differences. In a following step we used the population health modelling tool DYNAMO-HIA to estimate to what extent the variation in the modelling assumptions and (demographic) input data used in these models affect the outcome of health impact estimates.

Selection of models
We searched PubMed for research papers that calculated the long-term health impact of salt reduction published until August 2013, using 'salt reduction', 'health impact assessment' and 'modelling study' as key words. We limited our search to five models: CHD policy model [8,11]; PRIME model [10]; Proportional Multistate Life- Table (PMLT) [7]; Global burden of disease (GBD) [2] and RIVM-CDM [12].
We also identified three additional salt reduction models that fulfilled the above mentioned criteria, but at the time of our search the results had not yet been published (IMPACT model [13], DYNAMO-HIA model [14] and UK Health Forum model. Since the UK Health Forum Model on salt reduction is not yet published in a peer-reviewed journal, our present analyses concern seven models.

Identification of model features
We identified four key features of the models related to the aim of the study, the characteristics of the quantitative impact assessment model used and the output obtained (Fig 1). In our view the most relevant data needed in the model and assumptions that need to be made are clustered in seven boxes in Fig 1 for each feature. In Table 1 we described for each model which (demographic) input data were used and how assumptions were worked out.

Selection of input data or assumptions for modelling exercise
We selected the DYNAMO-HIA model to calculate the effect of the modification of the (demographic) input data and assumptions on the health impact estimates. A detailed description of the model can be found elsewhere [14,25]. We identified those input data and assumptions within sets of key features that differed between the selected models. Subsequently, we evaluated which input data and assumptions could be modified in the DYNA-MO-HIA approach. The ten selected input data and assumptions are marked bold in Fig 1. Some other assumptions or (demographic) input data differed between the selected models, but those assumptions are too closely related to the model structure or could not be modulated in another modelling setting. For example, a model may provide either dynamic or static projections. Static models have no dimension of time, while dynamic models make it possible to estimate changes over time, and as such take into account competing risks. Such a structure cannot be modified. Therefore such differences were not examined in the present analysis. We also choose not to model the impact of changing any input data or assumptions related to the characteristic "Population salt intake and scenario development".
Varying modifiable input data and assumptions using DYNAMO-HIA The selected assumptions were modified so that new input parameters were obtained. The shape and the source of the dose-response association for salt reduction to blood pressure  were combined into a single input parameter. We also adapted input data, such as the age of the population, disease sources and time frame of the simulation. In an additional simulation we mirrored the CHD policy model approach [11] in DYNAMO-HIA, using the following input 1) age range of the population from 35 to 80 years; 2) relative risks for the salt to SBP relation from a meta-analysis of randomized controlled trials; 3) change of mean blood pressure levels within categories; 4) relative risks for the BP to CVD relation from the Framingham Study and 5) 'other cause mortality' (that is, other causes than the modelled diseases IHD and stroke) depending on current blood pressure levels. We used DYNAMO-HIA version 2.07. In all situations, we simulated the health impact of a 3-gram salt reduction for the Dutch population in a closed cohort. The default situation of the DYNAMO-HIA approach and the alternative simulations are presented in Table 2. The alternative simulations were each compared with the default situation. For each simulation, we report the effect on the incidence of stroke and ischemic heart disease (IHD). We also estimated the effect on the life expectancy (LE) for a 60-year old individual. An overview of relative risks used in the calculations is presented in the supplementary information.

Results
Model features and its underlying assumptions and input data Table 1 shows the assumptions and input data of the main model features for all seven models. With respect to the population salt intake and scenario development, we observed that all models simulated a salt reduction scenario: estimating the effect of an intervention or a fixed target. In the scenarios, no lag time of the intervention scenario was assumed, except for the CHD policy and IMPACT model, and a change in salt intake at population level was estimated, except for RIVM-CDM where individual shifts in salt intake were used. With respect to the general input data of the model, the age range of the population exposed to the intervention also differed. All models estimated the effect of salt reduction on CVD mediated by SBP. GBD and PRIME included diseases other than CVD. Prevalence and incidence data of the diseases at the start of the simulation, necessary as input for the models, were based on countryspecific registries and databases. Concerning the main model structures, we identified four dynamic (CHD policy model, RIVM-CDM, DYNAMO-HIA and PMSLT) and three static models (PRIME, IMPACT and GBD). Salt intake and/or systolic blood pressure (SBP) were divided into categories in the CHD policy model, RIVM-CDM and DYNAMO-HIA, but were considered continuous in the other models. The effect of SBP changes over categories was approached differently between RIVM-CDM and the CHD policy model. In RIVM-CDM, the prevalence of the population within the SBP categories changed. In the CHD policy model the mean SBP within each SBP was decreased within each SBP category. Five HIA models derived the salt-blood pressure relation from the same meta-analysis of RCTs [16] and one (PMSLT) from a meta-analysis of observational studies [19]. The CHD policy approach obtained two dose-response relationships from two types of studies. The first is based on the meta-analysis of randomized controlled trials [16], and a second is based on data from clinical trials [17,18]. The dose-response relation obtained from the similar publication of He and MacGregor [16] could be interpreted as a separate linear dose-response relations for normotensive and hypertensive subjects (CHD Policy model, GBD, PRIME and IMPACT), or could be constructed in an exponential dose-response relationship that depended on blood pressure (PMLST, RIVM-CDM and DYNAMO-HIA). The association between blood pressure and CVD was derived in most models (RIVM-CDM, PMSLT, DYNAMO-HIA, PRIME and GBD) from the Prospective Studies Collaboration [23]. RIVM-CDM, PMLST and DYNA-MO-HIA adjusted the variance of the measured SBP levels of the population for the withinsubject variability (attenuation correction). PRIME and GBD used only the population-average SBP levels and not its variance. The CHD policy model and the IMPACT model obtained the effect size of the SBP to CVD morbidity relationship from specific cohort studies, namely the Framingham Cohort Study and the INTERHEART Study [22] [24].
A combination of disease incidence and mortality and integrative measures (such as DALYs) are mostly reported as outcome measure. The period of simulation varies between 10 years (CHD policy model and IMPACT model) to lifetime (PMSLT). Effect of the modifications of input data or assumptions on estimated health impact Table 3 shows the various estimates of the impact of a 3-gram salt reduction (using the DYNA-MO-HIA model) for the population aged 18 years and older in the Netherlands applying six alternative assumptions and four variations of input data. In the default situation, after 10 years, a 3 gram salt reduction resulted in a 10.5% reduction (N = 30,800) in stroke incidence Table 3. Effect of eight modifiable assumptions and input data on the health impact estimate of a 3 gram salt reduction using the DYNAMO-HIA model. and a 7.9% reduction in IHD incidence (N = 38,100). The gain in LE was 0.16 years for men and 0.14 years for women (Fig 2). The largest changes in health impact estimates were observed when risk estimates in the dose-responses association were changed. In the simulation using risk estimates for the relation between blood pressure and diseases from the Framingham Cohort Study, the absolute numbers as well as the percentage reduction of the estimated disease incidence was considerably lower compared with the default situation (7.5% (N = 22,000) reduction for stroke and 4.8% (N = 23,400) for IHD; Table 3). This means a 33% lower estimate for stroke and a 40% lower estimate for IHD compared with the default situation. The health impact estimates were higher if the linear association between salt intake and blood pressure taken from the study of Law were incorporated in the model (lower estimate of 12.5% (N = 36,500) for stroke and 9.6% (N = 46,300) for IHD). This means a 19% higher estimate for stroke and a 22% higher estimate for IHD compared to the default situation.

CVA incidence IHD incidence
Other substantial differences with the default simulation occurred when the prevalence of the population in each SBP categories shifted as a consequence of salt reduction (31% lower estimate for stroke and 32% lower estimate for IHD as compared to the default situation; see Table 3). Other changes have a small impact, such as no correction for usual blood pressure. Extending the calculations to 20 or to 59 years does not have an effect on percentage change, but the absolute number of incident cases is much higher when the calculations are extended to 20 or to 50 years. The combined approach with modifications similar to the CHD policy model led to a stroke reduction of 7.8% and to an IHD reduction of 5.0%. This means a 26% lower estimate for stroke and a 37% decreased estimate for IHD compared to the default situation.
Gain in life expectancy (Fig 2) followed a similar trend as the results in Table 3.

Discussion
Our overview of selected HIA models of salt reduction showed that despite the many differences between the models, there are important similarities. All studies examined the effect of salt intake on CVD as mediated by SBP, with substantial projected health gains (7.2% to 12.7% for stroke and 4.8% to 9.6% for IHD). Differences in assumptions between HIA models mainly concerned the strength of the relationships between salt intake and SBP, and between SBP and disease occurrence. In addition, we observed that an association obtained from literature could be interpreted differently in the modelling exercise. The models also differed in intrinsic model structures, such as categorization of salt intake and/or blood pressure levels and dynamic versus static approach.
In this study, we assessed to what extent model input data and assumptions may determine health impact estimates using a standard dynamic model (DYNAMO-HIA). In the default scenario, a 3 gram salt reduction reduced the incidence of stroke by 10.5% and the incidence of IHD by 7.9%, in the Dutch population over a period of 10 years. Changing the assumptions relating to the association between salt intake and blood pressure and between blood pressure and CVD changed the health impact estimates substantially. Changing the relative risks of blood pressure on CVD reduced the incidence by 33% for stroke and 40% for IHD. After this, using blood pressure in categories, and allowing salt intake change the proportion of the population in each category (reduction of HIA estimate by 27% for stroke and by 18% for IHD) appeared to have the most effect on the incident cases. Effects of changes in the input data had less effect; however, extending the time frame of the calculations had a large impact on the absolute number of incident cases. This is the first study that systematically compared various indirect and complex health modelling approaches for salt reduction based on four sets of key model features and their underlying assumptions and input data. Some limitations of the study need to be addressed. First, due to the selection of predefined key elements other potential differences, such as distinctions in subgroups, have not been taken into account. Second, we selected the DYNAMO-HIA model to quantify only for the Dutch population. The estimated differences in the alternative simulations may vary if this exercise is replicated in other models or in other populations. Finally, we only varied a limited set of input data and assumptions in DYNAMO-HIA model, and thus we cannot quantify the impact of remaining differences, such as the allowance of competing risks or using a static modelling approach. The potential difference between dynamic and static models was not assessed, as it was considered an un-adjustable, intrinsic aspect of the HIA models. However, we do assume that dynamic models are more adequate to estimate future health gain as they take into account selective mortality, ageing and competing risks.
In the present study, the health impact estimates changed when input data and assumptions were replaced by alternative input data or assumptions. Three assumptions or alternative input data are the most influential on the relative and absolute outcome of the health impact assessment: the sources of relative risks used in the blood pressure to health association, the dose-response between salt intake and blood pressure and the distribution of risk factors. Because of the importance of the effect of the relative risk on the outcome, it is important that the source is obtained from good quality, prospective studies. This study showed that using categorical risk factor distributions seemed to reduce the sensitivity of the model to changes in salt intake. This is probably due to the fact that this modelling approach lowers blood pressure in all subjects lowering salt intake will decrease blood pressure in all subjects, but only a few subjects will shift to a lower blood pressure category and thereby will have a lower risk of developing CVD.
In general, uncertainty analyses show how the health impact estimate depends on the underlying assumptions and (demographic) input data within a single HIA model and is therefore helpful to identify the range of the expected effect. However, uncertainty analyses are often only applied to a limited set of model assumptions or (demographic) input data, for example the relative risks (parametric uncertainty). The present study showed that also intrinsic model structures and (demographic) input data contribute to the variation in the health impact estimates (structural uncertainty), but is rarely assessed in modelling studies. Therefore, there is a clear need for transparency in HIA models, if necessary in a technical appendix, where transparency refers to the clear description of the model structures and (demographic) input data used, and also to describe the full range of uncertainty assessed by models. Our analysis showed that it is important to develop standard reporting guidelines for the field of non-communicable disease scenario modelling.
Comparing the variation in outcomes from substituting input data or assumptions in a model one by one was informative to identify the main assumptions that could contribute to the heterogeneity in the outcomes of published studies. In practice, HIA models vary in several underlying assumptions. Mirroring our approach to the approach used by the CHD policy model resulted in an impact estimate higher than the estimate of using the CHD policy model itself (for example, stroke 7.8% for DYNAMO-HIA vs 5.2% of CHD policy model). Thus, by making approaches comparable we could not fully explain the variation between the models. Obviously, there will be some remaining differences, such as the difference in demographic and socio-economic data (such as country-specific incidence and prevalence of CVD). Therefore, a comparative study of the various models using similar input data (demographic as well as intervention scenario) could help to understand how the impact assessment differs between the various models, taking into account the mutual differences between the models.
One aspect of interest since the completion of this analysis is the growing evidence of social inequalities in salt consumption as the more disadvantaged social groups not only have the highest burden of cardiovascular disease, but also the highest salt consumption [26][27][28][29]. A reduction in salt intake would therefore be likely to exert a greater health impact in those groups. Also, newer studies have been published that estimated the health impact of salt reduction (e.g. [30][31][32][33], using similar approaches. Therefore, we do not think that inclusion of these papers will change our conclusions. In conclusion, our study shows that especially differences in the strength and shape of the dose-response association from salt to health contributed to heterogeneity of the health impact estimates reported. We concluded that transparency of the models structures and (demographic) data used is essential to be able to interpret the outcomes of a health impact assessment. We advise to develop standard reporting guidelines for the field of non-communicable disease scenario modelling.