Unhealthy diet and low levels of physical activity are common behavioural factors in the aetiology of many non-communicable diseases. Recent years have witnessed an upsurge of policy and research interest in the use of taxes and other economic instruments to improve population health.
To assemble, configure and analyse empirical research studies available to inform the public health case for using economic instruments to promote dietary and physical activity behaviour change.
We conducted a systematic scoping review of evidence for the effects of specific interventions to change, or general exposure to variations in, prices or income on dietary and physical activity behaviours and corollary outcomes. Systematic electronic searches and parallel snowball searches retrieved >1 million study records. Text mining technologies were used to prioritise title-abstract records for screening. Eligible studies were selected, classified and analysed in terms of key characteristics and principal findings, using a narrative, configuring synthesis focused on implications for policy and further research.
We identified 880 eligible studies, including 192 intervention studies and 768 studies that incorporated evidence for prices or income as correlates or determinants of target outcomes. Current evidence for the effects of economic instruments and exposures on diet and physical activity is limited in quality and equivocal in terms of its policy implications. Direct evidence for the effects of economic instruments is heavily skewed towards impacts on diet, with a relative lack of evidence for impacts on physical activity.
The evidence-based case for using economic instruments to promote dietary and physical activity behaviour change may be less compelling than some proponents have claimed. Future research should include measurement of people’s actual behavioural responses using study designs capable of generating reliable causal inferences regarding intervention effects. Policy implementation needs to be carefully aligned with evaluation planning and design.
Citation: Shemilt I, Hollands GJ, Marteau TM, Nakamura R, Jebb SA, Kelly MP, et al. (2013) Economic Instruments for Population Diet and Physical Activity Behaviour Change: A Systematic Scoping Review. PLoS ONE 8(9): e75070. https://doi.org/10.1371/journal.pone.0075070
Editor: Daniel Tomé, Paris Institute of Technology for Life, Food and Environmental Sciences, France
Received: December 14, 2012; Accepted: August 11, 2013; Published: September 24, 2013
Copyright: © 2013 Shemilt et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was supported by a grant from the UK Department of Health Policy Research Programme (107/0001- Policy Research Unit in Behaviour and Health). This support is gratefully acknowledged. The views expressed are those of the authors and not necessarily those of the UK Department of Health. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
In 2008, non-communicable diseases (NCDs) caused 36 million (63%) of global deaths and 2.8 million people died as a consequence of being overweight or obese . A large proportion of NCD deaths occur prematurely , imposing large and avoidable costs in human, social and economic terms [2,3]. Alongside smoking and harmful use of alcohol, unhealthy diets and low levels of physical activity are common behavioural factors in the aetiology of the most prevalent and preventable NCDs .
The term ‘economic instruments’ encompasses fiscal or legislative government policies designed to change the relative prices of goods or services or people’s disposable income, and promotional practices used by retailers to change the relative prices of goods and services. Recent years have witnessed an upsurge of policy and research interest in the potential use of economic instruments to shape markets for specific goods and services in ways that incentivise healthier behaviours [5-8]. This can be seen as one dimension of a broader policy context in which national and local governments aspire to influence population health behaviours by altering the environments within which people make choices [9-11].
Extant research evidence largely corroborates the public health case for using economic instruments to discourage purchasing and use of tobacco and alcohol products [12-16]. However, the corresponding case for the use of these instruments to encourage healthier eating and physical activity remains controversial from both an evidence and a policy perspective. We therefore conducted a systematic scoping review to configure emprical evidence that is currently available to inform the case for or against the use of economic instruments to promote health-enhancing dietary and physical activity behaviour change.
The specific objectives of this scoping review were to assemble and conduct a configuring synthesis of empirical research evidence for the effects of economic instruments to change, or general exposure to changes in, prices or income on diet- and physical activity-related behaviours and corollary health-related outcomes. Evidence from intervention studies of the effects of economic instruments can be regarded as providing direct evidence, whilst evidence from studies that include assessment of the effects of general exposure to changes in prices or income can be regarded as providing supplementary, indirect evidence for the potential effects of economic instruments that are intended to operate via such price or income mechanisms.
Nature of the review
The methods applied in this systematic scoping review, including provisional study eligibility criteria, were pre-specified and documented in a protocol developed by all members of the review team (IS, GJH, TMM, RN, SAJ, MPK, MS and DO) (Protocol S1). Whilst scoping reviews follow a similar research process to systematic reviews  they differ from systematic reviews in terms of their objectives and key characteristics [18-21]. Scoping reviews explore, delimit and describe a broad evidence base whose boundaries are unclear at the outset, often as a preliminary stage to systematic reviews. Pre-specified eligibility criteria are therefore provisional and it is accepted that these may be refined and re-applied iteratively during the review, based on emergent knowledge of the studies and evidence encountered.
In parallel, scoping reviews are characterised by evidence synthesis strategies that focus on configuring or mapping evidence for effects [22,23], rather than on aggregating such evidence as exemplified by the use of meta-analysis to estimate pooled effect-sizes . In these circumstances, refined conceptual understanding of interventions and their proposed mechanisms of action becomes an intended output of the scoping process rather than its starting point. However, in conjunction with this more iterative, formative approach, scoping reviews can still be conducted using systematic, explicit methods. They can therefore be reported in compliance with established reporting guidelines for conventional systematic reviews of interventions [25,26] (Checklist S1).
Criteria for considering studies for this review
Final eligibility criteria for selecting studies into this scoping review were
Types of studies.
Empirical primary studies and reviews of any design (encompassing experimental, quasi-experimental, non-experimental and modelling studies; systematic and non-systematic reviews). Reviews were included if at least one of their included primary studies met other eligibility criteria. No restrictions for publication status, year or language were applied in searches.
Types of participants.
Human participants of any age, with no restrictions for demographic, socio-economic, or clinical characteristics. Animal studies were excluded.
Types of intervention or exposure.
Economic instruments (direct evidence) – Price promotions, taxes (including tax exemptions), supply-side subsidies (including subsidy removal), direct unit pricing legislation, or transfer payments (see Interventions S1 for a definition of each instrument). These interventions alter remunerative incentives by changing prices, except for transfer payments, which alter people’s budget constraints by changing their income. No restrictions were applied for type of comparator. Studies of personal financial incentives – defined as rewards or penalties with a monetary value, provided directly to individuals contingent on performance of specific behaviours or achievement of specific outcomes – were excluded due to our focus on population- rather than individual-level interventions. Exposures (indirect evidence) – Exposure to variation in prices of final consumer products or services and/or individual or household income (i.e. studies of prices or income as correlates or determinants of target outcomes).
Types of outcome measures.
i. Purchasing of food, non-alcoholic beverages or physical activity-related products or services; ii. other diet- or physical activity-related behaviours (e.g. food preparation, mealtime, snacking, physical activity or sedentary behaviours); iii. proximal consequences of such behaviours (i.e. those closer to the point of intervention or exposure in proposed causal pathways linking an intervention or exposure with final health outcomes), such as food, energy or nutrient intake, energy expenditure and physical fitness; and iv. distal consequences of such behaviours (i.e. those further from the point of intervention or exposure in proposed causal pathways linking an intervention or exposure with final health outcomes), namely modifiable physiological or metabolic risk factors for NCDs, such as body weight, blood cholesterol, blood glucose and blood pressure. Studies that did not report outcomes in at least one of these categories (i-iv) were excluded. For example, studies reporting measures of morbidity or mortality associated with NCDs (e.g. cardiovascular diseases, diabetes, cancers and chronic respiratory diseases) were excluded if they did not also report outcomes in at least one eligible category (i-iv). No restrictions were applied for type of measurement instrument or scale.
Location and selection of studies
Methods for locating and selecting eligible studies are described in detail elsewhere [27,28] and summarised here. We pre-specified provisional eligibility criteria, developed a coding guide to inform assessments of eligibility and used this to document any revisions made to provisional eligibility criteria during the study selection stage of the review. In practice, no substantive revisions were made to eligibility criteria. We conducted snowball searches in parallel with systematic searches of electronic literature databases. Snowball searches (i.e. scanning reference lists and forward citation tracking)  were conducted by one reviewer (IS) using PubMed and Google Scholar.
We designed systematic search strategies for MEDLINE (Ovid SP), EMBASE (Ovid SP) and PsycINFO (Ovid SP). These search strategies were necessarily highly sensitive, due to the following factors: our intention to progressively refine pre-specified, provisional eligibility criteria; our decision to impose few eligibility restrictions for types of participants, comparators, study designs or publication characteristics; the fact that search terms based on target intervention concepts were unlikely to be specific to titles, abstracts or indexing of eligible study reports; target outcomes encompassed multiple sets of health behaviours, proximal consequences of such behaviours, and the full range of physiological and metabolic risk factors for NCDs. We tested draft search strategies for their sensitivity to retrieve a reference set of 45 records of potentially eligible study reports, assembled using non-systematic internet searches and contacts with topic experts in the author team. We refined search strategies until they retrieved 100% of the reference set records indexed in each database, with a concurrent aim to minimise search yields so far as possible. The process of testing and refining draft search strategies confirmed that we could not achieve greater specificity without sacrificing sensitivity to retrieve eligible reference set records. We then adapted final versions for eight other electronic literature databases, based on close inspection of thesauri, scope notes and search syntax for each database.
Final systematic search strategies were executed between 11 July and 11 August 2011 in 11 relevant electronic literature databases from inception to present: MEDLINE (Ovid SP), EMBASE (Ovid SP), PsycINFO (Ovid SP), EconLit (EBSCO), SPORTDiscus with Full Text (EBSCO), Applied Social Sciences Index and Abstracts (CSA Illumina), Cochrane Database of Systematic Reviews (Wiley Online Library), Database of Abstracts of Reviews of Effects (Wiley Online Library), Health Technology Assessment Database (Wiley Online Library), NHS Economic Evaluation Database (Wiley Online Library), and Database of Promoting Health Effectiveness Reviews (EPPI Centre). Full details of systematic search strategies, search dates and yields are provided as Supporting Information files (Search Strategies S1 and Table S1).
All retrieved study records (title-abstract records) were exported to Endnote X4 reference management software and imported into EPPI Reviewer 4 systematic review software . We next ran EPPI Reviewer 4’s integrated automatic de-duplication software. We used text mining technologies to prioritise records for manual screening [31,32]. Text mining is “an automated process that can assist with the identification and structuring of patterns in the text of individual documents and across multiple documents” . In this scoping review, text mining technologies were applied to automatically analyse text in the growing pool of screened title-abstract records to identify patterns of terms likely to distinguish eligible from ineligible records, search for those unscreened records most likely to be judged eligible and prioritise these to be manually screened next. Our use of text mining technologies to support title-abstract screening was justified by the extremely large number of records retrieved by systematic searches, which made application of conventional screening methods [31,34] impractical within available resources.
A preceding stage to application of text mining technologies involved manual screening of a random sample of title-abstract records to estimate a baseline inclusion rate (BIR). The BIR is an a priori estimate of the proportion of retrieved title-abstract records expected to be assessed as provisionally eligible. Our estimated BIR was 0.00348, indicating that ≈0.3%, or ≈3,669 of >1 million, retrieved records could be expected to be assessed as provisionally eligible. This estimate was used as a target to monitor the overall progress of title-abstract screening. Sequential sets of title-abstract records were then prioritised using text mining and assigned for manual screening. One reviewer (IS) undertook all screening assignments prioritised by text mining. Final eligibility criteria were confirmed at the end of the title-abstract screening stage, in consultation with the multi-disciplinary review team and with reference to a purposive sample of provisionally eligible, or borderline eligible, full-text study reports. Provisional eligibility decisions were reviewed against final eligibility criteria.
Corresponding full-text reports were sought for all title-abstract records assessed as provisionally eligible via electronic library resources of three universities (University of Cambridge, University of East Anglia and King’s College, London). Full-text screening and eligibility assessment was performed by one reviewer (IS). Full-text reports located using snowball searches were also assessed against final eligibility criteria by one reviewer (IS). Further duplicates were identified manually and removed at this stage. Multiple reports of the same study were linked into a single study.
Data collection and analysis
All included studies were classified by behavioural domain (diet, physical activity, both), study type (intervention study, exposure study, both; primary study or review) and population (including a focus on high income country (HIC)  populations or population subgroups, including a focus on low and middle income country (LMIC)  populations or population subgroups, both), and types of outcomes (i. purchasing, ii. other behaviours, iii. proximal consequences, iv. distal consequences) by one reviewer (IS).
Due to the large number of included studies, the review team decided at this stage to prioritise studies of the effects of economic instruments (i.e. direct evidence for intervention effects) that included a focus on HIC populations or population subgroups for further, more detailed data collection and analysis. Data were collected from the latter set of studies using a pre-specified data extraction form, based in Microsoft Excel, which had been pilot-tested on a random sample of ten included studies. In addition to study characteristics collected from all included studies, we collected further information relating to primary study design (experimental, quasi-experimental, non-experimental, modelling), interventions (specific sub-types for each broad type of economic instrument), outcomes (details of specific measures within each broad category i-iv), and relevant principal findings and conclusions, the latter two being transcribed verbatim from full-text study reports. All data collection was undertaken and checked by one reviewer (IS). We did not plan or conduct assessments of risk of bias in included studies .
We did not plan or conduct statistical meta-analysis of the results of included studies. Summary statistics (frequencies and percentages) were calculated to inform development of a configuring synthesis of included studies in terms of their key characteristics and findings. This included developing narrative statements to summarise the evidence base for each broad type of intervention, based on the study characteristics data and transcribed principal findings and conclusions collected from the relevant subset of primary studies. It was important to base these summary statements on evidence from primary studies, rather than review articles, to avoid the risk of ‘multiple-counting’ of evidence derived from sets of primary studies that overlapped between included reviews. The cumulative evidence base was analysed and interpreted iteratively as the review proceeded through discussions involving all members of the multi-disciplinary review team (IS, GJH, TMM, RN, SAJ, MPK, MS and DO).
Figure 1 shows the flow of studies though each stage of the systematic scoping review process . Systematic searches retrieved 1,426,032 title-abstract records. Following automatic de-duplication, 1,053,908 records remained. We manually screened a total of 52,695 title-abstract records (of which 46,580 were prioritised using text mining) and selected 1,464 records as provisionally eligible. Corresponding full-text reports were assessed for 970 provisionally eligible records. Of the remaining records assessed as provisionally eligible, 120 were identified as further duplicates and discarded, whilst 374 corresponding full-text reports could not be assessed either because they could not be retrieved (328 study reports) or because they were published in a language other than English (46 study reports); translation was beyond the scope of this review.
Based on full-text screening, 774 study reports met final eligibility criteria and were selected for inclusion in the review, whilst the remaining 196 were excluded for reasons shown in Figure 1. Parallel snowball searches identified 290 full-text study reports that met final eligibility criteria, of which 118 were discarded because they duplicated study reports identified via systematic searches and text mining prioritised screening. Once multiple reports of the same study had been linked, 880 eligible studies were included in the review (comprising 946 study reports). Bibliographies of all included studies, study reports excluded based on full-text screening and those for which the full-text could not be assessed are provided as Supporting Information files (Bibliographies S1).
The final set of 880 eligible studies comprised 192 studies of the effects of economic instruments (i.e. intervention studies, of which 181 included a focus on HICs) and 768 studies (including 86 intervention studies) that included evidence for prices or income as correlates or determinants of target outcomes (i.e. studies of exposure to general variations in prices or income, of which 610 included a focus on HICs). In this article, our predominant focus is on summarising evidence from the 181 intervention studies that included a focus on HICs and were prioritised for detailed data collection and analysis (intervention studies – direct evidence). We also briefly describe the body of 610 studies that included evidence for prices or income as correlates or determinants of target outcomes (studies of exposures – indirect evidence) and included a focus on HICs.
Evidence from intervention studies
Table S2 (a Supporting Information file) summarises key characteristics of the 181 studies of the effects of economic instruments that included a focus on HIC populations or population subgroups. Table 1 and Table 2 show the overall distributions of evidence for the effects of broad categories and specific sub-types of economic instruments on (respectively) diet-related and physical activity-related outcomes across those 181 studies. Four broad categories of interventions were encountered across these 181 studies: price promotions, taxes, supply-side subsidies and transfer payments. No eligible studies of direct unit pricing legislation were identified. For some intervention types, the identified evidence base comprised more reviews than primary studies. There was much more evidence concerning diet-related outcomes than physical activity. The large majority of intervention studies (87%, 158 of 181) included a focus on United States (US) populations or population subgroups, whilst six per cent (11 of 181) included a focus on United Kingdom (UK) populations or population subgroups.
|Intervention category||Subsidiary intervention type||Primary studies (n)||Reviews (n)|
|Price promotions (n=34)||Simple discounts - price restructuring in discrete settings||13||17|
|Taxes (n=56)||Agricultural commodity tax||0||0|
|Soft drinks tax||18||12|
|Supply-side subsidies (n=47)||Agricultural commodities||0||3|
|Agricultural commodities – subsidy removal||1||2|
|Healthy non-alcoholic beverages||3||6|
|Food transportation or delivery||1||1|
|Direct unit pricing legislation (n=0)||Minimum pricing legislation||0||0|
|Maximum pricing legislation||0||0|
|Transfer payments (n=80)||Restricted income transfers, welfare benefits or welfare assistance programs||62||16|
|Unrestricted income transfers, welfare benefits or welfare assistance programs||3||1|
|Intervention category||Subsidiary intervention type||Primary studies (n)||Reviews (n)|
|Price promotions (n=3)||Any type of price promotion||0||3|
|Simple discounts - price restructuring in discrete settings||0||0|
|Taxes (n=3)||Tax exemptions||0||1|
|Supply-side subsidies (n=1)||Physical activity products||0||0|
|Physical activity services (programs)||0||1|
|Direct unit pricing legislation (n=0)||Minimum pricing legislation||0||0|
|Maximum pricing legislation||0||0|
|Transfer payments (n=5)||Restricted income transfers, welfare benefits or welfare assistance programs||2||0|
|Unrestricted income transfers, welfare benefits or welfare assistance programs||0||0|
Table S3 (a Supporting Information file) summarises evidence for the reported effects of each broad category of intervention, based on analysis of primary study characteristics (see Table S2) combined with examination of transcribed principal findings and conclusions across the relevant subset of primary studies. Overall, evidence for associations between each type of economic instrument and target diet-related outcomes was largely equivocal. By equivocal we mean that few consistent patterns of reported findings could be identified between heterogenous sets of primary studies of each type of economic instrument with respect to the directions of their associations with (or effects on) diet-related behaviours or corollary outcomes. An exception was evidence from a small, relatively homogenous cluster of prospective quasi-experimental studies conducted in US populations, which found that simple discounts to reduce the unit retail prices of healthy foods and drinks in school or workplace vending machines or cafeterias was associated with increased purchasing of those foods; see, for example [37-39]. We identified few studies of other types of price promotion such as multi-buy deals, price-pack deals, price deals, introductory pricing, couponing or rebates (three primary studies and three reviews).
Although a substantial body of evidence for the effects of taxes or supply-side subsidies on specific foods, drinks or nutrients (food-related taxes or subsidies) was identified, few studies measured behavioural responses to real interventions using experimental or robust quasi-experimental study designs. This is largely attributable to the fact that relatively few governments or legislatures outside the US have implemented such interventions to date . Instead, the prevailing approach involved simulating predicted effects of hypothetical tax or subsidy policy scenarios based on demand elasticities derived from purchasing data. For example, Andreyeva and colleagues estimated the predicted effects of a US national one cent-per-ounce tax on sugar contained in sugar sweetened beverages (SSBs) by constructing a model that projected future beverage consumption based on recent regional beverage consumption data, historic trends in beverage consumption and estimates of the price elasticity of SSB demand .
Studies of food-related taxes or subsidies collectively assessed a large and diverse set of specific outcomes. For example, the impacts of such taxes have been modelled in terms of ≈800 specific measures across included primary studies, only ≈5% of which were assessed in more than one study. Modelled tax rates were small compared with those typically applied to tobacco or alcohol products and were generally predicted to have small-to-moderate beneficial effects on purchasing, or negligible effects on body weight. Several authors concluded that higher tax or subsidy rates than those considered in their studies might need to be imposed to have a meaningful impact on target outcomes. Although most studies of food-related taxes modelled compensatory purchasing, whereby consumers substitute within or between taxed and untaxed sets of products, the range of substitute or complementary products incorporated into analyses was inevitably limited relative to the vast array of potential alternative food and beverage products available to consumers. For example, Andreyeva and colleagues highlighted that their prediction that estimated tax-induced reductions in SSB purchasing “could translate into losses in average body weight [of] up to 5 lbs/year... is certainly an upper bound given potential substitution to other caloric beverages and foods” and that “reliable estimates of the cross-price elasticities necessary to quantify the extent of possible substitution and the net impact on caloric intake are not available.” . Furthermore, few studies modelled potential supply-side responses, such as product reformulation to avoid taxes, or adjustment of retail prices to limit or amplify the extent to which the price effects of taxes or subsidies are passed through to consumers in the form of changes in relative unit prices at the checkout. Several authors highlighted, but had rarely demonstrated, the potentially regressive nature of food-related taxes.
The largest subset of studies of transfer payments involved analyses of large, observational datasets to investigate the impacts of US nutrition and federal food assistance programs (63 primary studies and 13 reviews), principally the Supplemental Nutrition Assistance Program (SNAP – formerly known as the Food Stamp Program). SNAP provides financial assistance for food purchasing to people and families on low or no incomes by income transfer via an Electronic Benefit Transfer (debit card) system. Use of SNAP benefits is generally restricted to the purchase of foods or food products for consumption at home, such as breads and cereals, fruits and vegetables, meats, fish and poultry and dairy products . Studies of transfer payments assessed large numbers of outcomes and typically reported mixed patterns of results. For example, Fox and colleagues conducted a secondary analysis of observational data collected using the Third National Health and Nutrition Examination Survey (NHANES-III) to investigate differences between participants in the US federal SNAP and two comparison groups of non-participants (income-eligible and higher income non-participants). Between-group differences were assessed in terms of (inter alia) frequency of meal consumption, seven measures of physical activity, dietary intake of nine key nutrients and dietary components, overall dietary quality (based on Healthy Eating Index total and component scores), and eight measures of physiological or metabolic risk factors for non-communicable disease (or related biomarkers), and the study found a mixed pattern of results across these outcomes in terms of both the existence and the direction of statistically significant differences between groups . Two other US programs studied were the National School Lunch Program (a federally assisted meal program that pays subsidies to public and non-profit private schools and residential child care institutions to support the provision of reduced price or free lunches to children each school day ) and the School Breakfast Program (a federally assisted meal program that provides subsidies to participant States to operate non-profit breakfast programs, in which breakfasts are provided at a reduced price in schools and residential childcare institutions ). Primary studies that assessed the impacts of participation in these programs – classified as studies of supply-side subsidies, alongside the smaller group of modelling studies that simulated the predicted effects of supply-side subsidies applied to healthier foods, non-alcoholic beverages or nutrients (described above and in Table S3) – were also typically characterised by large numbers of outcome measures and mixed patterns of results.
In summary, our analysis indicates that the public health case for using economic instruments to promote dietary behaviour change depends largely on evidence from US studies; from studies that vary in the degree to which they support causal inferences; and from studies modelling predicted (rather than actual) behavioural responses, which inevitably rely on simplifications and a variety of more or less credible assumptions. Moreover, there is a general lack of evidence to inform the case for or against the use of economic instruments to promote physical activity (see Tables 1, 2, S2 and S3). Overall, we identified two primary studies that assessed congestion charges (taxes) [45,46]; two that assessed employer-sponsored benefit schemes (transfer payments) to promote employees’ use of, respectively, health clubs and public transport [47,48]; and one that evaluated tax credits (transfer payments) provided to parents who had enrolled their children in organised physical activity programmes . This finding is consistent with, and adds little to, those of the six published reviews we identified that included coverage of the use of economic instruments to promote physical activity [50-55].
Further specific types of eligible interventions were encountered in three studies that included investigation of the effects of cigarette taxes [56-58] (of which one also considered a gasoline tax  and one also considered a tax on beer ), and two studies of the effects of housing  and childcare  subsidies. Outcomes assessed in these five studies comprised various measures of physical activity  and measures of body weight or body weight status [57-60]. These specific interventions are not included in Table 1 or Table 2 because the pathways by which they may influence these outcomes are likely to differ considerably from those of the other specific intervention types within the corresponding broader category (taxes and supply-side subsidies, respectively). For example, Mellor and colleagues propose a causal pathway by which cigarette taxes increase cigarette prices and costs, reduce smoking and increase food expenditures and consumption in the household, leading to increased likelihood of childhood obesity .
Across diet and physical activity, few studies of economic instruments included assessment of long-term effects and few incorporated formal economic evaluation. Whilst several studies of some specific types of intervention (e.g. food-related taxes and subsidies, subsidised school meals, or transfer payments restricted for use to purchase foods) incorporated assessments of distributions of effects by population sub-groups, such assessments were less frequently conducted for other intervention types. In addition, many included studies of economic instruments that are reliant on behaviour change as the pathway for modifying physiological or metabolic risk factors did not assess intermediate behavioural endpoints at all.
Evidence from studies of general exposure to variations in prices or income
610 studies conducted in HICs incorporated evidence for price or income levels as correlates or determinants of target outcomes. This body of evidence bears indirectly on the public health case for using economic instruments because such instruments are proposed to operate via price and income mechanisms to influence the target behaviours (e.g. price promotions change the relative unit retail prices of foods, and prices are, in turn, a determinant of food choices ). This evidence base was characterised by heterogeneity of study designs and of the populations, exposures, outcomes, and covariates assessed. However, a clear inference from reviewing this set of studies is that the relationships between prices or income and target outcomes may often be non-linear, moderated by a wide range of modifiable and non-modifiable factors and mediated by complex mechanisms of action. Also, many studies reported null associations or statistically significant associations in the opposite direction to that anticipated. Whilst further synthesis of evidence drawn from subsets of studies within this large body of literature may provide some useful pointers for policy design, it primarily serves to highlight that hasty generalisation about the causal pathways linking changes in prices of diet- and physical activity-related products or services and income to behavioural and health outcomes is not warranted.
Implications for policy and research
Current evidence for the effects of economic instruments on diet- and physical activity-related outcomes is limited in terms of the potential for causal inference and yields equivocal findings. The relative lack of direct evidence for the effects of economic instruments on physical activity is striking. Whilst direct evidence for effects on diet is much more prevalent, the dearth of experimental studies and inconsistency in findings between studies of the same types of interventions in terms of the specific outcomes they have assessed are key limitations of this cumulative evidence base.
Whilst economic instruments have been suggested in several studies to hold promise, mixed patterns of findings for most intervention types are likely to reflect the heterogeneous evidence base, as well as the complexity of behavioural responses to economic stimuli and of the causal pathways involved. This suggests a need for caution in developing policy based on limited evidence and overly simple assumptions about how people will respond to changes in prices and income. It does not necessarily imply that underlying economic theory, which holds that people respond rationally to incentives, or behavioural economic theory, which holds that rationality of choice is moderated by heuristics and biases attributable to various social, cognitive, and emotional factors , are flawed. Rather, it is likely that people’s responses to, say, a tobacco control tax are relatively predictable, whereas their responses to, say, a tax-stimulated increase in the prices of specific foods, relative to the vast array of alternative foods available, are less predictable and more complex in their relationships to health behaviours and corollary outcomes.
Crucially, our findings highlight the need to implement interventions in order to subject the logic models and programme theories involved to much closer scrutiny than they have been afforded in intervention research in these fields to date. In particular, people’s actual behavioural responses to interventions should be measured using prospective or ex-post evaluation studies capable of generating reliable causal inferences that increase our understanding of the conditions under which economic incentives work, as well as the reasons they may sometimes produce counter-intuitive behavioural responses . So far as recent policy interest translates into implementation of interventions designed to change prices or income, this represents an important opportunity for evaluations of public health impacts to be conducted. However, to maximise these opportunities, implementation strategies will need to be carefully aligned with evaluation planning and design.
In order to support causal inferences about the effects of interventions, there is a further need for future intervention studies to include reliable measures of impacts at different stages of proposed causal pathways between interventions and intermediate or final health outcomes . Where possible, for example, studies of economic instruments intended to alter people’s body weight, by changing first their food purchasing behaviour and then their food and nutrient intake, should include measures of both purchasing and intake alongside measures of body weight. However, we also recognise that intervention studies alone are unlikely to generate all the evidence needed to unpack all of the links in causal chains of effects. The limited feasibility of conducting intervention studies with sufficient length of follow-up to allow for measurement of long-term effects makes it likely that quantification of relationships between behavioural and proximal endpoints and final health outcomes (principally, mortality and morbidity associated with NCDs) will remain largely within the purview of observational epidemiology. Therefore, when the ultimate goal of intervention is to improve population health, the study of interventions that work via price or income mechanisms will usually require synthesis of evidence from both intervention and epidemiological studies and use of modelling to extrapolate from intermediate to final health outcomes. Given the current inconsistency between studies in the specific outcomes they assess, development of the cumulative evidence base for the health-related effects of economic instruments (and other population-level interventions) would also benefit from international consensus on a core set of outcome measures to be routinely included in future surveys, primary studies and systematic reviews.
Our findings have identified a general lack of economic evaluation of interventions that may need to be addressed in future research if policy is to be informed by considerations of how to maximise population health within available resources. Future primary studies and systematic reviews would be further strengthened by explicit assessments of distributional effects between population subgroups and of potential trade-offs or synergies between improving diet and physical activity ‘on average’ and reducing health and economic inequalities . Finally, the capacity of economic instruments to bring about sustained population-level shifts in diet and physical activity needs to be evaluated, given the potential for any beneficial effects to weaken as consumers become accustomed to altered relative prices or altered budget constraints.
Strengths and limitations of the review
A major strength of this systematic scoping review is its broad scope. Eligibility criteria adopted by previous systematic reviews in this area are not as inclusive across all components of the PICOS framework, and none have adopted such a sensitive, broad-based approach to locating and selecting eligible studies. Our development and application of innovative text mining methods to support study selection from an initially broad set of >1 million retrieved records enabled us to assemble, configure and describe a large, multi-disciplinary evidence base for relevant interventions and exposures on a scale that few reviews have managed previously. This is important, because the design and implementation of behaviour change programmes have been hampered in the past by their being located in single disciplines. In parallel, we have configured typologies of interventions, delimited sets of outcomes and identified study-level limitations in ways that can support future primary and secondary research.
However, our inclusive, broad-based approach also engendered limitations that need to be considered when interpreting our findings. First, based on the baseline inclusion rate, we estimate that only ≈40% of title-abstract records expected to be selected for full-text screening were identified within available resources. It is acknowledged that the obligation to identify every eligible study may be relaxed to some extent in scoping reviews, since they typically prioritise conceptual breadth (the aim to assemble a range and distribution of eligible studies that is representative of the target evidence base in terms of key study characteristics) over depth (the aim to assemble all eligible studies) . Nevertheless, the validity of our configuring synthesis is reliant on the range and distribution of included studies being representative of the unknown full set of eligible studies, which encompasses those represented in the ≈60% (2,205 of 3,669) title-abstract records that were expected to be selected as provisionally eligible but were not screened or analysed (as well as those 374 full-text study reports that could not be assessed because inter-library loans and translation of non-English language studies were beyond the scope of the review). Our parallel use of snowball searches, coupled with the observation that included reviews did not identify any further eligible types of study designs, interventions or exposures not captured in this review, may ameliorate concern about this limitation. Moreover, had we used conventional screening methods, we would have identified only ≈5% of provisionally eligible study records within available resources, compared with the ≈40% we were able to identify in practice, assisted by the use of text mining.
Second, the large absolute number of included studies (N=880) necessitated prioritisation of studies that included a focus on HIC populations for data collection and analysis (N=791), with the subset of intervention studies in HIC populations (N=181) prioritised for further, more detailed data collection and analysis. With limited scope to collect and analyse data from studies of general exposure to changes in prices or income in HIC populations, results derived from these studies should be viewed with caution. Furthermore, we did not generate any results pertaining to the effects of interventions or exposures on diet- or physical activity-related behaviours and corollary outcomes in LMIC populations. We have, however, assembled a large body of studies that comprise this evidence base for LMIC populations, which could, in principle, be synthesised in future systematic reviews.
In this scoping review, we did not plan to conduct formal risk-of-bias assessments, nor aggregative synthesis (such as meta-analysis) of the results of included studies. The scope for conducting formal risk-of-bias assessments was, in any case, limited by a lack of established methods for conducting such assessments on quasi-experimental and non-experimental study designs. The lack of formal aggregative synthesis of the results of included studies, which means we do not (and did not set out to) present pooled effect-sizes for economic instruments in terms of target outcomes, reflects differences between the objectives and characteristics of scoping reviews and those of systematic reviews, outlined in the introduction to this article. Clearly, reliable estimates of intervention effect-sizes are an important component of the basis for policy formulation, and meta-analysis of evidence for the effects of economic instruments will therefore form an important component of the design of subsequent systematic reviews that follow the preliminary, configuring synthesis performed in this systematic scoping review.
Finally, a further limitation of this scoping review was that a single reviewer completed the large majority of screening at the study selection stage. Whilst initial development of the scoping review protocol and judgements concerning potential revisions to provisional eligibility criteria and application of final eligibility criteria were based on discussions involving the wider review team and collective examination of examples drawn from the emerging body of eligible and borderline eligible studies, this limitation should be acknowledged. The ‘gold standard’ approach involves assessments of study eligibility being undertaken by at least two reviewers working independently, in order to reduce the possibility that eligible study reports will be discarded due to human error .
Our findings have exposed a complex, limited and largely equivocal evidence base, suggesting that the public health case for using economic instruments to promote dietary and physical activity behaviour change may be less compelling than some proponents have claimed. This conclusion provides an important counterpoint to what are, in our view, overly optimistic claims made by some authors of individual primary studies and reviews for the use of economic instruments to improve population health behaviour. It implies a need for caution in the development of public health policies intended to alter economic environmental stimuli to incentivise health-enhancing dietary and physical activity behaviour change at population level. In particular, policy implementation needs to be carefully aligned with evaluation planning and design.
Bibliographic details of included studies, excluded studies, and studies not assessed.
Defintions of interventions.
Protocol for the systematic scoping review.
Search strategies used in electronic searches.
Electronic search dates and yields, by database.
Characteristics of included studies.
We would like to acknowledge our collaboration with colleagues at the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), Social Sciences Research Unit, Institute of Education, University of London (Jeff Brunton, Sergio Graziosi, Alison O’Mara-Eves, Claire Stansfield and James Thomas) on the development and application of text mining technologies to facilitate the study screening and selection stage of this scoping review. We are also grateful to colleagues in the Behaviour and Health Research Unit, University of Cambridge (Martin Roland) and the UKCRC Centre for Diet and Activity Research, Cambridge (Vivien Hendry and Pablo Monsivais) for their comments on an earlier draft of this article.
Conceived and designed the experiments: IS GJH TMM SAJ MPK MS DO. Performed the experiments: IS RN. Analyzed the data: IS. Wrote the manuscript: IS GJH TMM RN SAJ MPK MS DO. Interpretation of results: IS GJH TMM RN SAJ MPK MS DO.
- 1. World Health Organization (2012) World Health Statistics 2012. Geneva: World Health Organization. 176pp.
- 2. Beaglehole R, Bonita R, Horton R, Adams C, Alleyne G et al. (2011) Priority actions for the non-communicable disease crisis. Lancet 377: 1438-1447. doi:https://doi.org/10.1016/S0140-6736(11)60393-0. PubMed: 21474174.
- 3. World Health Organization (2011) Global status report on noncommunicable diseases 2010. Geneva: World Health Organization. 162pp.
- 4. United Nations (2011) Draft political declaration of the high-level meeting on the prevention and control of NCDs. New York: United Nations. p. 13.
- 5. Mytton OT, Clarke D, Rayner M (2012) Taxing unhealthy food and drinks to improve health. BMJ 344: e2931. doi:https://doi.org/10.1136/bmj.e2931. PubMed: 22589522.
- 6. HM Government (2012) The Government’s Alcohol Strategy. London: the Stationery Office. 28pp.
- 7. HM Government (2011) Healthy Lives, Healthy People: A Tobacco Control Plan for England. London: Department of Health. 96pp.
- 8. World Health Organization (2003) WHO Framework Convention on Tobacco Control. Geneva: World Health Organization. 36pp.
- 9. Marteau TM, Ogilvie D, Roland M, Suhrcke M, Kelly MP (2011) Judging nudging: can nudging improve population health? BMJ 342: 263-265. PubMed: 21266441.
- 10. Hollands GJ, Shemilt I, Marteau TM, Jebb SA, Kelly MP et al. (submitted) (201#) Altering micro-environments to change population health behavior: towards an evidence base for choice architecture interventions. Am J Public Health.
- 11. Marteau TM, Hollands GJ, Fletcher PC (2012) Changing human behaviour to prevent disease: the importance of targeting automatic processes. Science 337: 1492-1495. doi:https://doi.org/10.1126/science.1226918. PubMed: 22997327.
- 12. International Agency for Research on Cancer (2012) Effectiveness of Tax and Price Policies for Tobacco Control (IARC Handbooks of Cancer Prevention in Tobacco Control, Volume 14). Geneva: World Health Organization. 374pp.
- 13. Thomas S, Fayter D, Misso K, Ogilvie D, Petticrew M et al. (2008) Population tobacco control interventions and their effects on social inequalities in smoking: systematic review. Tob Contr 17: 230-237. doi:https://doi.org/10.1136/tc.2007.023911. PubMed: 18426867.
- 14. Rice N, Godfrey C, Slack R, Sowden A, Worthy G (2008) A systematic review of the effects of price on the smoking behaviour of young people. York. Public Health Research Consortium. 181pp.
- 15. Booth A, Meier P, Stockwell T, Sutton A, Wilkinson A et al. (2008) Independent Review of the Effects of Alcohol Pricing and Promotion. Part A: Systematic Reviews. Sheffield: School of Health and Related Research. 243pp.
- 16. Brennan A, Purshouse R, Taylor K, Rafia R (2008) Independent Review of the Effects of Alcohol Pricing and Promotion. Part B: Modelling the Potential Impact of Pricing and Promotion Policies for Alcohol in England: Results from the Sheffield Alcohol Policy Model, version 2008(1-1). . Sheffield: School of Health and Related Research. 217 p.
- 17. Higgins JPT, Green S, editors (2011) 4: Guide to the contents of a Cochrane protocol and review. In: JPT HigginsS. Green. Cochrane Handbook for Systematic Reviews of Interventions Chapter Version 5.1.0; (updated March 2011) The Cochrane Collaboration. Available: www.cochrane-handbook.org. Accessed 1 November 2012.
- 18. Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD et al. (2011) 1: Introduction. In: JPT HigginsS. Green. Cochrane Handbook for Systematic Reviews of Interventions Chapter Version 5.1.0; (updated March 2011) The Cochrane Collaboration. Available: www.cochrane-handbook.org. Accessed 1 November 2012.
- 19. Arksey H, O’Malley L (2005) Scoping Studies: towards a methodological framework. Int J Soc Res Methodol 8: 19-32. doi:https://doi.org/10.1080/1364557032000119616.
- 20. Levac D, Colquhoun H, O’Brien KK (2010) Scoping studies: advancing the methodology. Implement Sci 5: 69. doi:https://doi.org/10.1186/1748-5908-5-69. PubMed: 20854677.
- 21. Gough D, Thomas J, Oliver S (2012) Clarifying differences between review designs and methods. Syst Rev 1: 28. doi:https://doi.org/10.1186/2046-4053-1-28. PubMed: 22681772.
- 22. Gough D, Thomas J (2012) Commonality and diversity in reviews. In: D. GoughS. OliverJ. Thomas. An Introduction to Systematic Reviews. London: Sage Publications Ltd.. pp. 35-66.
- 23. Hammersley M (2002) Systematic or Unsystematic, Is That The Question? Some reflections on the science, art and politics of reviewing research evidence. London: Health Development Agency Public. Health Steering Group. 11 p. Available: http://guidance.nice.org.uk/nicemedia/pdf/sys_unsys_phesg_hammersley.pdf. Accessed 1 November 2012..
- 24. Deeks JJ, Higgins JPT, Altman DG, editors (2011) 9: Analysing data and undertaking meta-analyses. In: JPT HigginsS. Green. Cochrane Handbook for Systematic Reviews of Interventions Chapter Version 5.1.0; (updated March 2011) The Cochrane Collaboration. Available: www.cochrane-handbook.org. Accessed 1 November 2012.
- 25. Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Med 6: e1000097.
- 26. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC et al. (2009) The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLOS Med 6: e1000100. PubMed: 196315071962251219621070.
- 27. Shemilt I, Simon A, Hollands GJ, Marteau TM, Ogilvie D et al. (201#) Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. Res Syn Meth (under review).
- 28. Shemilt I, Hollands GJ, Marteau TM et al.Effects of changes in the economic environment on diet- and physical activity-related behaviours and corollary outcomes: a large-scale scoping review. Cambridge: Behaviour and Health Research Unit. p. 201#.
- 29. Greenhalgh T, Peacock R (2005) Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ 331: 1064-1065. doi:https://doi.org/10.1136/bmj.38636.593461.68. PubMed: 16230312.
- 30. Thomas J, Brunton J, Graziosi S (2010) EPPI-Reviewer. p. 4.0: software for research synthesis. EPPI-Centre Software. London: Social Science Research Unit, Institute of Education.
- 31. Thomas J, McNaught J, Ananiadou S (2011) Applications of text mining within systematic reviews. Res Syn Meth 2: 1-14. doi:https://doi.org/10.1002/jrsm.27.
- 32. Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH (2010) Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics 11(55): 55-. PubMed: 20102628.
- 33. Gough G, Oliver S, Thomas J (2012) Glossary. In: G. GoughS. OliverJ. Thomas. An Introduction to Systematic Reviews. London: Sage Publications Ltd.. pp. 257-262.
- 34. Higgins JPT, Deeks JJ, editors (2011) 7: Selecting studies and collecting data. In: JPT HigginsS. Green. Cochrane Handbook for Systematic Reviews of Interventions Chapter Version 5.1.0; (updated March 2011) The Cochrane Collaboration. Available: www.cochrane-handbook.org. Accessed 1 November 2012.
- 35. Bank World World Bank Country and Lending Groups website. Available: http://data.worldbank.org/about/country-classifications/country-and-lending-groups. Accessed 2011 November 30.
- 36. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D et al. (2011) The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343: d5928. doi:https://doi.org/10.1136/bmj.d5928. PubMed: 22008217.
- 37. French SA, Hannan PJ, Harnack LJ, Mitchell NR, Toomey TL, Gerlach A (2010) Pricing and Availability Intervention in Vending Machines at Four Bus Garages. J Occup Environ Med 52: S29–S33. doi:https://doi.org/10.1097/JOM.0b013e3181c5c476. PubMed: 20061884.
- 38. French SA, Jeffery RW, Story M, Breitlow KK, Baxter JS et al. (2001) Pricing and promotion effects on low-fat vending snack purchases: the CHIPS Study. Am J Public Health 9: 112-117. doi:https://doi.org/10.1007/BF02962506. PubMed: 11189801.
- 39. French SA, Story M, Jeffery RW, Snyder P, Eisenberg M et al. (1997) Pricing strategy to promote fruit and vegetable purchase in high school cafeterias. J Am Diet Assoc 97: 1008-1010. doi:https://doi.org/10.1016/S0002-8223(97)00242-3. PubMed: 9284880.
- 40. Andreyeva T, Chaloupka FJ, Brownell KD (2011) Estimating the potential of taxes on sugar-sweetened beverages to reduce consumption and generate revenue. Prev Med 52: 413-416. doi:https://doi.org/10.1016/j.ypmed.2011.03.013. PubMed: 21443899.
- 41. US Department of Agriculture Special Nutrition Assistance Program website. Available: http://www.fns.usda.gov/snap/. Accessed 2013 July 16.
- 42. Fox MK, Cole N (2004a) Nutrition and health characteristics of low-income populations: Volume I, Food Stamp Program participants and nonparticipants. Alexandria, VA, US: Department of Agriculture, Economic Research Service. 396pp.
- 43. US Department of Agriculture National School Lunch Program website. Available: http://www.fns.usda.gov/slp. Accessed 2013 July 16.
- 44. US Department of Agriculture School Breakfast Program website. http://www.fns.usda.gov/sbp. Accessed 2013 July 16.
- 45. Bergman P, Grjibovski AM, Hagströmer M, Patterson E, Sjöström M (2010) Congestion Road Tax and Physical Activity. Am J Prev Med 38: 171-177. doi:https://doi.org/10.1016/j.amepre.2009.09.042. PubMed: 20117573.
- 46. Transport for London (2008) Central London Congestion Charging: Impacts monitoring. Sixth. Annu Rep, July (2008). London: Transport for London. 220 p.
- 47. Lucove JC, Huston SL, Evenson KR (2007) Workers’ Perceptions About Work site Policies and Environments and Their Association With Leisure-Time Physical Activity. Am J Health Promot 21: 196-200. doi:https://doi.org/10.4278/0890-1171-21.3.196. PubMed: 17233238.
- 48. Lachapelle U, Frank LD (2009) Transit and Health: Mode of Transport, Employer-Sponsored Public Transit Pass Programs, and Physical Activity. J Public Health Policy 30: S73-S94. doi:https://doi.org/10.1057/jphp.2008.52. PubMed: 19190584.
- 49. Spence JC, Holt NL, Dutove JK, Carson V (2010) Uptake and effectiveness of the Children’s Fitness Tax Credit in Canada: the rich get richer. BMC Public Health 10: 356. doi:https://doi.org/10.1186/1471-2458-10-356. PubMed: 20565963.
- 50. Faulkner G, Grootendorst P, Nguyen VH, Ferrence R, Mendelson R et al. (2010) Economic Policy, Obesity and Health: A Scoping Review. Toronto, Ontario: Exercise Psychology Unit. 80pp.
- 51. Brownson RC, Haire-Joshu D, Luke DA (2006) Shaping the Future Context of Health: A Review of Environmental and Policy Approaches in the Prevention of Chronic Diseases. Annu Rev Public Health 27: 341-370. doi:https://doi.org/10.1146/annurev.publhealth.27.021405.102137. PubMed: 16533121.
- 52. Levi J, Vinter S, St Laurent R, Segal LM (2010) F as in Fat: How Obesity Threatens America’s Future. Washington DC Trust America’s Health: 121.
- 53. Madore O (2007) The Impact of Economic Instruments That Promote Healthy Eating, Encourage Physical Activity and Combat Obesity: Literature Review. Ottawa Parliamentary Info Res Service. 3:
- 54. Matson-Koffman DM, Brownstein JN, Neiner JA, Greaney ML (2005) A Site-specific Literature Review of Policy and Environmental Interventions that Promote Physical Activity and Nutrition for Cardiovascular Health: What Works? Am J Health Promot 19: 167-193.
- 55. von Tigerstrom B, Larre T, Sauder J (2011) Using the Tax System to Promote Physical Activity: Critical Analysis of Canadian Initiatives. Am J Public Health 101: e10-e16. doi:https://doi.org/10.2105/AJPH.2011.300201. PubMed: 21680912.
- 56. McInnes MM, Shinogle JJ (2009) Physical activity: Economic and policy factors. Cambridge, MA: National Bureau of Economic Research. 42pp.
- 57. Mellor JM (2011) Do cigarette taxes affect children’s body mass index? The effect of household environment on health. Health Econ 20: 417-431. doi:https://doi.org/10.1002/hec.1598. PubMed: 21394814.
- 58. Rashad I, Grossman M, Chou S-Y (2006a) The super size of America: An economic estimation of body mass index and obesity in adults. East Econ J 32: 133-148.
- 59. Meyers A, Frank DA, Roos N, Peterson KE, Casey VA et al. (1995) Housing subsidies and pediatric undernutrition. Arch Pediatr Adolesc Med 149: 1079-1084. doi:https://doi.org/10.1001/archpedi.1995.02170230033004. PubMed: 7550809.
- 60. Herbst CM, Tekin E (2009) Child care subsidies and childhood obesity. Bonn: Forschungsinstitut zur Zukunft der Arbeit ( IZA) Institute for the Study of Labor. 41 p.
- 61. Hawkes C (2012) Food taxes: what type of evidence is available to inform policy development? Nutr Bull 37: 51-56. doi:https://doi.org/10.1111/j.1467-3010.2011.01949.x.
- 62. Kahneman D (2011) Thinking Fast and Slow. New York: Farrar, Strauss and Giroux. 512pp.
- 63. Medical Research Council (2011) Using natural experiments to evaluate population health interventions: guidance for producers and users of evidence. London: Medical Research Council. 29pp.
- 64. Kelly MP, Moore TA (2011) Methodological, Theoretical, Infrastructural and Design Issues in Conducting Good Outcome Studies. Rev Soc Works Practice 21: 644-653. doi:https://doi.org/10.1177/1049731511405070.
- 65. Capewell S, Graham H (2010) Will Cardiovascular Disease Prevention Widen Health Inequalities? PLOS Med 7: e1000320. PubMed: 20811492.
- 66. Brunton G, Stansfield C, Thomas J (2012) Finding Relevant Studies. In: D. GoughS. OliverJ. Thomas. An Introduction to Systematic Reviews. London: Sage Publications Ltd.. pp. 107-134.