Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Using a Marginal Structural Model to Design a Theory-Based Mass Media Campaign

Using a Marginal Structural Model to Design a Theory-Based Mass Media Campaign

  • Hiromu Nishiuchi, 
  • Masataka Taguri, 
  • Yoshiki Ishikawa



The essential first step in the development of mass media health campaigns is to identify specific beliefs of the target audience. The challenge is to prioritize suitable beliefs derived from behavioral theory. The purpose of this study was to identify suitable beliefs to target in a mass media campaign to change behavior using a new method to estimate the possible effect size of a small set of beliefs.


Data were drawn from the 2010 Japanese Young Female Smoker Survey (n = 500), conducted by the Japanese Ministry of Health, Labor and Welfare. Survey measures included intention to quit smoking, psychological beliefs (attitude, norms, and perceived control) based on the theory of planned behavior and socioeconomic status (age, education, household income, and marital status). To identify suitable candidate beliefs for a mass media health campaign, we estimated the possible effect size required to change the intention to quit smoking among the population of young Japanese women using the population attributable fraction from a marginal structural model.


Thirteen percent of study participants intended to quit smoking. The marginal structural model estimated a population attributable fraction of 47 psychological beliefs (21 attitudes, 6 norms, and 19 perceived controls) after controlling for socioeconomic status. The belief, “I could quit smoking if my husband or significant other recommended it” suggested a promising target for a mass media campaign (population attributable fraction = 0.12, 95% CI = 0.02–0.23). Messages targeting this belief could possibly improve intention rates by up to 12% among this population. The analysis also suggested the potential for regulatory action.


This study proposed a method by which campaign planners can develop theory-based mass communication strategies to change health behaviors at the population level. This method might contribute to improving the quality of future mass health communication strategies and further research is needed.


Mass media health campaigns are effective at influencing health behavior at the population level [1]. A recent systematic review showed that this is particularly the case for reducing smoking and secondhand smoke [2]. However, such campaigns can also have unfavorable effects on behavior [3]. For example, in Japan, a national youth antidrug media campaign resulted in increasing marijuana initiation and the “Think. Don’t Smoke” campaign may have inadvertently promoted tobacco smoking [46]. Given their potential to influence a large number of people, mass media communication strategies need to be carefully developed.

An essential first step in developing such strategies is to identify which determinants of the targeted health behavior are changeable by communication and have the greatest likelihood of effecting behavioral change [7]. Although the mechanisms of behavioral change are usually complex and involve several interacting components [8], behavioral theories can provide a framework to help identify beliefs that can be addressed through persuasive communication. Evaluating whether such beliefs are changed by health campaigns can allow us to understand why and how mass media health campaigns succeed or fail and will contribute to the future development of communication strategies.

Research on behavioral theory over the past century has suggested a number of determinants of health behavior. Careful consideration of these determinants suggests that only a limited number need to be considered in predicting and explaining a substantial proportion of variance in any behavior in any population [7,910]. These theories suggest that behavior is driven by intention. Past meta-analyses of the relationship between intention and behavior in prospective studies reveal that intentions account for 28% of the variance in behavior [11]. The implementation intention [12] (i.e., “I intend to do X in situation Y”) appears to be the key determinant of actual behavior [11].

Past research reveals that the intention to perform the behavior is based on three belief categories: (1) attitudes, (2) norms, and (3) perceived control (or self-efficacy) [7] based on the theory of planned behavior developed by Fishbein and Ajzen and applied to understand various health behaviors [13]. These are the specific beliefs that serve as the targets for intervention efforts, as they ultimately determine intention and behavior. Therefore, it is necessary to identify which attitudinal, normative, or control beliefs are the best candidates to address through persuasive communication. Given the variety of beliefs, however, the challenge for the development of mass communication strategies is to identify a more limited set of beliefs to serve as the focus for intervention efforts.

The simple approaches used in previous studies to evaluate effect included Pearson’s correlation coefficient or a single regression coefficient [13]. However, these methods can lead to overestimation of the effect of the identified belief on change in intention, because mass media campaigns usually use one message to address diverse audiences, while one-to-one communication can use messages tailored to each person. The correlation or regression coefficient used in the earlier work suggests the possible effect size at a personal level. However, an effective message at a personal level may not translate into an effective mass media campaign because the people for whom this message is effective might be very few in number and others might not be convinced. Hornik and Woolf [14] therefore proposed that a cross-sectional survey could be used to identify target beliefs that meet the following criteria: 1) the belief should predict who will or will not engage in the target behavior; 2) there should be enough people who do not already hold the belief to make the intervention worthwhile; and 3) the belief should be changeable by persuasive communication. Assessment of the correlation coefficient or single regression coefficient is only a rough indicator of the first of these and Hornik and Woolf noted that we should also consider the population with unfavorable or without favorable beliefs. They recommended evaluation of the possible effect size using the product of the difference in the rate of behavioral intention between people with unfavorable and favorable beliefs and the proportion of people with the unfavorable belief in the whole of the target population. This can be calculated using a simple cross-tabulation and similar concept appears in what epidemiologists call population-attributable fraction (PAF); epidemiologists have also proposed a more accurate method to estimate PAF, which controls for various confounders [15, 16]. Having a belief or a behavioral intention is influenced by individual characteristics, such as age, sex, and socioeconomic status, such that the baseline characteristics of those with the belief in question are likely to differ from those without. We therefore need to control for these differences when identifying the target belief for mass media health campaigns, or be unable to distinguish a significant difference in behavioral intention related to a changeable belief, and an apparent difference related to socioeconomic status.

The purpose of this study was to estimate an indicator of a mass media campaign’s effect on behavioral intention among a small set of beliefs. In this study, we aim to improve upon Hornik and Woolf’s approach by controlling for confounders, and to predict the extent of a campaign’s effect, providing a standard error and confidence interval (CI)of the possible effect size.


Data Set

Data were drawn from the 2010 Japanese Young Female Smoker Survey, which called 20–30 Dai Kituen Josei No Segmentation Ni Kansuru Chousa in Japanese [17]. This cross-sectional survey was conducted by the Ministry of Health, Labour and Welfare in Japan using an Internet-based research organization. Young females were the only demographic segment with increasing smoking prevalence in Japan, while other demographic segments showed decreased smoking prevalence; therefore, the government conducted a survey using careful formative evaluation research based on the theory of planned behavior. The Internet-based research organization has one of the largest research panels, with approximately 1,460,000 voluntarily registered subjects. After identifying age and smoking status of the registered subjects, 3,603 potential respondents, who were all female smokers aged 20–39 years, were randomly selected from the panel and invited via email to complete an Internet-based survey. Five hundred people completed the questionnaire between March 13 and 14, 2010, which was a response rate of 13.9%. Anonymous data were provided to the researchers.

Survey measures included implementation intention to quit smoking, psychological beliefs (attitude, norms, and perceived control) and baseline characteristics. Implementation intention to quit smoking was assessed as a dichotomous (yes/no) variable with the statement “Do you have a plan to quit smoking?”

Psychological Beliefs

The 2010 Japanese Young Female Smoker Survey used measures on psychological variables developed through formative elicitation research [13], which was conducted with 11 smokers. In total, 21 beliefs on attitude, 6 on norm, and 19 on perceived control were assessed as dichotomous (yes/no) variables and Table 1 presents the measures for each psychological belief.

Baseline Characteristics

Characteristics measured were age (20–29, 30–39 years), education (less than high school, high school graduate, some college or technical school, and college graduate or more), household income (< 4 million JPY, 4 to <8 million JPY, and ≥8 million JPY), employment status (unemployment, irregular employment, or regular employment) and marital status (married or not married).

Statistical Analysis

As mentioned previously, we can estimate the effect of a mass media campaign on behavioral intention using the product of 1) ΔY, the difference in the rate of behavioral intention between people with unfavorable and favorable beliefs; and 2) p, prevalence of unfavorable beliefs in the whole of the target population. ΔY × p is an indicator based on Hornik and Woolf’s [14] criteria, which is the numerator of what epidemiologists call PAF. We call this indicator population-attributability (PA).

Though we can estimate both ΔY and p easily from cross tabulation using cross-sectional data, they considered neither error nor confounders. Therefore, we proposed estimation of the standard error using the delta method under the assumption of (approximate) independence as follows:

Moreover, we can obtain the 95% CI of the PA by calculating and p-value, given asymptotic normality.

In addition, we also proposed estimation of the adjusted controls for various confounders such as socioeconomic status with the marginal structural model (MSM) [18, 19]. The adjusted and the standard error can obtain the regression coefficient and the robust standard error of MSM weighted by propensity score estimated from confounders such as age, education, employment status, household income, and marital status. SAS software (v. 9.3; SAS Institute, Cary, NC) was used for all statistical analyses. The estimation details are shown in the Appendix.

Ethical Considerations

The Japanese Ministry of Health, Labour and Welfare conducted the original survey using an Internet panel. Informed consent was obtained by filling out and submitting a Web form that we cannot access. Our research uses secondary analysis of anonymous data. The Office for Life Science Research Ethics and Safety, at the University of Tokyo approved our research (approval number: #15–1).


Descriptive Data

In total, 13% of the study participants had an implementation intention to quit smoking. Twenty-nine percent of participants were 20–29 years old, and 71% were 30–39 years old. Table 1 shows the characteristics of the study participants.

Population Attributable Fraction and Confidence Interval

Table 2 shows the PA for each belief obtained using Hornik and Woolf’s method [14]. The top five beliefs, by magnitude of PA, were: (1) I could quit smoking if my workplace became entirely smoke-free (PA = 0.27, 95% CI = 0.12–0.41); (2) I could quit smoking if places like transportation facilities, government offices, schools, and hospitals became entirely smoke-free (PA = 0.27, 95% CI = 0.06–0.47); (3) I could quit smoking if my close friends recommended it (PA = 0.23, 95% CI = 0.05–0.41); (4) I feel that it is foolish to ignore the warnings given to me about the dangers of smoking (PA = 0.17, 95% CI = 0.01–0.32; and (5) I could quit smoking if restaurants and amusement centers became entirely smoke-free (PA = 0.16, 95% CI = 0.04–0.28).

Table 3 shows the PA for each belief using the MSM and controlling for socioeconomic status. The top five beliefs, by magnitude of PA, were: (1) I could quit smoking if places like transportation facilities, government offices, schools, and hospitals became entirely smoke-free (PA = 0.21, 95% CI = 0.01–0.41); (2) I could quit smoking if my workplace became entirely smoke-free (PA = 0.19, 95% CI = 0.04–0.33); (3) I could quit smoking if restaurants and amusement centers became entirely smoke-free (PA = 0.16, 95% CI = 0.01–0.31); (4) I could quit smoking if my husband or significant other recommended it (PA = 0.12, 95% CI = 0.02–0.23); and (5) I think people think I am foolish because I ignore the harm caused by smoking (PA = 0.11, 95% CI = 0.03–0.20).

Table 3. Population-Attributable Fraction Using the Marginal Structural Model.


Not all mass media health campaigns have a positive impact on health behavior [3]. Therefore, it is important that campaign strategy development be based on behavioral theory to identify the critical beliefs that will play an important role in changing intention, rather than a more arbitrary development method [7]. Despite this, methods for selecting target beliefs for mass media health campaigns have received little attention since Hornik and Woolf [14].

The most important contribution of our study is to propose a new method, and demonstrate its use to identify a small set of critical beliefs that will lead to a greater possibility of changing behavioral intention. Our method, based on the MSM, identified slightly different target beliefs from Hornik and Woolf’s model, because we controlled for socioeconomic status. Though the differences between the two methods were very small in this study, the differences might be more important in other cases, for example, when designing mass media campaigns to target a more diverse population with more age groups, broader ethnicity and nationality or larger disparity of income and education. Though there were limited beliefs showing statistically significant PA by MSM, the belief “I could quit smoking if my husband or significant other recommended it” provides a reasonable primary target for a communication strategy, given the size of PA and the possibility of change by communication. Campaign planners could target young men, rather than young female smokers, to persuade them to advise their partners to stop smoking. This belief could not be identified using Hornik and Woolf’s method [14] because of confounding, and a mass media message targeting this belief is expected to achieve more behavioral change than other beliefs with less significant PAs. It is hoped that this method will help campaign planners to translate findings from behavioral research into daily practice. However, it is important to note that identifying the target belief does not necessarily lead to development of a suitable campaign message. Communication theory, rather than behavioral theory, should guide this aspect of campaign planning [7]. To develop effective communication strategies, further research will be required to clarify how messages that can change the identified target beliefs may be crafted.

The second important implication of our study is to identify critical beliefs that may be the target of regulatory policy. For example, the top three beliefs by magnitude of PA identified by MSM were “I could quit smoking if places like transportation facilities, government offices, schools, and hospitals became entirely smoke-free,” “I could quit smoking if my workplace became entirely smoke-free,” and “If restaurants and amusement centers became entirely smoke-free, I could quit smoking.” These aspects are not susceptible to persuasive communication, but could be changed by regulatory policy. Therefore, our method could inform not only mass media health campaign planners but also policy makers. The results also suggest the possibility of targeting mass media campaigns at administrators of public facilities, offices, and restaurants, rather than young female smokers.

This study has several limitations. First, we did not consider “priming,” which is an alternative communication strategy [7]. Hornik and Woolf [14] suggested that a majority of people should not hold the target belief for persuasive communication, but “priming” or enforcing existing beliefs held by a majority may be a suitable alternative strategy for communication design. It could be argued that emphasizing existing beliefs related to behavioral intention would increase the likelihood that they would be considered. Future research will be required to develop and examine methods to estimate the effect size and its CI by “priming” the target beliefs. Second, the estimated effect size may not be very close to the causal effect size, because the propensity score method cannot control for unknown or unmeasured confounders. The data set used in this study had very limited items thought to be potential confounders; therefore, and there might be others that were not identified. Although outside the scope of our research, using sensitivity analyses for unmeasured confounding [20] would help campaign planners understand the limitations of the estimation in effect size. Third, the data set only includes young Japanese female participants who could respond to Internet-based surveys. This might be biased population and our result could not be applicable to other population. Fourth, our proposed method did not consider the equality point of view. Changing the target belief at population level through a mass media health campaign might be counterproductive for a specific subgroup. Some researchers have argued that the population approach can be detrimental for vulnerable groups [21]; therefore, future research should explore and develop methods to identify suitable beliefs leading to a greater likelihood of reducing disparities in behavioral intention.

In summary, this study proposed and simulated a method to identify limited sets of theory-based beliefs to serve as targets for persuasive communication in a mass media health campaign. We hope this method will help to improve the quality of mass communication strategies by improving targeting.


Let A be the indicator variable of an exposure or unfavorable belief such as “Smoking is fun” and Y be an observed dichotomous outcome or behavioral intention. Y = 1 in this study means that the subject intends to quit smoking. X represents multicategory confounding factors such as age, education, employment status, household income, and marital status. Using the potential outcome model [22], Ya (a = 0,1) represents the counterfactual outcomes that would have been observed under exposure category a. Our objective is to estimate the possible reduction in the probability of the outcome under the ideal condition, in which any unfavorable belief obstructing the behavioral intention is completely removed. Using a formula, we wish to estimate population-attributability (PA) as follows: (1)

The last expression of Eq 1 is the product of the causal risk difference in the exposed group and the exposure probability. Unfortunately, Pr(Y0 = 1 | A = 1) in Eq 1 is not estimable without assumptions because Y0 is not observed in the exposed group. If exposure A is randomized effectively, we can expect the following unconditional exchangeability assumption to hold [23]: YaA. Under this assumption and the usual consistency assumption, which means YA = Y, Eq 1 can be re-expressed as: (2)

Expression (2) is the product of the unadjusted risk difference and the exposure probability, which we called ΔY and p in the body text. By replacing the probabilities in Eq 2 by sample proportions, we obtain an estimator of Eq 2, which is identical to that proposed by Hornik and Woolf [14].

However, in a cross-sectional survey, the assumption of unconditional exchangeability may not apply because of confounding. Instead, we make the following conditional exchangeability assumption [19]: YaA|X. This implies that there is no unmeasured confounding within levels of the measured confounders. Under this assumption, we can estimate the causal risk difference (in the exposed group) in (1) by using the marginal structural model (MSM) [18, 19]. This involves three steps:

  1. Fit the regression model for the association of confounder–exposure relationship. For instance, fit the following logistic regression model: (3) using standard software such as SAS (SAS Institute, Cary, NC).
  2. Using the predictive value of the fitted regression model (3), calculate weights for each individual i in the unexposed group (A = 0). wi = 1 for the exposed group (A = 1).
  3. Estimate causal parameters of the following MSM: (4) by fitting the association model Pr(Y = 1 | A = a) = γ0 + γ1a with weights wi.

From Eq 4, the proposed estimator of Eq 1 is obtained by: (5)

Similar estimators have been proposed in the context of estimation of the population attributable fraction (PAF) [1516]. The PAF proposed in the earlier study could be presented as: (6)

To estimate the confidence interval (CI) of the PA, we assumed (approximate) independence between and . and the robust standard error were estimated with SAS’s GENMOD procedure using the weight wi calculated by the logistic procedure. and the standard error can be estimated easily from the data step. Under the assumption of independence and based on the delta method, the standard error of the proposed estimator is (7)

Given asymptotic normality, we can obtain the 95% CI of by calculating ±1.96 SE, and the p-value. This also applies to Hornik and Woolf’s simple cross-tabulation. SAS v.9.3 (SAS Institute, Cary, NC) was used for all statistical analyses.


We would like to thank the Ministry of Health, Labour and Welfare for making the 2010 Japanese Young Female Smoker Survey data available.

Author Contributions

Analyzed the data: HN MT. Wrote the paper: HN MT YI.


  1. 1. Wakefield MA, Loken B, Hornik RC. Use of mass media health campaigns to change health behavior. Lancet. 2010; 376;9748: 1261–1271.
  2. 2. Guide to Community Preventive Services. Health communication and social marketing. Last updated: March 19, 2014.
  3. 3. Cho H, Salmon CT. Unintended effects of health communication campaigns. J Commun 2007;57: 293–317.
  4. 4. Hornik R, Jacobsohn L, Orwin R, Piesse A, Kalton G. Effects of the national youth anti-drug media campaign on youths. Am J Public Health. 2008;12: 2229–2236.
  5. 5. Wakefield M, Terry-McElrath Y, Emery S, Saffer H, Chaloupka FJ, Szczypka G, et al. Effect of televised, tobacco company–funded smoking prevention advertising on youth smoking-related beliefs, intentions, and behavior. Am J Public Health 2006;12: 2154–2160.
  6. 6. Farrelly MC, Healton CG, Davis KC, Messeri P, Hersey JC, Haviland ML. Getting to the truth: evaluating national tobacco countermarketing campaigns. Am J Public Health 2002;6: 901–907.
  7. 7. Fishbein M, Cappela JN. The Role of Theory in Developing Effective Health Communications. J Commun 2006;S1: 1–17.
  8. 8. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008;337: a1655. pmid:18824488
  9. 9. Fishbein M. The role of theory in HIV prevention. AIDS Care 2000;12: 273–78. pmid:10928203
  10. 10. Institute of Medicine. Speaking of Health: Assessing Health Communication Strategies for Diverse Populations. Washington, DC: The National Academies Press; 2002.
  11. 11. Sheeran P. Intention-behavior relations: A conceptual and empirical review. Eur Rev Soc Psychol. 2002: 1–36.
  12. 12. Gollwitzer PM. Goal achievement: The role of intentions. Eur Rev Soc Psychol. 1993: 141–185.
  13. 13. Montaño DE, Kasprzyk D. Theory of reasoned action, theory of planned behavior, and the integrated behavioral model. In Glanz K, Rimer B, Viswanth K, editors. Health behavior and health education: Theory, research, and Practice. San Fransico: Jossey-Bass; 2015, pp. 95–124.
  14. 14. Hornik R, Woolf KD. Using cross-sectional survey to plan message strategies. Soc Mar Q 1995;2: 24–41.
  15. 15. Sjölander A. Estimation of attributable fractions using inverse probability weighting. Stat Methods Med Res 2011;20: 415–428. pmid:20223785
  16. 16. Taguri M, Matsuyama Y, Ohashi Y, Harada A, Ueshima H. Doubly robust estimation of the generalized impact fraction. Biostatistics 2012;13: 455–67. pmid:22084301
  17. 17. Communication No Tebiki. Sukoyaka Seikatu Shukan Kokumin Undou. Available: Cited: June 7,2016. Japanese.
  18. 18. Robins JM, Hernán MA, Brumback B. Marginal structural model and causal inference in epidemiology. Epidemiology 2000;11: 550–560 pmid:10955408
  19. 19. Sato T, Matsuyama Y. Marginal structural model as a tool for standardization. Epidemiology 2003;14: 680–686. pmid:14569183
  20. 20. Brumback BA, Hernan MA, Haneuse S, Robins JM. Sensitivity analyses for unmeasured confounding assuming a marginal structural model for repeated measures. Stat Med 2004;23: 749–767. pmid:14981673
  21. 21. Frohlich KL, Potvin L. Transcending the known in public health practice: the inequality paradox: the population approach and vulnerable populations. Am J Public Health 2008;2: 216–221.
  22. 22. Rubin DB. Inference and missing data. Biometrika 1976;68: 581–92.
  23. 23. Hernán MA, Robins JM. Estimating causal effects from epidemiological data. J Epidemiol Community Health 2006; 60: 578–586. pmid:16790829