http://orcid.org/0000-0002-4416-2736
Figures
Abstract
Background
Despite offering little overall benefit and emerging concerns about their safety, dietary supplements have become increasingly popular. Trust in advertising them may contribute to high confidence in dietary supplements in public opinion.
Aim
To develop and validate a screening questionnaire intended for the general public regarding knowledge about dietary supplements and a questionnaire on trust in advertising dietary supplements, and to identify the association between these constructs.
Materials and methods
The development and validation of the measures was overseen by the panels of experts. The conceptual frameworks of the constructs were scientifically well grounded. A set of semi-structured interviews and anonymous web-based surveys was performed. The final questionnaire was applied to 220 non-medically educated people and 121 medically educated people.
Results
A 17-item questionnaire on knowledge about dietary supplements and eight-item questionnaire on trust in advertising dietary supplements were developed. The measures presented satisfactory proof of validity, however, the psychometric properties of the questionnaire on knowledge were modest. Both the knowledge about dietary supplements in the study group and trust in advertising them were low. A significant negative relationship was found between knowledge about dietary supplements and trust in advertising them among the general public (Pearson’s r = -0.42, 95%CI: -0.52 to -0.30, p<0.0001). This association was especially pronounced in people who reported not taking dietary supplements (Pearson’s r = -0.61, 95%CI: -0.76 to -0.39, p<0.0001).
Citation: Karbownik MS, Paul E, Nowicka M, Nowicka Z, Kowalczyk RP, Kowalczyk E, et al. (2019) Knowledge about dietary supplements and trust in advertising them: Development and validation of the questionnaires and preliminary results of the association between the constructs. PLoS ONE 14(6): e0218398. https://doi.org/10.1371/journal.pone.0218398
Editor: Ali Montazeri, Iranian Institute for Health Sciences Research, ISLAMIC REPUBLIC OF IRAN
Received: February 12, 2019; Accepted: June 1, 2019; Published: June 24, 2019
Copyright: © 2019 Karbownik et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The dataset files are available from the Mendeley Data repository: http://dx.doi.org/10.17632/h8f4t3ktd5.1, http://dx.doi.org/10.17632/nfw68tgw82.1.
Funding: This work was supported by the Medical University of Lodz, Poland (https://umed.pl/) with the grant received by TP (503/1-151-07/503-11-001). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: I have read the journal's policy and the authors of this manuscript have the following competing interests: EP is employed in OSOM STUDIO, an e-marketing agency. This does not alter our adherence to PLOS ONE policies on sharing data and materials. OSOM STUDIO had no role in study design, data collection and analysis, data interpretation, decision to publish, or preparation of the manuscript.
Introduction
Dietary supplements (DS) have become increasingly popular following inter alia the seminal work on “vitamins” by Kazimierz Funk [1] and the advocacy of Linus Pauling [2], a double Nobel Prize laureate. DS occupy a well-established and scientifically proven position in correcting malnutrition [3]. However, in high-income countries, DS are also perceived as offering hope for generally healthy people for warding off multiple civilization diseases, extending life expectancy, improving life quality, and even allowing human enhancement [4, 5]. Unfortunately, many of these claims have not been supported by the findings of large-scale high-quality studies, the majority of which found DS to have negligible effects [2, 6, 7]. Vitamin C was found not to protect from common cold in the general community [8], omega-3 fatty acids from cardiovascular events [6, 9], nor antioxidants from cancer [10, 11]. The latter may even have harmful effects [11, 12].
Despite such disappointing data, high confidence in dietary supplements persists among the general public [5, 13]. This persistence may be due to the prevalence of aggressive advertising [14–16] or, more specifically, the extent to which people trust advertising [17, 18]. Consequently, extensive advertising may lead to an accumulation of misleading information [19]. Yet, the economic and social development of a knowledge-based society demands the exchange of correct facts and well-established knowledge [20]. In this light, testing the hypothesis about the link between knowledge about DS and trust in their advertising appears to be of paramount importance. It may help to shape more prudent advertising strategies for DS, make people better informed about them, and contribute to improving health.
The aim of the current study was to report the development and validation of two psychometric tools to measure the above-mentioned constructs: knowledge about dietary supplements and trust in advertising them. The tools will be used in further research to examine the hypothesis about the association between these constructs in detail. Moreover, the aim of the present study is to report preliminary data regarding such association.
Materials and methods
In the present study, “dietary supplements” were defined as “foodstuffs the purpose of which is to supplement the normal diet and which are concentrated sources of nutrients or other substances with a nutritional or physiological effect, alone or in combination, marketed in dose form”. The definition is consistent with Directive 2002/46/EC of the European Parliament and of the Council on the approximation of the laws of the Member States relating to food supplements, which also applies in Poland [21].
The study was carried out in agreement with the latest version of Declaration of Helsinki. It was approved by the Bioethics Committee of the Medical University of Lodz (number RNN/253/18/KE received on 10 July 2018). All the questionnaires were gathered from mid-July to December 2018 with written or electronic informed consent received each time.
The process of questionnaires development was led by the guidelines reported [22, 23], and by papers describing the development of similar questionnaires [17, 24, 25].
Development and validation of the questionnaire on knowledge about dietary supplements
The development of the questionnaire on knowledge is illustrated in the flowchart given in Fig 1. The questionnaire was prepared in four stages: setting the conceptual framework, development, pre-testing and testing. More details on the methods used in its development are included in S1 Supporting Information.
Conceptual framework.
In order to assure the validity and reliability of testing “knowledge about dietary supplements", abbreviated to KaDS, a tool was developed and validated by a panel of experts. This panel included a pharmacist with academic experience, a pharmacist with clinical experience and a medical doctor with the title of Full Professor. The panel first developed a conceptual framework for the KaDS distinguishing its “general” and “specific” domains. The “general” KaDS was defined as “familiarity with the useful facts concerning legal status of dietary supplements in general”, and the following concepts of this domain were pointed out: 1) definition and 2) registration requirements. On the other hand, the “specific” KaDS was defined as “familiarity with common, scientifically proven and useful facts about popular dietary supplements”, and the following concepts of this domain were highlighted: 1) efficacy (with special attention to this category), 2) safety, 3) pharmacokinetics, and 4) interactions with medicines. The final measure was expected to be a concise questionnaire suitable for screening purposes rather than a detailed evaluation and decision-making tool to examine individual respondents; therefore, the final KaDS questionnaire was restricted to a maximum of 10–20 items.
Development.
The development of the KaDS questionnaire consisted of several steps:
- Pre-development. A series of pre-development semi-structured and structured interviews was performed with pharmacists and people with no medical education to establish the most commonly-purchased classes of DS, the ingredients which are the most identifiable by the public, and to collect common beliefs about DS.
- Questionnaire formatting. It was decided that each question stem will be presented as a declarative sentence, and that the response format will be of a binary nature (true/false).
- Item generation. Based on the results of the pre-developmental interviews, and a rigorous review of legal documents and scientific and medical literature, 100 statements were drawn up: 20 “general” and 80 “specific”. The statements were reviewed by a specialist in Polish linguistics and further modified according to her suggestions.
- Item reduction. A group of nine independent competent judges were asked to evaluate each statement in terms of its relevance and its value to public knowledge. Based on their opinion, 59 less relevant statements were deleted [22], and the remaining 41 were passed to the next step of the development process.
- Item understanding. The retained 41 statements were tested with regard to their degree of understanding by the public [26]. The vast majority of the statements were understood as assumed by the panel of experts, but four required further clarification and were modified accordingly.
Pre-testing.
The aims of the pre-testing stage were to identify the statements which are less familiar to the general public and to obtain data on face validity. The 41-item draft of the KaDS questionnaire was prepared in a web-based form (Google Forms, Google, Mountain View, CA, USA) and administered to a sample of 57 people with no medical education. An additional “don’t know” option was added to the true/false response format.
Four statements received more than 50% “don’t know” answers; these were hence assessed as unfamiliar to the public, and rejected from the questionnaire. The remaining 37 statements were retained. Feedback provided by the respondents indicated that the KaDS questionnaire was clear and comprehensive.
Testing.
The testing stage assessed the psychometric characteristics of the items and pruned those with less psychometric relevance. The 37-item draft of the KaDS questionnaire, obtained in the pre-testing stage, was again prepared in a web-based form (Google Form) with true/false response format, but without the “don’t know” option. This draft was combined with the recently-developed eight-item measure of “trust in advertising dietary supplements” (see below) to assess the association between the KaDS and trust in advertising DS. The survey was administered to a sample of 220 people with no medical education. Both the original and English language versions of the survey are available in S1 Survey.
Keeping the conceptual framework of the scale in mind, the questionnaire was refined with regard to the following psychometric properties:
- item difficulty–excessively difficult (> 90% of wrong answers) and easy (< 10% of wrong answers) statements were excluded [23, 27, 28],
- item discrimination–statements with an item discrimination (measured as a corrected item-total correlation with assumed subscales) of less than 0.20 were excluded [23],
- maximization of internal consistency [29],
- balancing the number of “true” and “false” correct answers [30] and content reanalysis.
The final version of the KaDS questionnaire consisted of 17 items. Exploratory and confirmatory factor analysis was performed to assess the consistency of the final questionnaire structure. To ensure the criterion validity of the final KaDS questionnaire, it was also applied to 121 medically educated people (healthcare practitioners and medical students). Criterion validity was assessed by comparing the mean test score between medically and non-medically educated people. Additionally, the correlation between the test score and a year of study was checked among a group of medical students.
The data of 220 responses of non-medically educated people and 121 responses of medically educated people for the testing of the KaDS questionnaire was deposited in Mendeley Data repository (http://dx.doi.org/10.17632/h8f4t3ktd5.1).
Development and validation of questionnaire on trust in advertising dietary supplements
The development of the questionnaire on trust is illustrated in the flowchart given in Fig 2. The questionnaire was prepared in four stages: setting the conceptual framework, development, pre-testing and testing. More details on the methods used in its development are included in S2 Supporting Information.
Conceptual framework.
A panel of experts was formed to develop a tool for testing “trust in advertising dietary supplements", abbreviated to TiADS, and ensure its validity and reliability. The panel included a psychologist with academic experience, a psychologist with clinical experience, and two marketing specialists. The panel first defined advertising dietary supplements as “any paid form of non-personal presentation and promotion of goods, services, or ideas (i.e. dietary supplements) by an identified sponsor” [31]. Various sources of advertising were included, inter alia television, radio, internet/mobile and newspapers/magazines. The panel then reviewed the popular Polish and English language dictionaries as well as psychological and marketing research literature [17, 32–34] to define the construct of TiADS as “a belief that advertising is a credible source of information about dietary supplements, arouses emotions and willingness to act on the basis of information conveyed by advertisements”. This definition is consistent with prior conceptualizations and identifies the cognitive, behavioral and emotional domains of the construct.
Development.
The development of the TiADS questionnaire consisted of several steps:
- Pre-development. In-depth lexical analysis of popular Polish and English language dictionaries as well as rigorous review of psychological and marketing research literature was performed to further explore the meaning of TiADS domains [17, 32–34].
- Questionnaire formatting. It was decided that a semantic differential scale will be used. The scale points were numbered from 1 (distrust) to 5 (trust).
- Item generation. In addition to the instructions for using the questionnaire, the panel of experts developed 12 items consisting of opposing adjectives concerning the information conveyed by the DS advertisements (cognitive domain), and eight items consisting of opposing expressions characterizing DS advertisements themselves (four belonging to the behavioral domain and four to the emotional domain). These 20 items were further reviewed by a specialist in Polish linguistics and modified according to her suggestions.
- Non-professional item review and understanding. Five non-specialists were recruited to review the developed items. They were asked to describe in their own words how they understood each item and were interviewed on how much they believed each item was relevant to DS advertising. Moreover, the respondents were asked to develop their own items according to the guidelines provided. The items not understood in the assumed way or assessed as irrelevant/redundant to DS advertising were deleted, whereas the items developed by non-specialists, and which were graded by the panel of experts as at least satisfactorily suited to the definition of the construct were added. This step resulted in retaining 19 items.
- Professional item review. Afterwards, eight independent competent judges were also asked to review the TiADS questionnaire. The judges were acquainted with the conceptual framework of the TiADS construct and asked to evaluate each statement in terms of its relevance to DS advertising. Irrelevant items were deleted. As a result of this review, 11 items were retained in the final draft of TiADS questionnaire.
Pre-testing.
Questionnaire pre-testing was performed to assess the structure of the TiADS construct and prune the psychometrically less suitable items. The 11-item draft of the TiADS questionnaire was administered in a web-based platform (Google Forms) to 174 respondents. The choice of the items to be retained in the final TiADS questionnaire was based on exploratory factor analysis with subsequent internal consistency assessment [29].
This stage also evaluated the criterion and concurrent validity of the final TiADS measure. Criterion validity was assessed by comparing the results of the TiADS questionnaire with those of the ADTRUST scale. ADTRUST is a 20-item questionnaire developed by Soh et al. [17] to measure “trust in advertising” in general. Concurrent validity was assessed by comparing the TiADS scores with the results of a single-item measure of “trust in advertising dietary supplements”. Both the additional measures were included to the web-based survey subjected to 174 respondents.
As a result of item pruning, the final TiADS questionnaire consisted of eight items. The data of for this stage of the TiADS questionnaire analysis was deposited in Mendeley Data repository (http://dx.doi.org/10.17632/nfw68tgw82.1).
Testing.
During this stage, the final eight-item TiADS questionnaire was externally validated in a separate sample of respondents reflecting the target population. Confirmatory factor analysis was used to verify the established before structure of the final TiADS questionnaire. The TiADS questionnaire was combined with the KaDS test (see above) for preparation for online distribution (Google Forms). This survey was applied to 220 people with no medical education as well as to 121 medically-educated people (see above). Both the original and English language versions of the survey are available in S1 Survey.
The data for testing the TiADS questionnaire was deposited in Mendeley Data repository (http://dx.doi.org/10.17632/h8f4t3ktd5.1).
Other tested variables
The questions on the following sociodemographic variables were included to the web-based tests throughout the study: sex (operationalized as a categorical variable: 0 = male, 1 = female), age (operationalized as a continuous variable), education level (operationalized as an ordinal variable: 1 = primary, 2 = secondary, 3 = higher-bachelor, 4 = higher-master, 5 = higher-doctorate) and perceived socioeconomic status (operationalized as an ordinal variable: 1 = low, 2 = middle, 3 = high). Respondents were also asked “whether they take DS” (the variable operationalized as an ordinal: 0 = not taking, 1 = taking from time to time, 2 = taking every day) and “whether they had contact with DS advertisements within the past week” (the variable operationalized as an ordinal: 0 = no, 1 = yes).
Data analysis
Descriptive statistics involved mean ± standard deviation or mean and 95% confidence intervals (95%CI), if not stated otherwise. Although the Likert and semantic differential scale data should be perceived as ordinal variables, the analyses in this study were performed with parametric tests such as Pearson’s r (further referred to as “r”), Student’s t-test and other general linear model (GLM) analyses. This was to allow for multivariate modeling. Each time, if possible, however, corresponding non-parametric tests were performed and they all yielded very similar results to the parametric ones. Adjustment for potential confounders was performed by including all the covariates to the GLM, linearly linked to dependent variable. Testing multiple hypotheses was not a subject for significance level correction because of the explorative and preliminary nature of the research. The proportion of missing data in the analyzed databases were acceptably low: 0.60% missing data in the database of 57 responses (KaDS pre-testing), 0.42% missing data in the database of 220 responses (KaDS and TiADS testing in non-medically educated people), 0.47% missing data in the database of 121 responses (KaDS and TiADS testing in medically educated people), 0.73% missing data in the database of 174 responses (TiADS pre-testing). Because of low proportions of missing values, all the analyses were based on complete cases only. P-values were presented with two significant figures and rounded to maximum four decimal places. P-values below 0.05 were considered statistically significant. The analysis was performed using STATISTICA 13.1 Software (StatSoft, Tulsa, OK, USA).
Results
The sociodemographic characteristics of the group of 220 people with no medical education who completed the KaDS and TiADS questionnaires in their testing stages is given in Table 1.
Questionnaire on knowledge about dietary supplements
Psychometric properties of the final questionnaire.
The following psychometric results of the final KaDS test are presented in Table 2: the mean grade awarded to each of the final 17 items of the KaDS questionnaire by the competent judges during the item reduction step, familiarity of the general public with the statements assessed in the pre-testing stage, the results of the item analyses performed in the testing stage, and the results of exploratory and confirmatory factor analyses, as well as their internal consistency indices. Psychometric analysis found the structure of the KaDS questionnaire indices to be modest. Consequently, the 17-item test appears not fully adequate for detailed knowledge evaluation and decision-making in individual respondents, but may be successfully used for rapid screening purposes.
The finally developed 17-item KaDS questionnaire satisfactorily fulfils the assumed requirements of criterion validity. The mean number of correct answers for the questionnaire in the group of respondents with medical education was higher than that of no medically educated by 2.1 (95%CI: 1.4 to 2.7) points (p<0.0001, Student’s t-test). The difference remained significant also after adjustment for multiple potential confounders, such as sex, age, education, perceived socioeconomic status and taking DS (2.0, 95%CI: 1.3 to 2.8, p<0.0001; GLM with KaDS as a dependent variable, medical education and covariates as independent variables). In the group of medical students (n = 77), the scores in final KaDS questionnaire was also found to positively correlate with a year of study: r = 0.39 (95%CI: 0.19 to 0.57), p = 0.0004.
Level of knowledge about dietary supplements.
Knowledge about dietary supplements in non-medically educated people was low. The mean number of correct answers in the 17-item questionnaire was 8.3 (95%CI: 7.9 to 8.7). The highest proportion of incorrect answers was found for questions related to the efficacy and absorption of the most well-known DS: vitamin C, magnesium, vitamin D and calcium (see Table 2). This reflects the discrepancy between common beliefs and scientific facts.
Among the tested covariates in the group of people with no medical education, the KaDS total score was positively correlated with respondent’s level of education (r = 0.17, p = 0.014) and negatively correlated with the use of DS (r = -0.21, p = 0.0024). None of the other tested sociodemographic variables were correlated with KaDS total score: sex (p = 0.35), age (p = 0.37), perceived socioeconomic status (p = 0.79).
Questionnaire on trust in advertising dietary supplements
Psychometric properties of the final questionnaire.
The results of exploratory factor analysis performed in the pre-testing stage of TiADS questionnaire development (174 respondents) as well as the results of confirmatory factor analysis performed in the subsequent stage of TiADS questionnaire testing (220 respondents with no medical education) are reported in Table 3. Internal consistency measures and distribution of the responses from the testing phase are also presented there. Confirmatory factor analysis found the structure of the final TiADS to be consistent with previous findings. The model fit, non-centrality-based indices and item loadings were adequate; this supports the robustness of the results.
The eight-item TiADS questionnaire was assessed for criterion and concurrent validity in the pre-testing stage of questionnaire development. TiADS scores were found to be significantly correlated with the Polish-version ADTRUST measure (r = 0.45, 95%CI: 0.32 to 0.56, p<0.0001) and with the answer to a single question on “how much does a respondent trust the information conveyed in dietary supplement advertising” (r = 0.55, 95%CI: 0.45 to 0.66, p<0.0001). The above correlations were not significantly affected by the medical education of the respondents (p-values for the interactions between the slopes were 0.41 and 0.83, respectively; GLM with TiADS as a dependent variable and the following independent variables: ADTRUST (or single-item measure of trust), medical education–as a three-level categorical variable–and their two-way interaction). This supports criterion and concurrent validity of the final TiADS questionnaire.
Level of trust in advertising dietary supplements.
The analysis of the results for question on “trust in the information conveyed in dietary supplement advertising” (pre-testing stage of questionnaire development) in non-medically educated people only (n = 77), indicated that such trust was assessed as low, reaching a median of 3 (25th-75th percentile: 2–4) in a seven-point scale with anchors of”absolutely trust” (7)—“don’t trust at all” (1), with 4 being neutral. Only four people (5.2%) rated their trust in DS advertising as more favorable than neutral. Similar results were obtained in the testing stage (n = 220), which indicate a particularly low level of trust in its “affective” dimension (see Table 3).
The results of testing stage indicated also that TiADS is significantly associated with taking dietary supplements (r = 0.26, 95%CI: 0.13 to 0.38, p = 0.0001), but not with any of the tested sociodemographic variables: sex (p = 0.88), education level (p = 0.23) or perceived socioeconomic status (p = 0.60), indicating a trend only for older people to be more trusting (r = 0.13, 95%CI: -0.00 to 0.26, p = 0.055). Applying the TiADS questionnaire additionally to 121 medically educated people revealed that they trusted DS advertising significantly less than those without a medical education (15.9 ± 4.7 points vs 18.8 ± 5.5 points, p<0.0001, Student’s t-test), and the difference remained significant after adjustment for all above mentioned covariates (16.3 ± 5.8 points vs. 18.6 ± 5.3 points, p = 0.0004; GLM with TiADS as a dependent variable, medical education and covariates as independent variables). Both groups, i.e. medically educated and non-medically educated people, displayed similar relationships between TiADS and all tested sociodemographic variables, with age being significantly linked to trust in medically-educated people (r = 0.19, 95%CI: 0.01 to 0.36, p = 0.041). TiADS was also significantly associated with taking dietary supplements in medically-educated people (r = 0.23, 95%CI: 0.05 to 0.40, p = 0.012).
Knowledge about dietary supplements and trust in advertising them
Association between the constructs.
In the group of respondents with no medical education (n = 220), a significant negative association between KaDS and TiADS was detected: r = -0.42 (95%CI: -0.52 to -0.30), p<0.0001. The KaDS-TiADS association is illustrated in Fig 3. All the associations between particular KaDS and TiADS subscales were also significantly negative (Table 4). The significance of the association between KaDS and TiADS does not depend much on the choice of items to the final questionnaire, as a significant association was found between the sum of all 32 KaDS items considered for the final version of the questionnaire (testing stage after removal of too easy and too difficult items) and total TiADS score (r = -0.28, 95%CI: -0.40 to -0.15, p<0.0001). After splitting the sample of 220 respondents into two subgroups of low and high trust based on median TiADS score, it was found that the 17-item KaDS score of the high trust group was 2.7 (95%CI: 2.0 to 3.5) points lower than that of the low trust group. In KaDS subscale analysis, the differences were: 1.3 (95%CI: 0.7 to 1.9) points for the 7-item “general” KaDS, and 1.4 (95%CI: 1.2 to 1.6) points for the 10-item “specific” KaDS. Hence, the effect of TiADS on KaDS may be at least as high as the effect of formal medical education (see above).
Knowledge about dietary supplements was measured as a total score in the 17-item questionnaire, whereas trust in advertising dietary supplements with the eight-item questionnaire. The size of the circles represents the number of cases with the given coordinates.
The values presented include Pearson’s correlation coefficient (95% confidence intervals) and p-value. P-values for each of the comparisons between the correlation coefficients for a given KaDS-TiADS subscale were higher than 0.05.
Similar negative links between KaDS and TiADS were observed in medically-educated people (n = 121): r = -0.34 (95%CI: -0.49 to -0.17), p = 0.0002 for the KaDS total score, r = -0.18 (95%CI: -0.35 to -0.00), p = 0.049 for “general” knowledge, and r = -0.33 (95%CI: -0.48 to -0.16), p = 0.0003 for “specific” knowledge. P-values for the interactions between the corresponding KaDS-TiADS slopes of people with and without medical education were 0.68, 0.48, and 0.89 for KaDS total score, “general” KaDS, and “specific” KaDS, respectively. This similarity of the associations supports the robustness of the results.
Variables affecting the extent of KaDS-TiADS association.
Interestingly, the use of DS was found to influence the association of KaDS-TiADS in non-medically educated people. This association was significantly more negative in people who reported not taking DS (r = -0.61, 95%CI: -0.76 to -0.39, p<0.0001) than those taking DS occasionally (r = -0.30, p = 0.0011) or everyday (r = -0.37, p = 0.018). The p-value for the interaction between the slopes was 0.020, and remained low (0.0062) after adjustment for the covariates: sex, age, education and socioeconomic status (GLM with KaDS as a dependent variable and the following independent variables: TiADS, taking DS (operationalized as a continuous variable), their two-way interaction, and, optionally, all the covariates). No other tested variables were found to significantly affect the extent of the association between KaDS and TiADS (p-values for the interaction between the slopes were 0.14 for sex, 0.17 for education, 0.40 for age, 0.91 for socioeconomic status; GLM with KaDS as a dependent variable, and the following independent variables: TiADS, a sociodemografic variable and their two-way interaction).
Having recent contact with dietary supplement advertisements and knowledge about dietary supplements.
Among those without a medical education, having recent contact (within the past week) with DS advertisements was hypothesized to strengthen the potential link between TiADS and KaDS or to demonstrate its own effect on KaDS. The negative correlation between the trust and knowledge seemed to be more profound in the respondents who declared having recent contact with DS advertisements than those who did not (Table 5); however, the interaction between having recent contact with DS advertisements and TiADS was found to be significant only for “specific” domain of KaDS, which is the main content of DS advertisements [36].
The values presented include Pearson’s correlation coefficient (95% confidence intervals) and p-value.
Similarly, the analysis of covariance (a model with different slopes for “specific” KaDS only) indicated that having recent contact with DS advertisements had a significant negative effect on the “specific” KaDS score (p = 0.039); however, the effect size was extremely low (-0.04 point in the 10-item “specific” subscale). The effect of recent contact with DS advertisements on “general” KaDS and KaDS total scores was insignificant (p = 0.46 and p = 0.66, respectively).
Discussion
Despite having minor overall benefits and emerging concerns about their safety, dietary supplements (DS) are consumed by roughly a half of the adult population of developed countries [2, 15, 35, 51, 52], with Poland not being any exception [53]. It is hypothesized that aggressive DS advertising shapes false beliefs among consumers to a certain extent and, consequently, drives supplement-taking behavior [14–16, 36]. The present study reports the development and validation of questionnaires to test knowledge about dietary supplements (KaDS) among the general public and to evaluate the trust in advertising them (TiADS). We also report a significant negative association between these constructs.
Knowledge about dietary supplements has been extensively studied in many population subgroups, including healthcare professionals [24, 54], medical students [52, 55], athletes [56] and the elderly [57]. However, few measures, if any, have been designed to assess commonly-identifiable beliefs, which are important from the perspective of public health, and few have used a rigorous methodology in the process of their development. The KaDS questionnaire reported in the current paper meets these needs. It was found to satisfactorily reflect the conceptual framework of the a priori defined construct and was refined to measure the possible aspects of knowledge commonly identified by the public. These advantages, together with its concise form (17 true/false items), make the tool suitable for screening the general population. Striving to preserve content validity of the questionnaire and keeping its limited length, however, was found to be in conflict with improving its psychometric properties, which is a common problem of short scales [58]. Indeed, the confirmatory factor analysis model fit and noncentrality-based indices as well as reliability estimates were modest and the loadings of some items low and even insignificant. Moreover, exploratory factor analysis with two factors could explain less than a third of variance. This all indicates that the nature of KaDS can hardly be limited to simply “general” and “specific”, nevertheless, adjusting the data to a more complex model was unjustified, as it could lead to misleading conclusions [59, 60]. Consequently, the presented 17-item KaDS test is likely inadequate for detailed knowledge evaluation and decision-making in individual respondents [58]. On the other hand, it appears suitable for rapid screening for research purposes. Indeed, similar short tests have already been used in research [61, 62].
Our findings indicate a strikingly low level of KaDS in the study group: the mean number of correctly assessed statements was close to a random guess. This raises substantial concerns regarding public health and highlights the need for adequate and evidence-based education about DS at all levels [57, 63, 64]. Even so, our findings remain in line with other similar reports, which conclude “knowledge (…) at a low level” [16], “deficits in consumer knowledge” about DS [65], “some knowledge deficiency” [52], “inadequate” and “limited” knowledge [55], “lack of knowledge or misinformation” [56], “substantial misconceptions” [57], and the presence of beliefs related to DS that “are rarely based on medical evidence” [15]. Surprisingly, the reported poor knowledge contrasts with the overestimated competence of the respondents, measured as self-reported familiarity with the statements. Such discrepancy may represent a cognitive bias identified by Kruger and Dunning [66], who argue that “people tend to hold overly favorable views of their abilities”. Intriguingly, this may be also attributed to the TiADS, which may lead to consolidation of illusory belief in having KaDS.
Trust is a multidimensional construct [34, 67], which presents a great challenge for its operationalization and quantification. A systematic and comprehensive analysis of “trust in advertising” was accomplished by Soh et al. [17]. The “trust in advertising dietary supplements” (TiADS) may have been unique and may require further refinement, as it applies to products directly related to human health–a value of great importance [68]. In addition, to reconcile several objectives of the present study, the measure should be concise and attainable. Similar existing measures employed by Menon et al. [69] or Huh et al. [70] for testing trust in online prescription drug information were not adopted for the present study due to their limited proof of validity. Instead, a new measure was developed from scratch. The TiADS questionnaire is an eight-item three-dimensional scale, which exhibits well-established content, criterion and concurrent validity as well as satisfactory internal consistency. The measure comprises “reliability” and “intelligibility” dimensions attributed to the cognitive domain and an “affect” factor, which reflects emotional domain. Contrary to similar measures [17], the present questionnaire does not include a behavioral domain; however, the items excluded from the measure, which theoretically reflected this domain, were found to correlate closely with the total TiADS questionnaire score. Furthermore, applying self-reported testing of claimed behavior may lead to biased results [71] and should be rather replaced with a more valid procedure employing tracking methods in consumer behavior research [72]. Apart from this, the item content of the TiADS questionnaire turned out to be largely similar to that of other measures [17, 70]. Nevertheless, it was noted that measuring trust in online prescription drug information based on different methods yields similar results [70].
The current study found overall trust in DS advertising to be low among the general public. This is intriguing in the light of high popularity of these products, but consistent with other research on DS advertising [5] and the reports on trust in online prescription drug information [69, 70]. Low trust in DS advertising also reflects the current global trend of growing crisis of trust [5]. Similarly to the above cited papers [69, 70], the TiADS in the current report was relatively stable across sociodemographic variables. The extent of trust in DS advertising was proportional to the frequency of DS use. In contrast, Huh et al. [70] report no relationship between trust in online prescription drug information and drug use. This may be explained by the fact that in contrast to DS, drugs are largely prescribed by doctors and not voluntarily selected by the consumer.
In 2017 Polish Supreme Audit Office (orig. Najwyższa Izba Kontroli) published an official report entitled Marketing of dietary supplements (orig. “Dopuszczanie do obrotu suplementów diety”), which highlighted numerous inefficiencies in the Polish national system of marketing, regulating and testing DS [16]. It proposes that inadequate monitoring of advertising plays a key role, with some advertisements offering unverifiable claims implying that DS may play a therapeutic role, and blurring the difference between the DS and over-the-counter medication. Another study performed in Poland found that 23 out of 27 advertisements recorded from TV or radio promised a beneficial effect, indicating benefits for organs or improved concentrations of biochemical indicators [36]. Such practices are against the law, which allows only certain and balanced health claims for DS advertising [73]. These findings in Poland are highly consistent with those obtained from other countries linking importunate marketing of DS with potential harm to the consumer [15, 35, 74, 75].
Our results confirm that extensive DS advertising is in conflict with the promotion of evidence-based knowledge about DS. A corresponding analogy to medical advertising was noted [76–81]: although many advertisements convey some factual information about drugs, they outweigh their benefits over the risks, undervalue lifestyle changes and employ emotional appeals [76]. These observations indicate that drug advertising [77], and perhaps that of dietary supplements offers little social value [82, 83]. As a consequence, our study, together with other Polish reports [16, 36, 83], emphasizes an urgent need to introduce more strict regulation and control over DS advertising, and even to consider a total ban on advertising them [83].
The current paper highlights a significant negative link between the level of KaDS and TiADS. This link was especially pronounced in people reporting not to take DS. The decision not to use DS may be a result of knowing much about them as well as holding little trust in their advertising. Other studies also suggest that the experience of consumers with dietary supplements may be a factor influencing reactions to new information about the product [84].
In contrast with TiADS, a negligible effect size was observed for the link between KaDS and having recent contact with DS advertisements, although this was statistically significant for “specific” knowledge. “Having contact” with DS advertising may only involve a fleeting interaction between a consumer and an advertisement. Consequently, such a contact may not be sufficient to substantially affect the knowledge about the advertised products, and a deeper commitment with the advert needs to be formed before trust can be built in the message [17, 85] thus justifying the acceptance of the advertising message [86]. Nevertheless, our findings indicate that having contact with DS advertising may strengthen the negative link between KaDS and TiADS, thus enhancing our understanding of mutual relationship between these factors.
Although an association was found between the KaDS and TiADS, this does not imply any causation, as highlighted by the Bradford Hill criteria [87]. In addition, a question on direction of potential causation remains also justified in this case [88]. The study design is incapable of providing a reply. Nevertheless, it is possible to speculate on the nature of the relationship between KaDS and TiADS:
- Possessing prior KaDS (e.g. learned from healthcare specialists or acquired from professional literature or during biomedical courses) develops prejudice and distrust towards deceptive or ambiguous content of DS advertisement. A knowledgeable person seems to have no need to seek information about DS through advertising and may more accurately and critically assess its claims as untruthful or misleading [35, 89]. Trust is thought to develop from knowledge of the object of trust, and previous experiences with it [84, 90]. In fact, our present results suggest that medically educated people were found to trust in DS advertising significantly less than non-medically educated people, and one may consider a formal medical education as antecedent to trust in advertising health-related products.
- This is also likely that the opposite relationship exists: the extent of TiADS shapes KaDS. It was suggested that individuals are more willing to process information from a trusted source [90]. This processing takes the form of attentiveness to facilitate learning [91, 92], increases involvement in advertising and decreases avoidance [17, 93]. This may allow trust in DS advertising to build and consolidate the illusory knowledge about the product. Such possibility is of great relevance for public health as it entails the need for continuous monitoring and strict control over DS advertising to foster evidence-based beliefs and encourage prudent behavior.
- Above all, the link between the above discussed constructs may be more complex. Knowledge and trust may constitute joint effects of a common cause [88] such as sources of KaDS affecting attitudes to TiADS, or self-confidence playing a role in the consumer decision-making process. They may also mutually affect each other in a feedback loop and be mediated by multiple other factors.
Finally, our present findings may seem obvious. Yet the objectives of advertising, by definition, is the “presentation and promotion” of the product [31]. As DS are largely ineffective in health improvement, the only way to promote them appears to be by introducing manipulative messages about their utility, without facilitating informed health care decisions [94]. Thus, trusting in such messages is in opposition to true knowledge about them. This reasoning is simplified and one-sided, but underlines the deceptive nature of the content of DS advertisements and raises major concerns about ethical standards in their marketing [94, 95]. Moreover, the effect size of the KaDS-TiADS link is relatively high, appearing to be at least as high as the link between the presence, or absence of a formal medical education and KaDS. Eventually, in the light of a recent publication by Byron Sharp [96], which challenged common marketing theories, any hypothesis in marketing science is worth investigating. The present study offers new findings linking KaDS with TiADS, which may be of value to scientists, healthcare specialists, marketing experts and public health legislators.
The present study does have some limitations. Firstly, the samples used were drawn by non-probability methods, which may compromise the generalizability of the results. Secondly, most of the results are based on self-reported declarative data, which may further introduce substantial bias. Thirdly, the KaDS was compared with the TiADS in the same dataset as the final choice of the items to the KaDS test and no further validation of the knowledge test was performed. However, this questionnaire was constructed with the researchers being blinded to the result of TiADS questionnaire. Even so, the trust in advertising DS was still significantly negatively correlated with the sum of all 32 KaDS items considered for the final choice of the statements to the questionnaire. Finally, the developed questionnaires follow classical test theory, and do not take into account the difficulty of the test items expressed in the item response theory.
The current preliminary report requires further exploration confirming the results in a larger and randomly drawn dataset, seeking the mechanisms linking the observed constructs, and, ideally, testing the strategy to improve public knowledge about dietary supplements. It is advised to incorporate a mixture of qualitative and quantitative research methods, to gain a more comprehensive understanding of the reported phenomena.
Conclusions
The developed questionnaires possess satisfactory validity when assessing knowledge about dietary supplements in the general public and their trust in advertising dietary supplements: both were found to be low in the study groups. The questionnaires appear to be adequate for screening purposes in research. A significant negative association was found between the two examined constructs, suggesting that extensive advertising of dietary supplements conflicts with the promotion of established facts regarding their properties. These results raise concerns regarding the level of knowledge of dietary supplements held by the general population as well as ethical standards in their marketing; they also suggest the need for educational campaigns regarding dietary supplements and their advertising, as well as for updated regulations, continuous monitoring and strict control over such advertising in order to build evidence-based beliefs and encourage responsible use. Our findings are only preliminary and require further confirmation and exploration to identify the mechanism linking the constructs and testing strategies to improve the existing situation.
Supporting information
S1 Supporting Information. Details on methods used in the development and validation of the questionnaire on knowledge about dietary supplements (KaDS).
https://doi.org/10.1371/journal.pone.0218398.s001
(DOC)
S2 Supporting Information. Details on methods used in the development and validation of the questionnaire on trust in advertising dietary supplements (TiADS).
https://doi.org/10.1371/journal.pone.0218398.s002
(DOC)
S1 Survey. Final survey “Dietary supplements and their advertising” subjected to 220 non-medically educated respondents.
https://doi.org/10.1371/journal.pone.0218398.s003
(DOCX)
Acknowledgments
We thank our colleagues: the members of both panels of experts for such great involvement in their tasks, mgr. Joanna Bednarek, a specialist in Polish linguistics, for thorough review of the statements in the questionnaires, and all the respondents devoting their time to be interviewed or completing the questionnaires.
References
- 1. Spedding S. Vitamins are more Funky than Casimir thought. Australas Med J. 2013; 6(2): 104–6. pmid:23483872
- 2. Kamangar F, Emadi A. Vitamin and mineral supplements: do we really need them? Int J Prev Med. 2012; 3(3): 221–6. pmid:22448315
- 3. Schneyder A. Malnutrition and nutritional supplements. Aust Prescr. 2014; 37(4): 120–3.
- 4. Mozaffarian D, Rosenberg I, Uauy R. History of modern nutrition science-implications for current research, dietary guidelines, and food policy. BMJ. 2018; 361: k2392. pmid:29899124
- 5. Nichter M, Thompson JJ. For my wellness, not just my illness: North Americans' use of dietary supplements. Cult Med Psychiatry. 2006; 30(2): 175–222. pmid:16841188
- 6. Manson JE, Cook NR, Lee IM, Christen W, Bassuk SS, Mora S, et al. Marine n-3 Fatty Acids and Prevention of Cardiovascular Disease and Cancer. N Engl J Med. 2019; 380(1): 23–32. pmid:30415637
- 7. Cohen PA. The Supplement Paradox: Negligible Benefits, Robust Consumption. JAMA. 2016; 316(14): 1453–54. pmid:27727369
- 8. Hemilä H, Chalker E. Vitamin C for preventing and treating the common cold. Cochrane Database Syst Rev. 2013; 1: CD000980.
- 9. Abdelhamid AS, Brown TJ, Brainard JS, Biswas P, Thorpe GC, Moore HJ, et al. Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease. Cochrane Database Syst Rev. 2018; 7: CD003177. pmid:30019766
- 10. Fortmann SP, Burda BU, Senger CA, Lin JS, Whitlock EP. Vitamin and mineral supplements in the primary prevention of cardiovascular disease and cancer: An updated systematic evidence review for the U.S. Preventive Services Task Force. Ann Intern Med. 2013; 159(12): 824–34. pmid:24217421
- 11. Myung SK, Kim Y, Ju W, Choi HJ, Bae WK. Effects of antioxidant supplements on cancer prevention: meta-analysis of randomized controlled trials. Ann Oncol. 2010; 21(1): 166–79. pmid:19622597
- 12. Mendelsohn AR, Larrick JW. Paradoxical effects of antioxidants on cancer. Rejuvenation Res. 2014; 17(3): 306–11. pmid:24828241
- 13. Dodge T, Kaufman A. What makes consumers think dietary supplements are safe and effective? The role of disclaimers and FDA approval. Health Psychol. 2007; 26(4): 513–7. pmid:17605572
- 14. Conner M, Kirk SF, Cade JE, Barrett JH. Why do women use dietary supplements? The use of the theory of planned behaviour to explore beliefs about their use. Soc Sci Med. 2001; 52(4): 621–33. pmid:11206658
- 15. Sirico F, Miressi S, Castaldo C, Spera R, Montagnani S, Di Meglio F, et al. Habits and beliefs related to food supplements: Results of a survey among Italian students of different education fields and levels. PLoS ONE. 2018; 13(1): e0191424. pmid:29351568
- 16. Polish Supreme Audit Office. On the authorization of dietary supplements. 2017; Available from: https://www.nik.gov.pl/aktualnosci/nik-o-dopuszczaniu-do-obrotu-suplementow-diety.html (05 February 2019, date last accessed).
- 17. Soh H, Reid LN, King KW. Measuring Trust In Advertising. J Advert. 2009; 38(2): 83–104.
- 18. Nielsen's Report. Global trust in advertising. Winning strategies for an evolving media landscape. 2015; Available from: https://www.nielsen.com/eu/en/insights/reports/2015/global-trust-in-advertising-2015.html (05 February 2019, date last accessed).
- 19. Millar CCJM, Choi CJ. Advertising and Knowledge Intermediaries: Managing the Ethical Challenges of Intangibles. J Bus Ethics. 2003; 48(3): 267–77.
- 20. Vali I. The Role of Education in the Knowledge-based Society. Procedia Soc Behav Sci. 2013; 76: 388–92.
- 21.
Directive 2002/46/EC of the European Parliament and of the Council of 10 June 2002 on the approximation of the laws of the Member States relating to food supplements (Official Journal of the European Union L 183 12.7.2002, p. 51, as amended); Available from: https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=celex%3A32002L0046 (05 February 2019, date last accessed).
- 22. Burns KE, Duffett M, Kho ME, Meade MO, Adhikari NK, Sinuff T, et al. A guide for the design and conduct of self-administered surveys of clinicians. CMAJ. 2008; 179(3): 245–52. pmid:18663204
- 23. Trakman GL, Forsyth A, Hoye R, Belski R. Developing and validating a nutrition knowledge questionnaire: key methods and considerations. Public Health Nutr. 2017; 20(15): 2670–2679. pmid:28735598
- 24. Steyn NP, Labadarios D, Nel JH, Heidi-lee R. Development and validation of a questionnaire to test knowledge and practices of dietitians regarding dietary supplements. Nutrition. 2005; 21(1): 51–8. pmid:15661478
- 25. Parmenter K, Wardle J. Development of a general nutrition knowledge questionnaire for adults. Eur J Clin Nutr. 1999; 53(4): 298–308. pmid:10334656
- 26. Hilton CE. The importance of pretesting questionnaires: a field research example of cognitive pretesting the Exercise referral Quality of Life Scale (ER-QLS). Int J Soc Res Methodol. 2017; 20(1): 21–34.
- 27. Pfäfflin M, Petermann F, Rau J, May TW. The psychoeducational program for children with epilepsy and their parents (FAMOSES): results of a controlled pilot study and a survey of parent satisfaction over a five-year period. Epilepsy Behav. 2012; 25(1): 11–6. pmid:22980075
- 28. Coolbrandt A, Van den Heede K, Clemens K, Milisen K, Laenen A, Wildiers H, et al. The Leuven questionnaire for Patient Self-care during Chemotherapy (L-PaSC): instrument development and psychometric evaluation. Eur J Oncol Nurs. 2013; 17(3): 275–83. pmid:22901548
- 29. Carpenter S. Ten Steps in Scale Development and Reporting: A Guide for Researchers. Communication Methods and Measures. 2018; 12(1): 25–44.
- 30. Bar-Hillel M, Attali Y. Seek whence: Answer sequences and their consequences in key-balanced multiple-choice tests. The American Statistician. 2002; 56(4): 299–303
- 31.
Kotler P, Armstrong G. Principles of Marketing, Global Edition. 17th ed. Hearlow: Pearson Education Limited; 2018.
- 32. Lewis JD, Weigert A. Trust as a Social Reality. Soc Forces. 1985; 63(4): 967–85.
- 33.
Wierzbiński J. Badanie zaufania do organizacji—problemy metodologiczne. Warszawa: Wydawnictwo Naukowe Wydziału Zarządzania Uniwersytetu Warszawskiego; 2009.
- 34. Zieliński T. Trust as Regulator of Economic Decision Making. J Manag Bus Adm Cent Eur. 2012; 20(4): 73–91.
- 35. Starr RR. Too little, too late: ineffective regulation of dietary supplements in the United States. Am J Public Health. 2015; 105(3): 478–85. pmid:25602879
- 36. Wierzejska R. Whether the advertisement of dietary supplements is objective source of data about their impact on health? Analysis of broadcasting advertisements in the terms of the food law. Wiad Lek. 2016; 69(1): 14–8. pmid:27162289
- 37. The Act of 25 August 2006 on Food and Nutrition Safety; consolidated act. Journal of Laws of Republic of Poland. 2018; item 1541 as amended. Available from: http://prawo.sejm.gov.pl/isap.nsf/DocDetails.xsp?id=WDU20180001541
- 38. Avenell A, Mak JCS, O'Connell D. Vitamin D and vitamin D analogues for preventing fractures in post-menopausal women and older men. Cochrane Database Syst Rev. 2014; 4: CD000227.
- 39. Bolland MJ, Grey A, Avenell A. Effects of vitamin D supplementation on musculoskeletal health: a systematic review, meta-analysis, and trial sequential analysis. Lancet Diabetes Endocrinol. 2018; 6(11): 847–58. pmid:30293909
- 40. Garrison SR, Allan GM, Sekhon RK, Musini VM, Khan KM. Magnesium for skeletal muscle cramps. Cochrane Database Syst Rev. 2012; 9: CD009402.
- 41. Sebo P, Cerutti B, Haller DM. Effect of magnesium therapy on nocturnal leg cramps: a systematic review of randomized controlled trials with meta-analysis using simulations. Fam Pract. 2014; 31(1): 7–19. pmid:24280947
- 42. Zhao JG, Zeng XT, Wang J, Liu L. Association Between Calcium or Vitamin D Supplementation and Fracture Incidence in Community-Dwelling Older Adults: A Systematic Review and Meta-analysis. JAMA. 2017; 318(24): 2466–82. pmid:29279934
- 43. US Preventive Services Task Force, Grossman DC, Curry SJ, Owens DK, Barry MJ, Caughey AB, et al. Vitamin D, Calcium, or Combined Supplementation for the Primary Prevention of Fractures in Community-Dwelling Adults: US Preventive Services Task Force Recommendation Statement. JAMA. 2018; 319(15): 1592–99. pmid:29677309
- 44. Jenkins DJA, Spence JD, Giovannucci EL, Kim YI, Josse R, Vieth R, et al. Supplemental Vitamins and Minerals for CVD Prevention and Treatment. J Am Coll Cardiol. 2018; 71(22): 2570–84. pmid:29852980
- 45. Rautiainen S, Lee IM, Rist PM, Gaziano JM, Manson JE, Buring JE, et al. Multivitamin use and cardiovascular disease in a prospective study of women. Am J Clin Nutr. 2015; 101(1): 144–52. pmid:25527758
- 46. Douglas RM, Hemilä H, Chalker E, Treacy B. Vitamin C for preventing and treating the common cold. Cochrane Database Syst Rev. 2007; 3: CD000980.
- 47.
Standing Committee on the Scientific Evaluation of Dietary Reference Intakes, Food and Nutrition Board, Institute of Medicine. Dietary Reference Intakes for Calcium, Phosphorus, Magnesium, Vitamin D, and Fluoride. 1st ed. National Academies Press; 1999.
- 48. Carr AC, Vissers MC. Synthetic or food-derived vitamin C—are they equally bioavailable?. Nutrients. 2013; 5(11): 4284–304. pmid:24169506
- 49. Rivers JM. Safety of high-level vitamin C ingestion. Int J Vitam Nutr Res Suppl. 1989; 30: 95–102. pmid:2507711
- 50.
Institute of Medicine (US) Panel on Dietary Antioxidants and Related Compounds. Dietary Reference Intakes for Vitamin C, Vitamin E, Selenium, and Carotenoids. 1st ed. Washington (DC): National Academies Press; 2000.
- 51. Kantor ED, Rehm CD, Du M, White E, Giovannucci EL. Trends in Dietary Supplement Use Among US Adults From 1999–2012. JAMA. 2016; 316(14): 1464–74. pmid:27727382
- 52. Aina BA, Ojedokun OA. Knowledge and use of dietary supplements by students of College of Medicine, University of Lagos, Idi-Araba, Lagos, Nigeria. J Basic Clin Pharm. 2014; 5(2): 34–9. pmid:25031497
- 53.
Statistics Poland, Social Surveys Department. Health care in households in 2016. Warszawa: 2018. Available from: https://stat.gov.pl/obszary-tematyczne/zdrowie/zdrowie/ochrona-zdrowia-w-gospodarstwach-domowych-w-2016-r-,2,6.html (05 February 2019, date last accessed).
- 54. Kwan D, Hirschkorn K, Boon H. U.S. and Canadian pharmacists' attitudes, knowledge, and professional practice behaviors toward dietary supplements: a systematic review. BMC Complement Altern Med. 2006; 6: 31. pmid:16984649
- 55. Axon DR, Vanova J, Edel C, Slack M. Dietary Supplement Use, Knowledge, and Perceptions Among Student Pharmacists. Am J Pharm Educ. 2017; 81(5): 92. pmid:28720920
- 56. Molinero O, Márquez S. Use of nutritional supplements in sports: risks, knowledge, andbehavioural-related factors. Nutr Hosp. 2009; 24(2): 128–34. pmid:19593480
- 57. Marinac JS, Buchinger CL, Godfrey LA, Wooten JM, Sun C, Willsie SK. Herbal Products and Dietary Supplements: A Survey of Use, Attitudes, and Knowledge Among Older Adults. J Am Osteopath Assoc. 2007; 107: 13–23. pmid:17299031
- 58. Ziegler M, Kemper CJ, Kruyen PM. Short scales–five misunderstandings and ways to overcome them. Journal of Individual Differences. 2014; 35(4): 185–9.
- 59. Van der Eijk C, Rose J. Risky business: factor analysis of survey data—assessing the probability of incorrect dimensionalisation. PLOS ONE. 2015; 10(3): e0118900. pmid:25789992
- 60. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods. 1999; 4(3): 272–99.
- 61. Duff JP, Cheng A, Bahry LM, Hopkins J, Richard M, Schexnayder SM, et al. Development and validation of a multiple choice examination assessing cognitive and behavioural knowledge of pediatric resuscitation: a report from the EXPRESS pediatric research collaborative. Resuscitation. 2013; 84(3): 365–8. pmid:22842012
- 62. Karbownik MS, Wiktorowska-Owczarek A, Kowalczyk E, Kwarta P, Mokros Ł, Pietras T. Board game versus lecture-based seminar in the teaching of pharmacology of antimicrobial drugs—a randomized controlled trial. FEMS Microbiol Lett. 2016; 363(7).
- 63. National Institutes of Health, Office of Dietary Supplements. Strategic Plan 2017–2021. Strengthening Knowledge & Understanding of Dietary Supplements. 2016; Available from: https://ods.od.nih.gov/pubs/strategicplan/ODSStrategicPlan2017-2021.pdf (05 February 2019, date last accessed).
- 64. Begdache L, Kianmehr H, Heaney CV. College Education on Dietary Supplements May Promote Responsible Use In Young Adults. J Diet Suppl. 2018; 1–14.
- 65. Dodge T, Litt D, Kaufman A. Influence of the dietary supplement health and education act on consumer beliefs about the safety and effectiveness of dietary supplements. J Health Commun. 2011; 16(3): 230–44. pmid:21120738
- 66. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999; 77(6): 1121–34. pmid:10626367
- 67.
Krot K, Lewicka D. Zaufanie w organizacji innowacyjnej. 1 st ed. Warszawa: C.H.Beck; 2016.
- 68. Lau RR, Hartman KA, Ware JE. Health as a value: methodological and theoretical considerations. Health Psychol. 1986; 5(1): 25–43. pmid:3720718
- 69. Menon AM, Deshpande AD, Perri M, Zinkhan GM. Trust in online prescription drug information among internet users: the impact on information search behavior after exposure to direct-to-consumer advertising. Health Mark Q. 2002; 20(1): 17–35. pmid:12749596
- 70. Huh J, Delorme DE, Reid LN. Factors affecting trust in on-line prescription drug information and impact of trust on behavior following exposure to DTC advertising. J Health Commun. 2005; 10(8): 711–31. pmid:16316935
- 71. Rosenman R, Tennekoon V, Hill LG. Measuring bias in self-reported data. Int J Behav Healthc Res. 2011; 2(4): 320–32. pmid:25383095
- 72.
Haugtvedt CP, Herr P, Kardes F. Handbook of Consumer Psychology. 1 st ed. New York: Psychology Press; 2012.
- 73.
Commission Regulation (EU) No 432/2012 of 16 May 2012 establishing a list of permitted health claims made on foods, other than those referring to the reduction of disease risk and to children’s development and health (Official Journal of the European Union L 136/1 25.5.2012); Available from: https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:136:0001:0040:EN:PDF (09 February 2019, date last accessed).
- 74. Wong MK, Darvishzadeh A, Maler NA, Bota RG. Dietary Supplement Nomenclature. Prim Care Companion CNS Disord. 2016; 18(2): pmid:27486550
- 75. Hoffmann D, Schwartz J. Stopping deceptive health claims: the need for a private right of action under federal law. Am J Law Med. 2016; 42(1): 53–84. pmid:27263263
- 76. Frosch DL, Krueger PM, Hornik RC, Cronholm PF, Barg FK. Creating demand for prescription drugs: a content analysis of television direct-to-consumer advertising. Ann Fam Med. 2007; 5(1): 6–13. pmid:17261859
- 77. Faerber AE, Kreling DH. Content analysis of false and misleading claims in television advertising for prescription and nonprescription drugs. J Gen Intern Med. 2014; 29(1): 110–8. pmid:24030427
- 78. Wilkes MS, Doblin BH, Shapiro MF. Pharmaceutical advertisements in leading medical journals: experts' assessments. Ann Intern Med. 1992; 116(11): 912–9. pmid:1580449
- 79. Tsai AC. Conflicts between commercial and scientific interests in pharmaceutical advertising for medical journals. Int J Health Serv. 2003; 33(4): 751–68. pmid:14758858
- 80. Othman N, Vitry A, Roughead EE. Quality of pharmaceutical advertisements in medical journals: a systematic review. PLoS ONE. 2009; 4(7): e6350. pmid:19623259
- 81. Ruiz R. Ten misleading drug ads. Forbes. 2010; Available from: https://www.forbes.com/2010/02/02/drug-advertising-lipitor-lifestyle-health-pharmaceuticals-safety.html#3a0e15c8c58d (09 February 2019, date last accessed).
- 82. Hobbs R, Broder S, Pope H, Rowe J. How adolescent girls interpret weight-loss advertising. Health Educ Res. 2006; 21(5): 719–30. pmid:16880220
- 83. Makowska M, Jasiński Ł. A discussion of the unresolved 2016/17 plans for regulating the Polish dietary supplements market. Health Policy. 2019; [Epub ahead of print] pmid:30981554
- 84. Mason MJ, Scammon DL, Fang X. The impact of warnings, disclaimers, and product experience on consumers’ perceptions of dietary supplements. J. Consumer Aff. 2007; 41(1): 74–99.
- 85.
Zhang X, Zhang Q. Online trust forming mechanism: approaches and an integrated model. 7th international Conference on Electronic Commerce. Xi'an, China. 2005; Available from: http://onemvweb.com/sources/sources/online_trust_mechanism.pdf (09 February 2019, date last accessed).
- 86. Longworth G. Knowledge on trust by Paul Faulkner. Analysis. 2012; 72(3): 623–4.
- 87. Hill AB. The environment and disease: association or causation? Proc R Soc Med. 1965; 58: 295–300. pmid:14283879
- 88. Mackie JL. The direction of causation. The philosophical review. JSTOR. 1966; 75(4): 441–66.
- 89. Nutbeam D. Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Promot Int. 2000; 15(3): 259–67.
- 90. Ball JG, Manika D, Stout P. Causes and consequences of trust in direct-to-consumer prescription drug advertising. Int J Advert. 2016; 35(2): 216–47.
- 91. Rotter JB. Generalized expectations for interpersonal trust. Am Psychol. 1971; 26(5): 443–52.
- 92. Bleich S, Blendon R, Adams A. Trust in scientific experts on obesity: Implications for awareness and behavior change. Obesity (Silver Spring). 2007; 15(8): 2145–56
- 93. Chatterjee SC, Chaudhuri A. Are trusted brands important? Mark Manage. 2005; 15(1): 1–16.
- 94. Schenker Y, Arnold RM, London AJ. The ethics of advertising for health care services. Am J Bioeth. 2014; 14(3): 34–43. pmid:24592839
- 95. Torok CB, Murray TH. Wielding the sword of professional ethics against misleading dietary supplement claims. Glycobiology. 2008; 18(9): 660–3.
- 96.
Sharp B. How brands grow: what marketers don't know. 1 st ed. Oxford University Press Australia & New Zealand; 2010.