http://orcid.org/0000-0002-7019-5072
Figures
Abstract
Methods
The two-phase study utilized quantitative and qualitative methods. Phase 1 of the study consisted of developing an initial survey (English and Spanish versions) assessing weight literacy based on a review of the literature; conducting semi-structured interviews with content experts (N = 9) to refine survey items; and conducting in-person cognitive interviews with 20 study participants (N = 10 English-speaking and N = 10 Spanish-speaking adults) for survey pre-testing. Survey items were modified based on Phase 1 findings. Phase 2 consisted of a psychometric study of the Weight Literacy Scale developed in Phase 1. Procedures included administering the Weight Literacy Scale to 200 study participants (N = 100 English-speaking and N = 100 Spanish-speaking adults), a quantitative survey assessing dietary and physical activity behaviors and sociodemographics, measuring participants’ height and weight, and assessing the scale’s validity and internal reliability. A subset of Phase 2 participants (N = 71) completed the weight literacy scale at two-weeks follow-up to assess test-retest reliability. Participant recruitment and study procedures took place in community settings in central Massachusetts for both study phases. Weight literacy scale scores were calculated as the sum of total correct items. Three rounds of factor analysis were performed to identify items for elimination. The Kuder Richardson’s Coefficient of reliability was calculated. Correlations between the Weight Literacy Scale scores and related measures (body mass index and weight status, dietary behaviors, physical activity behaviors, and confidence in filling out medical forms) were examined.
Citation: Wang ML, Little TV, Frisard C, Borg A, Lemon SC, Rosal MC (2018) Development and validation of a Weight Literacy Scale in English and Spanish. PLoS ONE 13(10): e0204678. https://doi.org/10.1371/journal.pone.0204678
Editor: José Guilherme Behrensdorf Derraik, University of Auckland, NEW ZEALAND
Received: April 20, 2018; Accepted: September 12, 2018; Published: October 24, 2018
Copyright: © 2018 Wang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This publication is a product of a Health Promotion and Disease Prevention Research Center grant supported by Cooperative Agreement Number (U48 DP005031) from the Centers for Disease Control and Prevention (awarded to MCR and SCL), and a grant from the National Institute of Minority Health and Health Disparities (1P60 MD006912) (awarded to MCR and SCL). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Obesity is a growing epidemic nationally and globally [1], with over one third (36.5%) of U.S. adults being currently obese [2, 3]. Given the premature morbidity and mortality and staggering health care costs associated with obesity [4], interventions that successfully help adults prevent and manage obesity through behavior change (e.g., diet, physical activity) are needed.
Several factors facilitate successful weight management, including health literacy, defined by the Institute of Medicine as "the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions [5]." Studies suggest an inverse association between health literacy and body mass index (BMI), with lower health literacy associated with higher BMI among adults [6, 7]. Studies examining health literacy within the context of obesity and/or weight loss efforts have primarily focused on assessing participants’ sources of health information. An increasing percentage of individuals, particularly those who are obese, utilize the internet and mobile apps as the primary sources of weight loss information and advice [8, 9], though some research has found that overweight or obese African American women reported television shows (e.g., Oprah, The Biggest Loser) as the main source of dieting information [10]. Other common sources of weight loss information include in-person exchanges with friends/family, publications like women’s magazines, diet books and newspapers, and health personnel i.e. physicians, nurses, and dieticians [8, 10, 11].
Health literacy may play an important role in facilitating successful behavioral weight loss or management, though the mechanisms of this association are not well established. One study among obese African American women found that participants with average health literacy levels were more likely to join weight loss programs, increase physical activity for weight loss, and seek weight loss information online compared to participants with low health literacy levels [11]. Among overweight and obese adults in a behavioral weight loss intervention, high health literacy was significantly correlated with successful weight loss [12]. Even less research has been conducted on the assessment of health literacy specific to weight loss or management. Health literacy in the context of weight management is complex and requires an in-depth understanding of the knowledge, skills, and behavioral targets needed to make informed decisions about weight management. Understanding individuals’ capability to obtain and process weight loss information and make decisions regarding weight management, or “weight literacy,” may help elucidate barriers and facilitators of successful weight loss.
Building on the definition of health literacy from the Institute of Medicine, we propose to define weight literacy as “the degree to which individuals can obtain, process, understand, and communicate about weight-related information needed to make informed decisions about weight management.” This definition reflects our view for the need for health care consumers to be able to communicate their understanding of weight-related information; substituted “basic” and “health” information for weight-related information specifically; and focused on “informed” decision-making because “appropriate” care may not be culturally congruent with the patient and an “informed” decision is more patient-centered. We distinguish weight literacy from nutrition literacy, defined as “the degree to which individuals have the capacity to obtain, process, and understand nutrition information and skills needed in order to make appropriate nutrition decisions” [13], as weight literacy encompasses information and skills specific for weight loss or management (e.g., energy balance, goal-setting and tracking for diet and physical activity).
To our knowledge, a validated weight literacy scale that is linguistically appropriate for ethnically diverse populations does not exist. Such a scale would be instrumental in enhancing our understanding of the role of weight literacy in behavioral weight loss/management and identifying areas to improve weight literacy and ultimately weight loss/management behaviors. Specific aims of this study included developing items for the Weight Literacy Scale; modifying the scale based on survey cognitive testing; and evaluating the scale for reliability and validity.
Materials and methods
This two-phase study was informed by the six steps of survey development recommended by Hinkin [14]. Phase 1 included inductive item generation with expert content validity assessment. Phase 2 included scale administration to a large sample; exploratory factor analysis and item reduction; assessment of the scale’s internal consistency and validity; and evaluation of the scale’s test re-test reliability. Study procedures for Phases 1 and 2 were approved by the Institutional Review Board of the University of Massachusetts Medical School; all participants provided written informed consent to participate in the study.
Phase 1
Scale development.
We first conducted a review of the literature related to health literacy and weight management to operationalize weight literacy (described above). The study team developed preliminary survey items on energy balance and related factors according to key dimensions of weight literacy, including knowledge and awareness of managing energy balance, as well as biological, psychological, behavioral, and environmental (social and physical) factors associated with managing weight. To inform the next iteration of the scale, one-hour, semi-structured interviews were conducted with content experts (N = 9), including a clinical psychologist, an epidemiologist, a dietician, project managers of obesity-related research, and medical students. Survey items were modified based on feedback from the content experts and translated into Spanish by bilingual study team members (a graduate research assistant and the study PI) with expertise in developing materials appropriate for low-literacy populations.
Participants and procedures for survey cognitive testing.
Cognitive interviewing, a method of improving survey development [15], involved administering items to participants and asking them to explain their understanding of each item and corresponding response options, as described below. Eligibility criteria for participating in cognitive testing of the preliminary survey included: English and/or Spanish-speaking adults (18 years or older); residency in Worcester, MA; and able to provide informed consent to participate. Exclusion criteria included having health or psychiatric conditions that could impair participation in a 1-hour, audio-recorded interview. Study staff recruited participants through outreach to community organizations via in-person, telephone, and email communications. A convenience sample of English-speaking (N = 10) and Spanish-speaking (N = 10) participants were recruited.
Cognitive interviews were conducted by the study team in community locations in Central Massachusetts. During the 1-hour cognitive interviews, participants were asked to read and complete the surveys and highlight confusing or unclear survey items. Study staff then reviewed each survey item with each participant. Participants were asked to paraphrase each item in their own words to identify any problems with comprehension and intended meaning, and study staff noted items that participants perceived as confusing or unclear. All interviews were audio-recorded. Participants received $25 cash for their time to complete the interview. After reviewing the audio-recordings and interview notes, the graduate research assistant and study PI further refined survey items for pilot-testing.
Phase 2
Participants and procedures for survey pilot-testing.
Criteria for eligibility for survey pilot-testing (Phase 2) were similar to that of Phase 1: English and/or Spanish-speaking adults (18 years or older); residents of Worcester, MA; and able to provide informed consent to participate. Study staff recruited participants through outreach to community partners and disseminating study information through English- and Spanish-language materials, including flyer postings in community centers and online postings through a clinical research volunteer registry. A total of 200 participants (N = 100 English-speaking and N = 100 Spanish-speaking adults) were recruited for survey pilot-testing. Participants completed a 30-minute in-person visit to complete the Weight Literacy Survey and additional assessments (see Measures section). A subset of participants (N = 71) was invited for a second visit approximately two weeks after the initial visit to complete the survey for test-retest reliability. Assessments took place in a convenient community location or at the medical academic campus. Participants received a $35 gift card for their time.
Measures.
All Phase 2 measures were collected via in-person assessments at the time of the survey pilot-testing. Participants completed the initial 50-item Weight Literacy Scale, which consisted of true/false and multiple-choice questions. Anthropometric measures (height, weight, and waist circumference measurements) were obtained by trained, bilingual research staff. Weight was measured using portable digital scales (readings were recorded to the nearest 2/10th pound), height was measured to the nearest 1/8th inch using portable stadiometers, and waist circumference was to the nearest 1/8th inch using tape measures. BMI was calculated as weight in kilograms divided by meters squared (kg/m2). BMI-based weight status categories included: underweight (<18.5 kg/m2), healthy weight (18.5–24.9 kg/m2), overweight (25.0–29.9 kg/m2), or obese (≥30.0 kg/m2).
Dietary intake, physical activity, and sociodemographics were obtained through self-reported, self-administered surveys. Dietary intake was assessed using the Block Rapid Food Screener, a validated measure for identifying high-fat intake and low-fruit/vegetable intake [16]. Physical activity was assessed using the Brief Physical Activity Questionnaire [17], a validated measure of weekly minutes of physical activity. Sociodemographics assessed included: gender, age, race, ethnicity, marital status, education level, annual household income level, current employment status, and country of origin. Participants were also asked to rate their level of confidence in filling out medical forms on their own.
Statistical analysis.
Frequencies for categorical variables and means and standard deviations for continuous variables were computed to describe Phase 2 (survey pilot-testing) study sample characteristics. Participants’ responses on the initial 50-item Weight Literacy Scale were scored as correct or incorrect for each item, with higher scores indicating greater level of weight literacy. Three rounds of factor analyses were used to determine the final composition of the Weight Literacy Scale. To ensure each item’s unique contribution, variables with factor loadings of .40 or greater were considered to load on that factor[18]. Items that did not achieve factor loadings of .40 or greater were excluded from the scale. Principle component factor analysis was initially forced to 3 factors. Ten questions were eliminated in the first round. As this scale is intended to be a homogeneous measure, the second and third round of factor analysis forced the items to load onto one factor. Nine additional questions were eliminated in the second and third rounds.
Tests of reliability and validity were conducted using the final scale based on results of the factor analyses. Because the item scoring mechanism is binary, we used the Kuder Richardson Coefficient of reliability [19] to determine the scale’s internal consistency. The Kuder Richardson Coefficient of reliability was calculated for the overall study sample and by language (English vs. Spanish). Test-retest reliability was assessed using Pearson’s correlation between average weight literacy scale scores at baseline and 2 weeks follow-up among a subset of Phase 2 participants (N = 71). To examine construct validity, associations between Weight Literacy Scale scores and gender, education level, self-efficacy in filling out medical forms, BMI, weight status, fruit and vegetable intake (servings per day and fruit and vegetable dietary Block scores), and physical activity (minutes of physical activity per week and meeting guidelines of ≥ 150 minutes of physical activity per week) were explored using Pearson Correlation for continuous variables and bivariate regression modeling for categorical variables. Analyses were preformed using Stata version 12 (StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP).
Results
Sample characteristics
Of the 200 participants in Phase 2 of the study, 62.0% were female, the mean age was 41.8 (SD = 14.9), 50.5% reported having less than a college degree, and 58.5% reported an annual household income of less than $40,000. The mean BMI of participants was 30.3 kg/m2 (SD = 6.9), with 33.0% of participants falling in the overweight category and 43.5% in the obese category. Over half of participants reported feeling extremely (42.5%) or quite a bit (24.0%) confident in their ability to fill out medical forms on their own, though this varied by primary language spoken (83.0% of English-speaking participants reported feeling ‘extremely’ or ‘quite a bit’ confident vs. 50.0% of Spanish-speaking participants; p<0.001). Additional details of study sample characteristics and differences by primary language spoken are presented in Table 1.
Factor analysis
Of the 50 initial scale items, 31 had factor loadings of 0.40 or greater (see Table 2). Ten items were excluded in round 1, 8 items were excluded in round 2, and 1 item was excluded in round 3 of the factor analysis, resulting in 31 items included in the final scale (see S1 Table). The item difficulty index (percent of respondents who answered each question correctly) for the final scale ranged from 35.5% to 90.0% (see Table 2).
Weight literacy scale
Participants’ mean Weight Literacy Scale score on the final 31-item scale was 20.6 (SD = 7.0). The median score was 21.0; range of 0–31.0. Women on average scored 2 points higher on the scale compared to men (p = 0.05).
Internal consistency
The overall Kuder Richardson Coefficient of reliability for the final Weight Literacy Scale was 0.90. Similar coefficients were found by language (range of 0.84–0.88).
Construct validity
Participants’ weight literacy scores were positively associated with education level (β = 3.6; 95% CI: 2.8–4.3); p<0.001) and confidence in ability to fill out medical forms on their own (β = 7.5; 95% CI: 5.8–9.4; p<0.001). Those who reported being ‘extremely’ or ‘quite a bit confident’ in filling out medical forms on their own on average scored 7.5 points higher on the Weight Literacy Scale than those who reported feeling ‘somewhat,’ ‘a little bit,’ or ‘not at all confident’ in filling out medical forms. Weight Literacy Scale scores were not significantly correlated with minutes of physical activity per week, number of fruit and vegetable servings, fruit and vegetable dietary Block score, or BMI. Weight Literacy Scale scores were not significantly associated with meeting physical activity guidelines (≥ 150 minutes of physical activity per week) or weight status category.
Discussion
This two-phase study describes the development of the Weight Literacy Scale, an instrument designed to measure the degree of individuals’ capability to obtain, process, and understand weight loss information needed to make informed decisions regarding weight management, and the results of psychometric tests of the scale among 200 English- and Spanish-speaking adults.
Principal findings
Study findings support the reliability and validity of using the Weight Literacy Scale to assess weight literacy among a predominantly low socioeconomic status, ethnically diverse population. Weight Literacy Scale scores did not correlate with participants’ diet, physical activity, or BMI, but were significantly associated with gender and positively associated with education level.
Interpretation of findings
On average, participants answered two thirds of the Weight Literacy Scale items correctly (20.6 out of 31 items), indicating substantial room for improvement in weight literacy.
As expected, Weight Literacy Scale scores were positively associated with education level and self-efficacy to complete medical forms. Participants who reported a higher degree of confidence in being able to complete medical forms on their own had on average higher Weight Literacy Scale scores than those who reported less confidence. Weight Literacy Scale scores were not significantly associated with measures of diet, physical activity, BMI, or weight status. A possible explanation for this finding is that while individual-level literacy in weight control may facilitate successful weight management, factors at multiple levels (individual, interpersonal, community, and environmental) across multiple domains shape individuals’ diet, physical activity, and weight. For example, individuals with high weight literacy may still face substantial environmental and social barriers to achieving healthy eating and physical activity goals needed for weight loss [20]. Thus, BMI, weight, and weight management behaviors (e.g., diet, physical activity) may not necessarily be the strongest constructs to validate weight literacy. Previous efforts and experiences to lose weight may be more closely linked with weight literacy, though we did not collect this type of data for this study.
Clinical implications
Clinical settings and situations where this scale may be useful include interactions with providers and clinicians where weight loss or weight management is discussed as a health goal. The Weight Literacy Scale may serve as a tool to screen individuals who may need more assistance with developing a knowledge base around weight loss and with making informed decisions as part of weight loss or weight management (e.g., healthy gestational weight gain among pregnant women) interventions. Individuals’ responses on the scale can inform which strategies are needed to promote comprehension, self-efficacy, and/or skills needed for weight loss among those with low weight literacy. Though this scale was tested among individuals with lower educational levels (>70% had a high school degree or less), the scale may be administered to adult populations with varying levels of education, including college degree or higher.
Research implications
Overall, the Weight Literacy Scale demonstrates strong internal consistency and reasonable construct validity and test-retest reliability. This study has several strengths, including the development of a novel, linguistically-appropriate weight literacy instrument in English and in Spanish using formative, mixed methods research with an ethnically diverse study sample and conducting test-retest reliability with a subset of the study sample. Though the scale was developed for a broad adult population and not intended solely for individuals seeking weight management, the use of a convenience sample recruited from selected community locations within a city in central Massachusetts limits the generalizability of study findings. Latinos were primarily of Caribbean origin, thus it is unknown whether the instrument would have the same psychometric characteristics in other Hispanic sub-groups (e.g., Mexican Americans). Future research should examine the reliability and validity of the Weight Literacy Scale among other populations and assess participants’ previous efforts to lose weight as a construct for additional validity testing. Findings from this study indicate that the Weight Literacy Scale may be particularly useful as a tool to identify specific target areas for improving individuals’ understanding and capabilities related to weight management in future randomized controlled or quasi-experimental trials or to examine the association between weight literacy and weight-related outcomes in future observational studies.
Conclusions
Studies on the role of health literacy in the context of behavioral weight management are limited. The Weight Literacy Scale may be used to assess weight literacy in studies that seek to further elucidate the role of this construct in weight loss and management, screen individuals who may need more assistance with making informed decisions related to weight loss and management, as well as help identify educational targets in the context of weight loss interventions.
Supporting information
S1 Table. Final 31-item Weight Literacy Scale–English.
Note: Items 26 and 27 contain multiple questions; each question is counted as an item. * denotes the correct answer.
https://doi.org/10.1371/journal.pone.0204678.s001
(DOCX)
S1 Dataset. Study sample demographics and construct validity data.
https://doi.org/10.1371/journal.pone.0204678.s002
(CSV)
S2 Dataset. Study sample weight literacy score data.
https://doi.org/10.1371/journal.pone.0204678.s003
(CSV)
Acknowledgments
We thank Dr. Sherry Pagoto, Dr. Lori Pbert, and the University of Massachusetts Medical School medical students for their content expertise on scale development. We further thank the community organizations that supported study recruitment. We also acknowledge Cynthia Kambuni and Elizabeth Katz for their roles in conducting literature reviews and formatting the manuscript. The findings and conclusions in this article are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the National Institute of Minority Health and Health Disparities.
References
- 1.
Centers for Disease Control. National diabetes fact sheet: National estimates and general information on diabetes and prediabetes in the United States, 2011 [December 11, 2012]. Available from: http://www.cdc.gov/diabetes/pubs/estimates11.htm#4.
- 2. Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of childhood and adult obesity in the United States, 2011–2012. Jama. 2014;311(8):806–14. pmid:24570244.
- 3. Ogden CL, Carroll MD, Fryar CD, Flegal KM. Prevalence of Obesity Among Adults and Youth: United States, 2011–2014. NCHS Data Brief. 2015;(219):1–8. Epub 2015/12/04. pmid:26633046.
- 4. Finkelstein EA, Trogdon JG, Cohen JW, Dietz W. Annual medical spending attributable to obesity: payer-and service-specific estimates. Health affairs. 2009;28(5):w822–31. Epub 2009/07/29. pmid:19635784.
- 5.
In: Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health Literacy: A Prescription to End Confusion. Washington (DC)2004.
- 6. Lassetter JH, Clark L, Morgan SE, Brown LB, VanServellen G, Duncan K, et al. Health literacy and obesity among native Hawaiian and Pacific Islanders in the United States. Public health nursing. 2015;32(1):15–23. Epub 2014/10/03. pmid:25273848.
- 7. Cheng YL, Shu JH, Hsu HC, Liang Y, Chou RH, Hsu PF, et al. High health literacy is associated with less obesity and lower Framingham risk score: Sub-study of the VGH-HEALTHCARE trial. PloS one. 2018;13(3):e0194813. Epub 2018/03/29. pmid:29590183; PubMed Central PMCID: PMCPMC5874050.
- 8. Lieffers JRL, Arocha JF, Grindrod K, Hanning RM. Experiences and Perceptions of Adults Accessing Publicly Available Nutrition Behavior-Change Mobile Apps for Weight Management. Journal of the Academy of Nutrition and Dietetics. 2017. Epub 2017/06/20. pmid:28625662.
- 9. Lewis S, Thomas SL, Blood RW, Castle D, Hyde J, Komesaroff PA. 'I'm searching for solutions': why are obese individuals turning to the Internet for help and support with 'being fat'? Health expectations: an international journal of public participation in health care and health policy. 2011;14(4):339–50. Epub 2011/01/05. pmid:21199200; PubMed Central PMCID: PMCPMC5060594.
- 10. James DC, Pobee JW, Oxidine D, Brown L, Joshi G. Using the health belief model to develop culturally appropriate weight-management materials for African-American women. Journal of the Academy of Nutrition and Dietetics. 2012;112(5):664–70. Epub 2012/06/20. pmid:22709771.
- 11. James DC, Harville C, Efunbumi O, Martin MY. Health literacy issues surrounding weight management among African American women: a mixed methods study. J Hum Nutr Diet. 2015;28 Suppl 2:41–9. Epub 2014/06/04. pmid:24890122.
- 12. Carels RA, Selensky JC, Rossi J, Solar C, Hlavka R. A novel stepped-care approach to weight loss: The role of self-monitoring and health literacy in treatment outcomes. Eat Behav. 2017;26:76–82. Epub 2017/02/13. pmid:28189945.
- 13. Silk KJ, Sherry J, Winn B, Keesecker N, Horodynski MA, Sayir A. Increasing nutrition literacy: testing the effectiveness of print, web site, and game modalities. Journal of nutrition education and behavior. 2008;40(1):3–10. Epub 2008/01/05. pmid:18174098.
- 14. Hinkin TR. A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods. 1998;2(1):104–21.
- 15.
Willis GB. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, California: Sage Publications; 2005.
- 16. Block G, Gillespie C, Rosenbaum EH, Jenson C. A rapid food screener to assess fat and fruit and vegetable intake. American journal of preventive medicine. 2000;18(4):284–8. Epub 2000/05/02. pmid:10788730.
- 17. Johnson-Kozlow M, Rock CL, Gilpin EA, Hollenbach KA, Pierce JP. Validation of the WHI brief physical activity questionnaire among women diagnosed with breast cancer. American journal of health behavior. 2007;31(2):193–202. Epub 2007/02/03. pmid:17269909.
- 18.
Hatcher L. A Step-by-Step Approach to Using the SAS System for Factor analysis and Structural Equation Modeling. Cary, NC: SAS Institute, Inc; 1994.
- 19. Kuder GF, Richardson M. W. The theory of the estimation of test reliability. Psychometrika. 1937;2(3):151–60.
- 20. Wang ML, Pbert L, Lemon SC. Influence of family, friend and coworker social support and social undermining on weight gain prevention among adults. Obesity (Silver Spring, Md). 2014;22(9):1973–80. pmid:24942930; PubMed Central PMCID: PMC4435839.