https://orcid.org/0000-0002-2870-8364
Figures
Abstract
With the increase in digitalization of adolescents and health services, the population must demonstrate digital health literacy skills to be able to navigate online health information, as well as identify, evaluate, and apply relevant information. The present study aimed to investigate the psychometric properties of the adapted version of the eHealth Literacy Scale (eHEALS-BrA) for Brazilian adolescents. This study was conducted between September 2022 and May 2023, involving a total of 260 adolescents aged 13 to 19, with a mean age of 15.64 years (SD = 1.84), all of whom were enrolled in public schools in Brazil. Structural validity was evaluated by confirmatory factor analysis (CFA). The reliability of the instrument was assessed using Cronbach’s alpha (α) and McDonald’s omega (ω), and test-retest reliability. Questionnaires about sociodemographic aspects, health-related characteristics, and internet use were administered and used as discriminant validation measures. Convergent Validity was determined by correction with the domains of the World Health Organization Quality of Life-Bref (WHOQOL-Bref) instrument. For test-retest reliability, 67 participants completed the eHEALS-BrA a fortnight later. The average eHEALS-BrA score was 28.05 points (SD = 5.05). CFA revealed that the model with the best fit had a single factor (χ2 = 49.884 [df = 20, p = 0.0002], CFI = 0.934, TLI = 0.908, RMSEA = 0.076(CI:0.05–0.10), and SRMR = 0.045). The instrument demonstrated good reliability, with satisfactory internal consistency (α and ω = 0.71) and stability (ICC = 0.82, 95% CI:0.70–0.89). The eHEALS-BrA was correlated with the physical, psychological, and environmental domains of the WHOQOL-Bref (p = 0.001) (convergent validity). Higher scores were found among male adolescents, individuals who practiced physical activity, those with better self-rated health, those who considered the internet useful for searching for health information (discriminant validity), and those with better self-rated skills related to using the internet (convergent validity) (p < 0.05). The eHEALS-BrA demonstrated adequate psychometric properties for measuring digital health literacy in Brazilian adolescents.
Citation: Barbosa MCF, Baldiotti ALP, Resende JL, Perazzo MdF, Firmino RT, Granville-Garcia AF, et al. (2024) Assessment of the psychometric properties of the eHealth Literacy Scale for Brazilian adolescents. PLoS ONE 19(11): e0314099. https://doi.org/10.1371/journal.pone.0314099
Editor: Rodrigo Toniol, Federal University of Rio de Janeiro: Universidade Federal do Rio de Janeiro, BRAZIL
Received: July 2, 2024; Accepted: November 6, 2024; Published: November 25, 2024
Copyright: © 2024 Barbosa et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This study was funded by Universidade Federal de Minas Gerais (UFMG - Federal University of Minas Gerais) and the following Brazilian fostering agencies: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES - Coordination for the Advancement of Higher Education Personnel) and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPQ - National Council of Scientific and Technological Development) (CNPq N° 09/2022 - Process: 309181/2022-4 and CNPq N° 18/2021 - Process: 406204/2021-7), and Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG - Research Support Foundation of the State of Minas Gerais) (APQ-00360-22).
Competing interests: The authors have declared that no competing interests exist.
Introduction
Adolescence is the transition period between childhood and adulthood and is considered a stage of development in which important physical, neural, cognitive, behavioral, and psychological maturation occurs [1, 2]. Adolescents experience an increase in awareness about the body and health, a reduction in parental influence and a greater need for acceptance by peers [2, 3].
Adolescents are digital natives, that is, a generation that was born in the era of the popularization of the internet and that is growing up in a society that uses computers, tablets, and smartphones on a daily basis [2, 4]. Thus, social media are indispensable for the lives of these young people, who use digital platforms for entertainment purposes, searching for information, and communication [2, 5–7]. Moreover, adolescents often use the internet to better understand and manage their health. Indeed, the internet has become an attractive interactive resource with easy access to a wide range of health content and allows anonymous searches for sensitive information, such as that related to sexual and mental health [6, 8]. In Brazil, an estimated 90.2% of adolescents are connected to the internet and use this resource to obtain information [9, 10]. However, it is important to note that being born and raised in a highly digitalized environment does not necessarily mean they possess the skills to use it effectively [11].
The availability of high-quality health information in social media can significantly impact an individual’s health outcomes [6]. Nevertheless, information on the internet comes from a wide variety of servers and has sources that are difficult to control, which can lead to problems related to the quality of the information posted and the risk of biased content circulating in the media [12–14]. As adolescents could be highly sensitive to acceptance and are developing reflective processes and cognitive control, these young people may be vulnerable to misinformation and may have difficulty identifying reliable content [2, 6, 15].
Digital health literacy (DHL) can be defined as the ability of individuals to search, find, understand, and health information posted online and apply knowledge assertively to address or solve a health problem [16]. With this aspect in mind, Norman and Skinner developed the eHealth Literacy Scale (eHEALS) [17], a widely used self-report scale composed of eight items that measure DHL in a wide range of age groups and populations [18–22]. This instrument has recently been validated for adults [23] (18 to 80 years old) and university students [24] in Brazil.
Considering the peculiarities of adolescence and the importance of measuring DHL in this age group, eHEALS was recently cross-culturally adapted for adolescents in Brazil [25]. For this adaptation, the following steps were carried out: a) assessment and adequacy of cultural equivalence by a committee of experts; b) back-translation; c) synthesis of back-translations; and d) cognitive testing with 42 Brazilian adolescents, using cognitive interviews with probing questions. However, for recommendation and utilization, it is necessary to conduct a psychometric evaluation of the adapted instrument with larger samples of Brazilian adolescents, including an analysis of its dimensional structure through test-retests and assessment of construct validity, using convergent and divergent validity [25]. Therefore, the aim of the present study was to assess the psychometric properties of the adapted version of the eHealth Literacy Scale (eHEALS-BrA) for Brazilian adolescents.
Methods
Sample
This methodological study assessed the psychometric properties of the eHealth Literacy Scale, which was previously adapted for Brazilian adolescents (eHEALS-BrA) [25]. The sample for this study comprised adolescents aged 13 to 19 who were enrolled in five public schools conveniently selected in Belo Horizonte, Minas Gerais, Brazil. The sample size was based on the recommendation of Anthoine et al. [26]: two to 20 individuals for each item on the instrument and an absolute minimum of 100 to 250 individuals. Thus, we determined a minimum sample of 160 participants (20 per item). For greater representativeness, schools distributed throughout the municipality were selected randomly and had different basic education development indices (IDEB-2019), an indicator of the quality of teaching at Brazilian public schools.
Adolescents of both sexes, native speakers of Brazilian Portuguese with access to the internet, and who were present on the data collection days were included. Adolescents up to 12 years of age were excluded due to the 1998 Children’s Online Privacy Protection Act, which determines a minimum age of 13 years for creating accounts on social media and using digital services. Adolescents who were illiterate and those who had self-reported or school-reported problems (vision, hearing, or cognitive) that made it impossible to participate in the study were also excluded.
Instrument
The eHealth Literacy Scale is a self-report instrument for measuring digital health literacy [17] developed in Canada’s English language and cultural context. The scale consists of eight items and the response options for each item are organized on a five-point Likert scale ranging from “strongly disagree” to “strongly agree”, depending on the respondent’s perception. The total score ranges from 8 to 40 points, with higher scores denoting a better self-perception of digital health literacy [17]. Despite having been validated for Brazilian adults [23] and university students [24], the version for adolescents in Brazil (eHEALS-BrA) [25] has not previously had its psychometric properties assessed. The full version of the eHEALS in Brazilian Portuguese adapted for adolescents is available in the S1 File of this manuscript.
Data collection
This study was conducted from August to December 2022. Guardians first answered a questionnaire on their sociodemographic characteristics (age, degree of kinship to the adolescent, education, and income) and those of the adolescent (birth order, change in health, and use of medications). The adolescents answered the following instruments: the eHEALS-BrA (digital health literacy) [25], the World Health Organization Quality of Life-Bref (WHOQOL-Bref) [27, 28], and a questionnaire addressing their demographic characteristics (sex, birth order, age, education, relationship to guardian, guardian’s schooling), general health (regular physical activity, self-assessment of general and oral health), internet access (place of access, main means of access, mobile internet, frequency of access, social network accounts), self-assessment of skills related to using the internet, and search behavior for health information (researched a doctor/dentist, followed bloggers’ recommendations, uses the internet to read/search for information on health, schedule appointments, use health apps, research symptoms, and ask questions to a health professional).
Additionally, we used two introductory questions from the original eHEALS instrument: “How useful do you think the internet is in helping you make decisions about your health?” (not at all useful, not very useful, not sure, useful, very useful) and “How important is it for you to be able to access health information/resources available on the internet?” (not important, not very important, not sure, important, very important) [17].
The WHOQOL-Bref is an abbreviated version of the World Health Organization Quality of Life questionnaire (WHOQOL-100) consisting of 26 questions with response options on a Likert scale (1 to 5) [27, 28].
To evaluate stability, the eHEALS-BrA was re-administered to 25% of the sample within a 15-day interval following the initial application. Participants were selected to ensure that representation from each school was maintained. As in the first application, they completed the printed version of the instrument in a self-administered format in a designated room at their school.
Statistical analysis
Data were analyzed using SPSS Statistics, version 21.0 (IBM Corp., Armonk, NY, USA) and Mplus software (Muthén & Muthén, version 8.2, Los Angeles, CA, USA). Confirmatory factor analysis (CFA) was performed to test the dimensionality of the instrument based on theoretical knowledge and empirical research [29]. One-, two-, and three-factor hypotheses [17, 21–24] were tested. In AFC the analytic approach used estimation adjusted using the mean- and variance-adjusted weighted least squares. Different statistical parameters were employed to assess the goodness of fit of the model, such as the Comparative Fit Index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). For the CFI and TLI, values ≥ 0.95 were considered indicative of an excellent fit and values between 0.90 and 0.95 were considered acceptable. RMSEA values ≤ 0.06 were considered indicative of an excellent fit of the model and values between 0.06 and 0.08 were considered acceptable [30, 31]. For the SRMR, values < 0.08 were considered indicative of a good fit [20, 21]. Communalities and factor loadings equal ≥ 0.4 were considered acceptable.
Reliability was investigated using two measures of internal consistency: Cronbach’s alpha (α) and McDonald’s omega (ω). For both coefficients, values ≥ 0.70 were considered adequate [32, 33]. Instrument stability and test-retest reliability were measured using the intraclass correlation coefficient (ICC), which was interpreted as follows: ≤ 0.40 = weak correlation; 0.41–0.60 = moderate correlation; 0.61–0.80 = good correlation; 0.81–1.00 = excellent correlation [30].
Construct validity was measured based on convergent and discriminant validity considering the eHEALS validation method and theoretical framework. The Mann-Whitney and Kruskal-Wallis U tests were used to identify the convergent and discriminant validity of the eHEALS-BrA by comparing the total score between categories of demographic variables, self-perception of general and oral health, internet access, and search behavior for health information on social media platforms. Spearman’s correlation coefficients were calculated to demonstrate the convergent validity of the eHEALS-BrA in relation to the WHOQOL-Bref.
Ethical aspects
This study was conducted with authorization from the authors of the original instrument [17], received approval from the Human Research Ethics Committee of the Federal University of Minas Gerais (certificate number: 58603022.8.0000.5149), and followed the ethical principles stipulated in Resolution 466/2012 of the National Board Health Council and the Declaration of Helsinki.
Results
The validity and reliability of the eHEALS-BrA were determined in a sample of 260 adolescents with a mean age of 15.63 years (SD: 1.84), 142 of whom (54.6%) were girls and 139 (53.7%) had up to eight years of schooling. Average age of the guardians was 43.35 years (± 8.14), with guardian’s education ≥ nine years of study (79.1%) and an average family income of R$2780.69 (US$526.64).
According to the CFA analysis, the model with the best fit was composed of a single factor. The goodness-of-fit statistics were χ2 = 49.884 (DF = 20, p = 0.0002), CFI = 0.934, TLI = 0.908, RMSEA = 0.076 (0.050–0.102) and SRMR = 0.045, indicating a good fit of the model. Factor loadings for the single-factor solution ranged between Q1 = 0.519 and Q4 = 0.656 (Table 1).
The mean eHEALS-BrA score was 28.05 (SD = 5.05; range: 12 to 40) and mean administration time was 3.06 minutes. The instrument demonstrated good reliability. Internal consistency of the total scale was adequate (Cronbach’s alpha = 0.71 and McDonald’s omega = 0.71). Table 2 displays details on the means, variances, and Cronbach’s alpha if an item were deleted from eHEALS-BrA as well as item-total correlations and communalities. Test-retest reliability analysis demonstrated excellent reproducibility [ICC = 0.818 (95% CI: 0.70 to 0.89, p < 0.001)].
Table 3 displays the mean eHEALS-BrA score according to demographic characteristics, health behaviors, and self-perception of general and oral health. Discriminant validity of the instrument was demonstrated by higher scores among male adolescents (p = 0.037), adolescents with better self-rated general health (p = 0.007), and those who performed physical activity regularly (p < 0.001). Table 4 displays the average eHEALS-BrA score according to access to social media and search behaviors for health information on the internet. Higher eHEALS-BrA scores were found among adolescents who considered the internet very useful/useful for making health decisions (p < 0.001) and those who considered it very important/important to be able to access health information and/or resources on the internet (p = 0.036).
Convergent validity of the eHEALS-BrA was demonstrated by the significant correlation with three domains of the WHOQOL-Bref: physical (r: 0.228, p < 0.001), psychological (r: 0.229, p < 0.001), and environmental (r: 0.298, p < 0.001). Convergent validity was also demonstrated by the higher eHEALS-BrA scores among adolescents with a ’very good’ self-assessment of their abilities to use the internet (p = 0.001), those who had previously researched (p = 0.003) and read (p = 0.006) health information on the internet, those who used health apps (p = 0.003), and those who asked health-related question to a professional online (p = 0.034).
Discussion
The use of an assessment measure in a different culture, population, or age group should only be recommended after a careful process of cross-cultural adaptation and the assessment of its psychometric properties [34, 35]. The present study demonstrated that the eHEALS-BrA has good properties for measuring digital health literacy in Brazilian adolescents. Furthermore, the instrument is simple and fast to apply (approximately three minutes) and requires minimal training from the professional.
The mean total eHEALS-BrA score is similar to that found for university students in Brazil (28.0) [24] and Turkish adolescents (27.52) [36]. However, the mean score was higher than that found for adults [23], which may be explained by the greater familiarity of adolescents with information and communication technologies, which can increase their self-confidence in using such technologies for health purposes [6]. Furthermore, as adolescents are in a period of physical, emotional, psychological, and cognitive maturation [2], they may present better self-assessment (eHEALS) than actual LDS skills. Recently, a study carried out with Australian adolescents highlighted the discrepancy between adolescents’ perceived and actual digital health literacy, measured for served practical search tasks and follow-up interviews [37].
Good correlations were found between the items and total score, indicating that no item should be removed from instrument [38]. As occurred in the analysis of the original instrument [17], CFA revealed that the single-factor model had the best fit, with factor loadings ranging from 0.519 to 0.656 for the eight items on the scale. The dimensionality of an instrument is a fundamental psychometric property to determine the number of factors necessary to characterize the construct adequately. The single-factor solution was supported in most validations of this instrument [24, 39–41].
The eHEALS-BrA exhibited adequate internal consistency demonstrating good reliability. These values are lower than those found in the validation study of the original instrument with Canadian adolescents and young adults (13–21 years of age) (α = 0.88) [17] and cross-cultural adaptations for adolescents in Serbia (α = 0.88) [42] as well as Brazilian adults (α and ω = 0.95) [23] and university students (α = 0.88) [24]. Nonetheless, assessment measures with α ≥ 0.70 are considered acceptable and recommended for use [32, 33].
Furthermore, although this study’s alpha value is not very high, it is important to highlight that instruments with a low number of questions, poor inter-item correlation, or heterogeneous constructs may interfere with the evaluation. Additionally, while Cronbach’s alpha is not the most appropriate measure of internal consistency for multidimensional scales, it is described in the literature as a reliable measure for unidimensional instruments, as is the case in this study [43]. Despite this, to confirm these findings, we also used Omega as a measure to evaluate internal consistency [33].
In terms of reproducibility, the eHEALS-BrA exhibited excellent test-retest stability. The original authors used Pearson’s correlation test between four administration times, during which smoking interventions were conducted [17]. Therefore, it is not possible to compare our results. However, the ICC in the present study was higher than that found in the validation study involving Brazilian university students with a similar interval between the test and retest (ICC = 0.71) [24].
As occurred in the validation study of the original instrument, the average digital health literacy score was higher in male adolescents [17]. Furthermore, the eHEALS-BrA score was associated with better outcomes in terms of quality of life and self-rated general health as well as regular physical activity among the adolescents. A higher eHEALS-BrA score was positively associated with a better self-perception of skills in using the internet, asking questions online to a professional, searching for and reading health information on the internet, and using health applications. In line with our results, a recent study with Serbian adolescents found that being male, reporting greater use of the internet to research health, using health applications, and performing better on the eHEALS were associated with a greater influence of online health information on health-related decision-making [44].
Although a randomly selected sample is preferable for validation studies, it is not necessary for analyzing the psychometric properties of an assessment measure. It’s important to have a sample that reflects and captures the range of the target population [45]. This study was conducted with adolescent students in public schools within the same municipality, where we randomly selected schools with varying assessments of teaching quality and distribution throughout the area municipality.
The eHEALS has some limitations, as it measures an individual’s perception of health information from electronic sources and does not encompass the practical assessment of skills in searching, finding, understanding, and assessing this information. Patient-reported outcome measures (PROMs) are questionnaires that collect subjective information directly from patients regarding specific or general conditions, contributing to clinical and functional outcomes by transforming unmeasurable subjective qualities into quantitative measures [46, 47]. In this research, we used two PROMS, eHEALS-BrA and WHOQOL-Bref. PROMs are essential for health investigations, as their completion prompts patients to reflect on their health and encourages them to raise healthcare issues.
The eHEALS scale has the limitation of not evaluating all aspects related to the current scenario of high digital interaction (Health 2.0). It is worth noting that while other instruments have been developed to measure DHL, eHEALS is the most widely used to investigate this construct due to its pioneering nature, versatility for different age groups and cultural contexts, evidence of adequate psychometric properties, and ease of application [41]. Furthermore, in this study, we used related constructs and variables to Digital Health Literacy to measure its convergent validity because, during the data collection period, we did not have another cross instrument- Culturally adapted and validated for measuring Digital Health Literacy among adolescents in Brazil to compare with eHEALS-BrA.
The eHEALS has some limitations, as it measures an individual’s perception of health information from electronic sources and does not encompass the practical assessment of skills in searching, finding, understanding, and assessing this information. Patient-reported outcome measures (PROMs) are essential for health investigations, as their completion prompts patients to reflect on their health and encourages them to raise healthcare issues. PROMs are questionnaires that collect subjective information directly from patients regarding specific or general conditions, contributing to clinical and functional outcomes by transforming unmeasurable subjective qualities into quantitative measures [46, 47]. In this research, we used two PROMS, eHEALS-BrA and WHOQOL-Bref. We utilized related constructs and variables to Digital Health Literacy to measure its convergent validity because, during the data collection period, we did not have another instrument that was cross-culturally adapted and validated for measuring Digital Health Literacy among adolescents in Brazil to compare with eHEALS-BrA.
Furthermore, the eHEALS scale has the limitation of not evaluating all aspects related to the current scenario of high digital interaction (Health 2.0) [41]. It is worth noting that while other instruments have been developed to measure DHL, eHEALS is the most widely used to investigate this construct due to its pioneering nature, versatility for different age groups and cultural contexts, evidence of adequate psychometric properties, and ease of application [41].
With the acceleration of the digitalization of health services as a result of the COVID-19 pandemic and the worsening of the infodemic, there was a need to measure digital health literacy to identify the current situation in the country, formulate measures to promote the improvement of DHL, and disseminate reliable information based on scientific evidence to combat misinformation [48, 49].
The adaptation of eHEALS for Brazilian adolescents was the first attempt to obtain an instrument for measuring DHL in this age group in Brazil [25]. However, it was necessary to provide evidence of its psychometric properties. This study stands out due to the emerging need to investigate digital health literacy in different age groups to understand the challenges and disparities in access to health information in social media and the use of digital health services by the population. Future research is needed to assess the DHL of adolescents in Brazil, investigate gender differences in digital health literacy, and explore the associated health outcomes. Furthermore, it is necessary to advance instruments in Brazilian Portuguese that measure HDL more broadly (Health 1.0 and 2.0).
Conclusion
The eHEALS-BrA demonstrated adequate psychometric properties for measuring the digital health literacy of Brazilian adolescents.
Acknowledgments
The authors would like to thank the Federal University of Minas Gerais (UFMG) and the Belo Horizonte Municipal Department of Education for making this research possible. In addition, they would like to thank the members of the expert committee who helped with the cross-cultural adaptation of the instrument.
References
- 1. Gaete V. Adolescent psychosocial development. Rev Chil Pediatr. 2015;86(6): 436–43.
- 2. Crone EA, Konijn EA. Media use and brain development during adolescence. Nat Commun. 2018;9(1): 588. pmid:29467362
- 3. Ferreira EZ, Oliveira AMN, Medeiros SP, Gomes GC, Cezar-Vaz MR, Ávila JA. Internet influence on the biopsychosocial health of adolescents: An integratitive review. Rev Bras Enferm. 2020;73: e20180766. pmid:32236377
- 4. Stahl T. How ICT savvy are digital natives actually? Nord J Digit Lit. 2017;12: 89–108.
- 5. Freeman JL, Caldwell PHY, Bennett PA, Scott KM. How Adolescents Search for and Appraise Online Health Information: A Systematic Review. J Pediatr. 2018;195: 244–55. pmid:29398062
- 6. Park E, Kwon M. Health-related Internet use by children and adolescents: systematic review. J Med Internet Res. 2018;20(4): e120. pmid:29615385
- 7. Brown JD, Bobkowski PS. Older and newer media: patterns of use and effects on adolescents’ health and well-being. J Res Adolesc. 2011;21(1): 95–113.
- 8. Wartella E, Rideout V, Montague H, Beaudoin-Ryan L, Lauricella A. Teens, health and technology: a national survey. Media Commun. 2016;4(3): 13–23.
- 9. IBGE. Pesquisa Nacional por Amostra de Domicílios Contínua: acesso à Internet e à televisão e posse de telefone móvel celular para uso pessoal, 2019 [cited 2023 Mar 11]. Available from: https://www.ibge.gov.br/estatisticas/sociais/populacao/17270-pnadcontinua.html?edicao=30362
- 10. Núcleo de Informação e Coordenação do Ponto BR (NIC.br). Pesquisa sobre o uso da Internet por crianças e adolescentes no Brasil: TIC Kids Online Brasil, 2022 [cited 2023 Mar 12]. Available from: http://cetic.br/pt/arquivos/kidsonline/2022/pais
- 11. Sánchez J, Salinas A, Contreras D, Meyer E. Does the new digital generation of learners exist? A qualitative study. Br J Educ Technol. 2011;42(4): 543–56.
- 12. Zanatta ET, Wanderley GPM, Branco IK, Pereira D, Kato LH, Maluf EMCP. Fake news: the impact of the internet on population health. Rev Assoc Med Bras. 2021;67(7): 926–30.
- 13. Fast AM, Deibert CM, Hruby GW, Glassberg KI. Evaluating the quality of Internet health resources in pediatric urology. J Pediatr Urol. 2013;9(2): 151–6. pmid:22281281
- 14. Kim K, Shin S, Kim S, Lee E. The Relation Between eHealth Literacy and Health-Related Behaviors: Systematic Review and Meta-analysis. J Med Internet Res. 2023;25: e40778. pmid:36716080
- 15. Sebastian CL, Tan GCY, Roiser JP, Viding E, Dumontheil I, Blakemore SJ. Developmental influences on the neural bases of responses to social rejection: implications of social neuroscience for education. Neuroimage. 2011;57: 686–94. pmid:20923708
- 16. Norman CD, Skinner HA. Ehealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. 2006;8(2): e9. pmid:16867972
- 17. Norman CD, Skinner HA. eHEALS: the eHealth Literacy Scale. J Med Internet Res. 2006;8(4): e27. pmid:17213046
- 18. Brors G, Wentzel-Larsen T, Dalen H, Hansen TB, Norman CD, Wahl A, et al; CONCARD Investigators. Psychometric Properties of the Norwegian Version of the Electronic Health Literacy Scale (eHEALS) Among Patients After Percutaneous Coronary Intervention: Cross-Sectional Validation Study. J Med Internet Res. 2020;22(7): e17312. pmid:32720900
- 19. Giger LT, Barnhart S, Feltner F, Slone M, Lawler MJ, Windsor L, et al. Validating the eHealth Literacy Scale in Rural Adolescents. J Rural Health. 2021;37(3): 504–16. pmid:32881112
- 20. Paramio Pérez G, Almagro BJ, Hernando Gómez Á, Aguaded Gómez JI. Validation of the eHealth Literacy Scale (eHEALS) in Spanish University Students. Rev Esp Salud Publica. 2015;89(3): 329–38.
- 21. Tomas CT, Queiros PJP, Ferreira TDJR. Analysis of the psychometric properties of the Portuguese version of an eHealth literacy assessment tool. Referência. 2014;4(2): 19–28.
- 22. Xu RH, Zhou L, Lu SY, Wong EL, Chang J, Wang D. Psychometric Validation and Cultural Adaptation of the Simplified Chinese eHealth Literacy Scale: Cross-Sectional Study. J Med Internet Res. 2020;22(12): e18613.
- 23. Mialhe FL, Moraes KL, Sampaio HAC, Brasil VV, Vila VSC, Soares GH, et al. Evaluating the psychometric properties of the eHealth Literacy Scale in Brazilian adults. Rev Bras Enferm. 2022;75(1): e20201320.
- 24. Lotto M, Maschio KF, Silva KK, Ayala Aguirre PE, Cruvinel A, Cruvinel T. eHEALS as a predictive factor of digital health information seeking behavior among Brazilian undergraduate students. Health Promot Int. 2021: daab182.
- 25. Barbosa MCF, Baldiotti ALP, Dias MLLS, Granville-Garcia AF, Paiva SM, Ferreira FM. Cross-cultural adaptation of the eHealth Literacy Instrument (eHEALS) for application in Brazilian Adolescents. Braz Oral Res. 2024;38: e094.
- 26. Anthoine E, Moret L, Regnault A, Sébille V, Hardouin JB. Sample size used to validate a scale: a review of publicationson newly-developed patient reported outcomes measures. Health Qual Life Outcomes. 2014;12(1): 176.
- 27. Fleck MPA, Louzada S, Xavier M, Chamovich E, Vieira G, Santos L, et al. Aplicação da versão em português do instrumento abreviado de avaliação da qualidade de vida “WHOQOL-bref”. Rev Saude Publica. 2000;34(2): 178–88.
- 28. The Whoqol Group: The word Health Organization Quality of Life Assessment (WHOQOL): Position paper from the Health Organization. Soc Sci Med. 1995;41(10): 1403–09.
- 29.
Norman, Streiner. Biostatistics: the bare essentials. 3.ed. Raleigh, NC: PMPH USA; 2008.
- 30.
Kline RB. Principles and practice of structural equation modeling. 4rd ed. New York: Guilford Press; 2015.
- 31. Pakpour AH, Yekaninejad MS, Zarei F, Hashemi F, Steele MM, Varni JW. The PedsQL Oral Health Scale in Iranian children: reliability and validity. Int J Paediatr Dent. 2011,21: 342–52.
- 32. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3): 297–334.
- 33. Dunn TJ, Baguley T, Brunsden V. From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation. Br J Psychol. 2014;105(3): 399–412. pmid:24844115
- 34. Karanicolas PJ, Bhandari M, Kreder H, Moroni A, Richardson M, Walter SD, et al. Evaluating agreement: Conducting a reliability study. J. Bone Joint Surg Am. 2009;91: 99–106. pmid:19411507
- 35. Praveen S, Parmar J, Chandio N, Arora A. A Systematic Review of Cross-Cultural Adaptation and Psychometric Properties of Oral Health Literacy Tools. Int J Environ Res Public Health. 2021;18(19): 10422. pmid:34639729
- 36. Tümer A, Sümen A. E-health literacy levels of high school students in Turkey: results of a cross-sectional study. Health Promot Int 2022;37(2): daab174. pmid:34718540
- 37. Taba M, Allen TB, Caldwell PHY, Skinner SR, Kang M, McCaffery K, et al. Adolescents’ self-efficacy and digital health literacy: a cross-sectional mixed methods study. BMC Public Health. 2022;22(1): 1223. pmid:35725389
- 38.
Field A. Descobrindo a estatística usando o SPSS. 2ª ed. Porto Alegre: Artmed; 2009.
- 39. Mitsutake S, Shibata A, Ishii K, Okazaki K, Oka K. Developing Japanese version of the eHealth Literacy Scale (eHEALS). Nihon Koshu Eisei Zasshi. 2011;58(5): 361–71. pmid:21905612
- 40. Diviani N, Dima AL, Schulz PJ. A Psychometric Analysis of the Italian Version of the eHealth Literacy Scale Using Item Response and Classical Test Theory Methods. J Med Internet Res. 2017;19(4): e114. pmid:28400356
- 41. Lee J, Lee EH, Chae D. eHealth Literacy Instruments: Systematic Review of Measurement Properties. J Med Internet Res. 2021;23(11): e30644. pmid:34779781
- 42. Gazibara T, Cakic J, Cakic M, Pekmezovic T, Grgurevic A. eHealth and adolescents in Serbia: psychometric properties of eHeals questionnaire and contributing factors to better online health literacy. Health Promot Int. 2019;34(4): 770–8. pmid:29800141
- 43. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2: 53. pmid:28029643
- 44. Gazibara T, Cakic M, Cakic J, Grgurevic A, Pekmezovic T. Familiarity with the internet and health apps, and specific topic needs are amongst the factors that influence how online health information is used for health decisions amongst adolescents. Health Info Libr J. 2022: 1–8.
- 45. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health. 2020;6: 149.
- 46. Greenhalgh J, Gooding K, Gibbons E. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J Patient Rep Outcomes. 2018;2: 42. pmid:30294712
- 47. Krogsgaard MR, Brodersen J, Christensen KB, Siersma V, Kreiner S, Jensen J, et al. What is a PROM and why do we need it? Scand J Med Sci Sports. 2021;31(5): 967–971. pmid:33249660
- 48. Eysenbach G. How to Fight an Infodemic: The Four Pillars of Infodemic Management. J Med Internet Res. 2020;22(6): e21820.
- 49. Tangcharoensathien V, Calleja N, Nguyen T, Purnat T, D’Agostino M, Garcia-Saiso S, et al. Framework for Managing the COVID-19 Infodemic: Methods and Results of an Online, Crowdsourced WHO Technical Consultation. J Med Internet Res. 2020;22(6): e19659. pmid:32558655