https://orcid.org/0000-0002-2514-2342
Figures
Abstract
As data literacy has emerged as a critical skill for professionals across industries, educators in higher education have incorporated it into their curricula and instruction. Understanding and evaluating the factors that shape an individual’s data literacy is important for benchmarking proficiency and tailoring curricula, yet the underlying components and structure of data literacy for students in four-year institutions are unknown. This study validated the Data Literacy Self-Efficacy Scale (DLSES) with 1,816 students enrolled in two four-year institutions. Exploratory and confirmatory factor analyses were conducted to determine the construct of the scale, and the item analysis was used to address the validity of the items on the scale. The exploratory factor analysis identified eight distinct factors comprising 29 items. The results of confirmatory factor analysis showed a good model fit, CFI = 0.994, TLI = 0.994, RMSEA = 0.053, SRMR = 0.044. This study demonstrated the 29-item refined version of the DLSES to be a reliable and valid tool for measuring individuals’ self-efficacy levels for data literacy. Furthermore, the scale could form the basis for curriculum development and help educators design targeted interventions that address specific learning needs.
Citation: Kim J, Hong L, Yoon A (2025) University students’ self-assessment of data literacy: A validation study. PLoS One 20(4): e0322104. https://doi.org/10.1371/journal.pone.0322104
Editor: Yasir Ahmad,, National University of Sciences and Technology, PAKISTAN
Received: June 1, 2024; Accepted: March 16, 2025; Published: April 28, 2025
Copyright: © 2025 Kim et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data for this study are publicly available from the UNT Digital Library repository (https://digital.library.unt.edu/ark:/67531/metadc2358197).
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Data is now part of our everyday lives. As we contend with data’s impact on the nature of communication and knowledge, data literacy has emerged as an essential skill everyone requires in their personal and professional lives. Data literacy is a dynamic and evolving form of literacy and highly socially situated concept [1]. Literacy itself, indeed, is understood as “a collection of communicative and sociocultural practices shared among communities,” [2] and it continues to develop along with societal and technological shifts. This evolution has introduced terms like “information literacy,” “media literacy,” “computer literacy,” and “digital literacy” into our vocabulary, reflecting new demands and advancements. As a natural extension of these concepts, data literacy has recently gained significant attention across various fields, including education, information science, public health, and others [3–5].
In an era witnessing unprecedented data growth, developing data literacy has become critical in higher education, where teaching and research increasingly rely on data-intensive practices. Higher education, especially in four-year institutions, faces growing pressure to prioritize data literacy because of its benefits in a competitive labor market. As workplaces become more data-oriented and AI-driven, data literacy is expected to be one of the most in-demand skills [6,7]. Preparing students with this competency is vital for their workforce transition. Many institutions have taken steps to integrate data literacy into their curricula to address this need. These efforts include establishing dedicated data science departments, introducing data-related majors, offering undergraduate elective courses on data literacy, and expanding library programs to teach data literacy alongside traditional information literacy [8–10]. Such initiative aims to equip students with the skills they need to thrive in a data-driven world.
Given the recognized importance of data literacy, studies have explored the concept and components of data literacy, which is critical for tailoring curricula and training. Data literacy is defined as a multifaceted competency [11–13], combining cognitive abilities, critical thinking, and technical skills. It includes skills in identifying, cleaning, analyzing, and interpreting data; evaluating data sources and reliability; identifying data misuse; and utilizing data tools and technologies. Nevertheless, there is a dearth of systematic empirical work operationalizing this concept and developing reliable methods or scales for its measurement.
Recent research has designed data literacy scales. One example is a Data Literacy Self-Efficacy Scale (DLSES) developed by our research team, designed to measure students’ perceptions of their data literacy competency [14]. This scale was originally designed for use in higher education, but it was only validated in a community college setting. Previous validation efforts emphasized the need for further validation to strengthen the scale’s reliability and assess its applicability in diverse contexts [14,15]. Notably, it was recommended that additional validation should target four-year universities, which provide students with a comprehensive and specialized education, preparing them for professional careers or advanced academic studies.
Therefore, the purpose of the current study is to validate the DLSES with students at four-year institutions, including undergraduate and graduate students, who typically face higher expectations and demands for data literacy competency. The study aims to identify the scale’s underlying factor structure and explore its the validity for a broader application within higher education.
Materials and methods
Adapting the DLSES
The original version of the DLSES was created to measure students’ perceived proficiency levels in data literacy. The scale comprised 33 items grouped into ten domains: Data Awareness, Data Collection, Data Cleaning, Data Analysis, Data Visualization, Data Storytelling, Data Quality Evaluation, Data Organization, Data Storage, and Data Ethics [14]. In the current study, we included all 33 items from the original scale rather than the 31 items retained in our previous validation; this decision was made because the focus of our current study is not investigating the replicability of the factor structure. Instead, it aims to propose an alternative factor structure for a different population – i.e., students at four-year institutions.
A minor modification was made to the response format. Originally, scale items were rated from novice to expert, but the subjective nature of terms like “novice” and “expert” could bias respondents’ perceptions. To mitigate this, the response categories were revised using the framework proposed by Sherer’s Self-Efficacy Scale [16], allowing participants to respond to the items using a 5-point Likert scale, ranging from “strongly disagree” to “strongly agree.” Each item was phrased with “I believe I can successfully…” to reflect this approach.
The survey questionnaire incorporated this modification along with questions about students’ demographic and academic characteristics, including age, gender, race, year in college, field of study, and enrollment status. It was piloted with a small convenience sample of students to estimate completion time, assess the item’s readability, and evaluate the scale’s usability.
Procedure
The survey questionnaire, deployed via Qualtrics, was disseminated to students at two institutions classified as “Doctoral Universities: Very High Research Activity” by Carnegie Classification. These institutions are also categorized as “Four-year, Large” with fall enrollment data showing at least 10,000 degree-seeking students in full-time equivalent (FTE) enrollment. As state universities, they offer a greater variety of undergraduate, graduate, and doctoral programs across various fields of study to serve diverse student populations. From February 1, 2024 to March 17, 2024, the invitation to the survey was distributed through college/departmental listservs and a student Facebook group. A total of 2,671 responses were collected. This sample size was deemed sufficient for this study, as it is generally recommended to have at least ten participants for each item in the scale [17].
This study was approved as exempt by the University of North Texas Institutional Review Board and Indiana University Human Research Protection Program.
Data analysis
Data screening was conducted to make sure that data accurately represented what was measured and all the data met the underlying statistical assumptions. First, careless respondents who either selected the same answer for every item or completed the survey in a very short time were removed [18]. Then, outliers were assessed using univariate z-scores ±3.29 [19] and Mahalanobis distance [17].
After the data cleaning process, a total of 1,816 valid responses remained. This data was then randomly and evenly split for the exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) [20,21]. The latent constructs of the scale were assessed using factor analysis, a critical step for scale validation [22]. Specifically, EFA with the first data half and then replicating the factor structure using CFA. To determine the sampling adequacy of data for factor analysis, the Bartlett’s test of sphericity was conducted [23]. Given the Likert nature of the items, a polychoric correlation matrix was used as an input for EFA. Principal axis factoring and oblique rotation were applied as they do not require the data to be normally distributed [24]. The optional number of factors to retain was determined by using parallel analysis, Very Simple Structure [25], and Velicer Minimum Average Partial (MAP) [26] along with Bayesian Information Criterion (BIC) value [27]. Item deletion and retention were determined based on specific criteria, including factor loadings (with a cutoff of ≥ 0.40), the absence of cross-loading between items, and a minimum of three items for each factor [21,23]. After determining the number of factors and items to retain, each factor was labeled based on the concepts represented by the items in that category. The three authors reached a consensus on the labeling of the factors.
Then CFA was conducted to confirm the factor structure obtained from the EFA. The CFA was implemented with the remaining 908 survey responses using the lavaan package in R (version 4.3.0). The diagonally weighted least squares estimator was implemented to account for the ordered nature of the Likert items. To assess the model’s fit with the data, the following indices and criteria were used: a relative chi-square with a non-significant result (p > 0.05), Comparative Fit Index (CFI) > 0.90, Tucker-Lewis Index (TLI) > 0.90, Root Mean Square Error of Approximation (RMSEA) within 0.05 and 0.08, and Standardized Root Mean Square Residual (SRMR) < 0.08 [17,28].
Cronbach’s alpha was employed to assess the consistency of the scale. The minimum acceptable value for Cronbach’s alpha coefficient is 0.70 [21]. Additionally, item-level analysis was conducted to further investigate the performance of individual items within the scale and to identify any potential revisions needed for future validation. For each item, item-level statistics were conducted, including mean, standard deviation, and response distributions. Item-total correlations were calculated to assess how well each item relates to the rest of the scale and its ability to differentiate between respondents with higher or lower overall responses.
Results
Participants characteristics
As presented in Table 1, the participants exhibited diverse demographic and educational backgrounds, with almost half falling within the 20–24 age range. This aligns with the fact that nearly 40% of survey participants are undergraduate students. The gender distribution was slightly skewed toward females, with 57.88% female and 39.05% male. The race/ethnicity distribution was also imbalanced, with 42.49% of survey participants being Asian, 38.89% being white, and the remainder comprising other groups.
Students from various academic majors, such as computer science, business analytics, accounting, education, psychology, history, criminal justice, and economics participated in the survey. After converting the primary academic major input into the Integrated Postsecondary Education Data System (IPEDS) code, it was found that 60.89% were from science majors, encompassing fields such as computer science, health science, and life science, followed by other non-science and engineering majors. Among these students, 80.42% were full-time, and 19.58% were part-time.
Exploratory factor analysis results
The KMO was 0.95, well above the commonly accepted threshold of 0.6 [19], indicating that the sample size is sufficient. Bartlett’s test chi-square was statistically significant, with χ² (528) = 39784.978, p < 0.001, suggesting that factor analysis was appropriate [19].
The EFA yielded a 29-item instrument that accounted for 76% of the variance in the scale. Four of the original 33 items were dropped from subsequent analyses because they had extremely low item-total correlations or high coefficients on more than one factor. The final model contained eight factors, as confirmed by parallel analysis, Velicer MAP, and BIC.
The factor pattern and factor structure coefficients are presented in Table 2, along with communalities (h2) of the measured variables. All 29 items had communalities of at least 0.60 and above [29]. It should be noted that the pattern coefficient of some factors is slightly greater than 1.0 due to the use of an oblique rotation, which allows the factors to correlate; this does not imply an error but rather indicates a high degree of multicollinearity in the data [30].
For the full statements corresponding to their respective item numbers, please refer to Appendix 1 in the S1 Appendix.
The first factor, Data Awareness, has high loadings on three items related to understanding the foundational concepts and use of data. The second factor, Data Collection, comprises three items covering topics such as identifying the source of data and using appropriate methods to collect data. The third factor, Data Analytics, has six items with factor loadings ranging from 0.67 to 1.02. This factor reflects items on methods and techniques in data cleaning and analysis. The fourth factor, labeled Data Visualization, consists of three items that pertain to data visualization methods and the interpretation of visualization. The fifth factor is Data Storytelling, which includes three items about presenting and communicating data insights to others using different methods or techniques. The sixth factor, Data Quality Evaluation, consists of three items with factor loadings of 0.77, 0.79, and 1.08. The items are centered on the evaluation of data use and data quality. The seventh factor, Data Management, has four items relevant to the documentation and storage of data for share and reuse. The item DO 8.3 has a relatively low factor loading of 0.41; this implies that this specific item has a moderate relationship with the broader concept of data management and could indicate that the item is less critical or less directly related to the core components of data management. The last factor identified is Data Ethics, which comprises four items focused on understanding laws, regulations, and guidelines relevant to data use in terms of privacy, security, and intellectual property. The items have loadings ranging from 0.75 to 0.91.
A reliability analysis was conducted to assess the scales’ internal consistency. The Cronbach’s alpha ranges from 0.77 to 0.90 for the eight factors, which all exceeded the acceptable threshold of 0.70 [31], indicating good internal consistency.
Confirmatory factor analysis results
The 29-item model with an eight-factor solution extracted from the EFA was used for the CFA. The model was structured with each item assigned to its corresponding factor, and the factors were allowed to correlate. This is based on the EFA results and reflects the theoretical perspective that the factors are distinct but interrelated. The model’s fit was assessed through the fit indices discussed in the Data Analysis section of this paper, all of which indicated that the model fit the data well; χ2 (349) = 1225.60, p <.001. The model fit had a good model fit, CFI = 0.994, TLI = 0.994, RMSEA = 0.053, SRMR = 0.044.
Average variance extracted (AVE) ranged from 0.64 to 0.82 across the eight factors, which is above the 0.50 recommendation [21]. This indicated that between 64% and 82% of the indicator variance is part of the construct score. The composite reliabilities with a correction for the ordinal nature of the items were extracted and ranged from 0.8 to 0.95 across the eight factors, above the 0.70 recommendation [32].
Fig 1 denotes the CFA results. The standardized loadings of all items were in the range of 0.74 to 0.92, indicating strong associations. All items’ loadings were statistically significant (p < 0.001), which confirms that the items are good indicators of their respective factors. The correlation among the scale factors is between 0.31 and 0.70, and the correlation between each factor is significant (p < 0.01). This indicates that eight factors constitute the components of students’ self-assessed data literacy, but they are related constructs.
Item analysis results
Psychometric analysis at the item level is presented in Table 3, which shows the descriptive statistics for the items, item-total correlations, and the Cronbach’s alpha coefficients. DCO 2.1 “Collecting primary data...” and DCO 2.2 “Locating secondary data...” have the highest mean among all items, whereas DA 4.4 “Applying data mining to explore patterns and trends of data” has the highest standard deviation. Items belonging to Data Management and Data Ethics exhibit slightly lower means and higher standard deviations than other items.
All items had item-total correlations of at least 0.76, well above the acceptable threshold of 0.40 [33]; this indicates that each item demonstrated satisfactory discrimination. The coefficient value for each factor ranged from 0.76 to 0.89, exceeding the acceptable Cronbach’s alpha level of 0.70 for scale development; this indicates strong convergence and internal consistency.
Discussion
This study was the first to examine the factor structure of the DLSES among students enrolled at four-year institutions. Given the variability of factor structure across different populations [23], EFA was used as an inductive process to identify a factor structure that may be unique to university students. Indeed, the study resulted in eight factors on the scale, which were significantly correlated with one another: Data Awareness, Data Collection, Data Analytics, Data Visualization, Data Storytelling, Data Quality Evaluation, Data Management, and Data Ethics. A subsequent CFA supported the factors found in the EFA; this proves that the DLSES measured and reflected the aforementioned eight latent components, providing empirical evidence to build the construct validity of the scale.
The factor structure derived from the current study differed from the previous study done with community college students [14]. This difference could arise from differences in both exposure and experience with data literacy and actual competency levels among various populations. Due to their longer and more specialized academic training, students at four-year institutions may have a higher level of proficiency and familiarity with data-related concepts and skills. Therefore, they may show a more refined and comprehensive understanding of data literacy than community college students. This is further supported by the results showing that the mean scores of items across the scale for university students range from 3.40 to 4.27, higher than the scores of community college students in the previous study, which ranged from 2.39 to 2.99 [14].
It is also important to highlight that the Data Management and Data Ethics competencies have lower mean scores compared to other areas. This is likely because these competencies are often less emphasized in data literacy. Additionally, the fact that 69% of the sample consists of students in STEM fields, who typically engage more directly with data than non-STEM students, may also contribute to this result. Their coursework and research frequently involve data analysis, interpretation, and the application of statistical methods, leading to stronger data skills due to the emphasis on quantitative analysis and problem-solving in STEM disciplines. The slightly higher standard deviations observed for Data Management and Data Ethics also reflect variability in participants’ responses, which may be attributed to differences in understanding, experiences, or beliefs regarding these competencies. This suggests that more attention should be given to developing these competencies, which often require critical thinking and a deeper understanding of how to manage data effectively and handle it responsibly. This approach will help bridge the observed gaps and improve overall data literacy in education.
Limitations and future recommendations
A limitation of this study is the potential for biased sample selection, which may affect the reliability of the findings. Given that participation in the survey was self-selected, it is possible that those who volunteered had a specific interest in data literacy or had confidence in their data skills. To address this, further psychometric evaluations are needed to validate the scale across different groups. Future research should include a more diverse and representative sample from various academic disciplines and consider cross-cultural validation to improve generalizability.
Moreover, it is worth emphasizing that scale development and validation are iterative processes involving multiple stages of refinement and testing to ensure reliability and validity [33]. Future studies should seek feedback from experts and potential users to improve the scale’s clarity, relevance, and comprehensiveness. Additionally, updates to data literacy competencies may be needed as the field evolves.
Conclusions
The DLSES validated in this study is anticipated to become a promising tool for researchers and educators. Researchers can use the scale to gauge individuals’ confidence in their skills to understand, analyze, and use data effectively across different academic disciplines. Also, given that there is a call for more proactive approaches to integrating data literacy training into current higher education, the scale can be used to screen students’ data literacy competency as a preliminary needs assessment. Educators can use this scale to identify areas where students may need additional support or instruction, ultimately helping students become more proficient and confident in navigating data.
Data literacy education is an emerging field of study, and there is a need for more strategies and best practices in teaching data literacy. This involves equipping students with the technical skills required to analyze and interpret data and fostering a deeper understanding of the context in which data is used. Moreover, given the growing emphasis on “critical data literacy” [34,35], the scope of data literacy education must expand beyond questioning data sources, analytical methods, and conclusions. It should also include understanding the origins of data, the perspectives of those managing it, the broader implications of data analysis and interpretation, and the ethical dimensions of its use.
References
- 1. Barone D. The fluid nature of literacy: a look at how the definition of literacy has changed, and what it means for today’s students. Lit Today. 2015;33(1):6–8.
- 2. National Council of Teachers of English. Definition of literacy in a digital age. 2019. Available from: https://ncte.org/statement/nctes-definition-literacy-digital-age/.
- 3. Coners A, Matthies B, Vollenberg C, Koch J. Data skills for everyone! – an approach to assessing the integration of data literacy and data science competencies in higher education. J Stat Data Sci Educ. 2024;1:1–37.
- 4. Burress T, Mann E, Neville T. Exploring data literacy via a librarian-faculty learning community: a case study. J Acad Librariansh. 2020;46(1):102076.
- 5. Shah J, Bather JR, Chen Y, Kaul S, Dias JJ, Goodman MS. Increasing interest in data literacy: the quantitative public health data literacy training program. J Stat Data Sci Educ. 2024:1–13.
- 6. Vogt J, Voigt T, Nowak A, Pawlowski JM. Development of a job advertisement analysis for assessing data science competencies. Data Sci J. 2023;22.
- 7. Mei L, Feng X, Cavallaro F. Evaluate and identify the competencies of the future workforce for digital technologies implementation in higher education. J Innov Knowl. 2023;8(4):100445.
- 8. Pratsri S, Nilsook P, Wannapiroon P. Synthesis of data science competency for higher education students. Int J Educ Inf Technol. 2022;16:101–9.
- 9. Marchant M, Eliason E. Evaluating the inclusion of data literacy in undergraduate economics curriculum at research-intensive universities. J Educ Bus. 2024;99(6):390–9.
- 10. Taş E. Data literacy education through university-industry collaboration. Inf Learn Sci. 2023;125(5/6):389–405.
- 11. Calzada Prado J, Marzal MÁ. Incorporating data literacy into information literacy programs: core competencies and contents. Libri. 2013;63(2).
- 12. Wolff A, Gooch D, Cavero Montaner JJ, Rashid U, Kortuem G. Creating an understanding of data literacy for a data-driven society. J Commun Inf. 2016;12(3).
- 13. Gebre E. Conceptions and perspectives of data literacy in secondary education. Br J Educ Technol. 2022;53(5):1080–95.
- 14. Kim J, Hong L, Evans S. Toward measuring data literacy for higher education: developing and validating a data literacy self-efficacy scale. J Assoc Inf Sci Technol. 2024;75(8):916–31.
- 15. Kim J, Hong L, Evans S, Oyler-Rice E, Ali I. Development and validation of a data literacy assessment scale. Proc Assoc Inf Sci Technol. 2023;60(1):620–4.
- 16. Sherer M, Maddux J, Mercandante B, Prentice-Dunn S, Jacobs B, Rogers R. The self-efficacy scale: construction and validation. Psychol Rep. 1982;51(2):245–54.
- 17.
Kline RB. Principles and practice of structural equation modeling. 4th ed. New York: Guilford Publications; 2016.
- 18. Ward MK, Meade AW. Dealing with careless responding in survey data: prevention, identification, and recommended best practices. Annu Rev Psychol. 2023;74:577–96. pmid:35973734
- 19.
Tabachnick BG, Fidell LS. Using multivariate statistics. 7th ed. New York: Pearson; 2019.
- 20. Carpenter S. Ten steps in scale development and reporting: a guide for researchers. Commun Methods Meas. 2017;12(1):25–44.
- 21.
Joseph F. Hair J, Hult GTM, Ringle CM, Sarstedt M. A primer on partial least squares structural equation modeling (PLS-SEM). 3rd ed. Los Angeles: Sage Publications; 2021.
- 22. Clark LA, Watson D. Constructing validity: new developments in creating objective measuring instruments. Psychol Assess. 2019;31(12):1412–27. pmid:30896212
- 23. Watkins MW. Exploratory factor analysis: a guide to best practice. J Black Psychol. 2018;44(3):219–46.
- 24. Costello A, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10(1):7.
- 25. Revelle W, Rocklin T. Very simple structure: an alternative procedure for estimating the optimal number of interpretable factors. Multivariate Behav Res. 1979;14(4):403–14. pmid:26804437
- 26. Velicer WF. Determining the number of components from the matrix of partial correlations. Psychometrika. 1976;41(3):321–7.
- 27. Schwarz G. Estimating the dimension of a model. Ann Stat. 1978;6(2):461–4.
- 28.
Schumacker RE, Lomax RG. A beginner’s guide to structural equation modeling. 3rd ed. New York: Routledge; 2010.
- 29. MacCallum RC, Widaman KF, Zhang S, Hong S. Sample size in factor analysis. Psychol Methods. 1999;4(1):84–99.
- 30. Jöreskog KG. How large can a standardized coefficient be. 1999. Available from: https://www.statmodel.com/download/Joreskog.pdf.
- 31.
Nunnally JC, Bernstein IH. Psychometric theory. 3rd ed. New York: McGraw-Hill; 1994.
- 32.
Joseph F. Hair J, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 8th ed. Edinburgh Gate: Pearson; 2013.
- 33. Ebel R. Confidence weighting and test reliability. J Educ Meas. 1965;2(1):49–57.
- 34.
Louie J. Critical data literacy: Creating a more just world with data. Technical Report. Workshop on Foundations of Data Science for Students in Grades K-12. Washington, DC: National Academy of Sciences; 2022.
- 35. Atenas J, Havemann L, Timmermann C. Reframing data ethics in research methods education: a pathway to critical data literacy. Int J Educ Technol High Educ. 2023;20(1):11. pmid:36846483