https://orcid.org/0000-0002-4180-7244
Figures
Abstract
As the industrial structure changes, the severe shortage of high-quality technical and skilled talent in China is one of the most significant factors affecting the high-quality development of China’s economy. Bridging the gap between cultivating talent from new undergraduate vocational universities and the demand for industrial talent is regarded as an efficient strategy to address the talent shortage. In addressing the gap, China is hindered by a lack of clarity regarding student development goals and effective assessment instruments. Thus, this study aimed to use the Fuzzy Delphi Method (FDM) and the Analytical Hierarchy Process (AHP) to overcome the above challenges. Specifically, we used the FDM to establish a five-level undergraduate vocational education student development model with two 2nd-level indicators, three 3rd-level indicators, eight 4th-level indicators, and 33 5th-level indicators to clarify student development goals. Then, the AHP was applied to determine the indicator weights, and a student development assessment instrument was developed to help universities acquire student development data and improve the matching degree between talent supply and demand. This study could help undergraduate vocational universities cultivate high-quality technical and skilled talent quickly to meet the demand for China’s new economic system and to promote industry independence and global competitiveness.
Citation: Xing Q, Tan HP, Gan SW (2024) Constructing a student development model for undergraduate vocational universities in China using the Fuzzy Delphi Method and Analytic Hierarchy Process. PLoS ONE 19(3): e0301017. https://doi.org/10.1371/journal.pone.0301017
Editor: Yasir Ahmad, National University of Sciences and Technology, PAKISTAN
Received: October 25, 2023; Accepted: March 9, 2024; Published: March 22, 2024
Copyright: © 2024 Xing et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors declare that there are no competing interests.
1. Introduction
China urgently needs high-quality technical and skilled talent, due to the constant acceleration of industrialization and the emergence and development of new high-tech industries. In response to these talent requirements, China has established 32 undergraduate vocational universities since 2019 to cultivate high-quality technical and skilled talent. China’s Ministry of Education also declared that it would further encourage and develop them, with the aim of increasing the number of their students to no less than 10% of higher vocational education students by 2025. Now, undergraduate vocational universities are responsible for cultivating high-quality technical and skilled talent and promoting the high-quality development of China’s economy.
Improving the match between students’ abilities and the requirements for high-quality technical and skilled talent is the key to the effective development of undergraduate vocational universities. In recent years, the Ministry of Education, universities, and academics have paid close attention to the high-quality development of undergraduate vocational education. However, data on the ability structure and current development level of undergraduate vocational students remain limited [1]. Relevant policies have been established, but no clear and explicit criteria concerning the concept or student development goals exist. The existing theoretical achievements only focus on connotations, distinctions from other applied talent, development dilemmas, and principles of talent cultivation goals. Still, almost no research has been conducted on the talent that undergraduate vocational education needs to cultivate, its student development model, or the assessment instruments employed. The lack of clarity in goal orientation severely restricts the sustainability of undergraduate vocational education development in the future.
To tackle the above problems, it is imperative to establish a clear, pertinent, and actual student development goal (model) and to develop an effective student development assessment instrument for undergraduate vocational universities. In this study, we applied the Fuzzy Delphi Method (FDM) and the Analytical Hierarchy Process (AHP) to develop a model and assessment instrument for student development at undergraduate vocational universities. In particular, we focus on two issues: First, what levels, dimensions, and indicators should be included in the student development model? Second, what are the differences between the weights of student development indicators at different levels in the assessment instrument? This study explored the above issues to highlight the goal and direction and provide instrumental support for undergraduate vocational universities to cultivate high-quality technical and skilled talent.
2. Literature review
2.1 Undergraduate vocational education
Undergraduate vocational education belongs to the undergraduate level of higher education and is typologically subordinate to vocational education. It is characterized by higher education, vocational training, and local features [1]. It has the fundamental characteristics of higher education, while the vocational training aspect means that it focuses on enhancing students’ vocational techniques and abilities to make them more employable. Local features refer to its talent cultivation dovetailing with local industry development and enterprise production to boost local economies. Based on these features and previous research [1–3], in this study, undergraduate vocational education is defined as vocational education that cultivates high-quality technical and skilled talent by providing students with adequate professional theoretical knowledge, the ability to lead practical and technological applications and on-site handling, and transferable and innovative abilities that enable them to adapt to the needs of multiple industries.
Talent cultivation objectives are the provisions made by the cultivation unit for the educator’s cultivation direction, requirement specification, etc., based on educational goals and social needs [2]. In recent years, the literature on talent cultivation in undergraduate vocational education has focused on distinguishing its objectives from those of college vocational education and general undergraduate education, specifically in terms of “level” and “type” dimensions [4]. Undergraduate vocational education cultivates students with more profound technical and theoretical knowledge, more advanced practical skills, and a certain level of management, job conversion, technology transfer, and innovation abilities compared to colleges. Meanwhile, general undergraduate education cultivates academic talent with theoretical knowledge, professional ability, and moral quality among the three categories. In conclusion, academics have not reached a consensus regarding the details of talent cultivation in undergraduate vocational education.
2.2 Student development
College student development refers to the concept of human development within the context of higher education. Bowen (1977) argued that student development should encompass cognitive learning, practical abilities, and emotional and moral development [5]. Astin (1993) measured student development in terms of cognitive aspects (i.e., knowledge and ability) and emotional gains (i.e., students’ perceptions, attitudes, and values) [6]. According to Pang Bo (2012), student development in higher education involves five components: knowledge, ability, social relationships, quality, and personality [7]. He Xiangling regards student development as the holistic development of their knowledge, abilities, values, and interpersonal relationships in higher education [8]. This study defines student development as college students’ ongoing progress and improvement in their knowledge, abilities, and comprehensive qualities, representing the added value of college students’ four years of enrollment before graduation.
Scholars generally agree that assessing student development involves evaluating a university’s ability to cultivate certain qualities. Student development assessment is in a constant state of exploration worldwide. The United States was the first country to develop student development measurement instruments, including the famous College Student Experiences Questionnaire (CSEQ), developed in 1994, and the National Survey of Student Engagement (NSSE), published in 2000. In the CSEQ, student development covers general education, personal development, scientific and technological knowledge, and professional and practical abilities. The NSSE evaluates student development in terms of three dimensions: knowledge, ability, and value gains.
Student development assessment instruments in China are derived from the adaptation of traditional measurement instruments from other nations to the Chinese context. Representative instruments in China include the Ten Years of Student Development in Shanghai biennial research project (1998–2007), the State of Student Development in Beijing annual survey project (2006), and the Tracking Study of Learning and Development of Chinese University Students (CCSS, 2007). The CCSS is China’s most popular and longest-running assessment instrument for student learning engagement. The above instruments can be used to evaluate student development, learning engagement, and factors influencing student development [8]. However, they only apply to general undergraduate universities, vocational colleges, and Beijing’s prestigious universities.
2.3 FMD and AHP
In traditional Multiple Criteria Decision-Making, where rating criteria are accurately known, scholars often use the Delphi method to predict the results. The Delphi method anonymously solicits experts’ opinions in multiple rounds. They are repeatedly solicited, summarized, modified, and technically processed to obtain a consensus among experts and ultimately predict indicators.
However, criteria cannot be expressed under many conditions in terms of accurate data [9, 10]. Fuzzy theory has been proposed to cope with this limitation. Many scholars use the FDM to screen the indicators when researching evaluation indicator systems or models [11–14]. The FDM uses statistical analyses and fuzzy calculations to transform experts’ subjective opinions into quasi-objective data. It considers the uncertainty and fuzzy nature of experts’ subjective thinking for further screening indicators.
The AHP is often used to determine the importance of indicators [15–18]. It is a decision-making method combining qualitative and quantitative analyses by decomposing the problem into related factors and levels, such as objectives, criteria, and options. It is characterized by the hierarchical and quantitative nature of the decision-making process, which is based on the laws of thinking and psychology. It is widely used to solve complex decision-making problems to determine indicator weights.
Many researchers combine the FDM and AHP to enhance their findings (Table 1) [19–21]. This combination not only allows the expert’s opinion to be used as a decision-making parameter to screen indicators that express a particular item, but it also utilizes the expert’s two-by-two judgment of indicators to determine each indicator’s importance for the whole to determine a complete model that can express the item.
In summary, this study proposed a combination of the FDM and AHP to construct a student development model. The student development model concretizes student development goals by layering student development indicators according to student development theory. It can provide a realistic reference for undergraduate vocational universities to improve the matching of talent supply and demand.
2.4 Research gap
The literature review shows that academics and professionals alike are interested in undergraduate vocational education and student development. In the previous sections, this study features the following research gaps:
Section 2.1 highlights the lack of specific talent development objectives in undergraduate vocational education. Student development is the manifestation of talent cultivation. Consequently, student development model indicators also concretize talent cultivation objectives. It is essential to clarify the student development model indicators as undergraduate vocational education is responsible for cultivating talent for China’s high-quality economic and social development.
Section 2.2 identifies the research gap in student development assessment instruments for undergraduate vocational universities. Universities need instruments to assess the achievement of student development goals. They optimize student development programs based on assessment results to maximize the match between student development and social talent requirements.
Section 2.3 confirms the wide applicability of the FDM and AHP in model and indicator system research. To the best of our knowledge, no previous work has analyzed student development combining the FDM and AHP.
3. Methodology
This study proposed a four-stage methodology for developing a model and assessment instrument for student development in undergraduate vocational education, including preliminary construction, refinement, screening, and weight determination indicators. Fig 1 shows the detailed research framework for this study, and the following sections explain each stage. Prior to the data collection, this study obtained a written approval from the Universiti Tunku Abdul Rahman Scientific and Ethical Review Committee to support collecting research data. Written informed consent was obtained from all participants for inclusion in the study.
3.1 Stage 1: Preliminary construction of the student development model
The researcher’s initial model indicators of student development in undergraduate vocational education were based on China’s actual talent demand, the literature, and vocational education policies. Six experts (from the vocational education department, undergraduate vocational universities, career development planning, corporate human resources, technical executives, and student development studies) who were sufficiently familiar with the topic of this study and had worked in the field for ten years or more, were invited to participate in the indicator construction process. They discussed indicators and constructed a preliminary five-level student development model with the two elements of cognition and non-cognition, the four dimensions of knowledge, ability, value, and quality, eight factors, and 33 program indicators (Table 2).
3.2 Stage 2: Refining of student development model
This study conducted a first round of the expert survey to refine indicators and improve their reliability and comprehensiveness before screening the indicators. The first author distributed the indicator refinement questionnaire to 22 experts (the same experts as those surveyed for the FDM and AHP and not duplicated by the six experts) from January 4 to 28, 2022, by whom 20 surveys were returned. They provided their opinions on the modifying, adding, and merging indicators, explaining the reasons (Table 3). Then, the expert panel of six experts was invited to discuss further and analyze the experts’ advice to refine the student development indicators. The results are shown in Table 4.
3.3 Stage 3: Screening indicators using the FDM
The first author conducted a second round of expert surveys between February 15 and 29, 2022, using the FDM to screen the student development model’s key indicators. The Delphi method was first proposed by Dalkey and Helmer in 1963 to obtain group decisions by consulting experts [40]. However, in many situations, the judgment of experts cannot be quantified precisely. Murray, Pipino, and Gigch proposed the Fuzzy Delphi method (FDM) in 1985 to overcome the limitations of the traditional Delphi method using fuzzy theory. Compared with the conventional Delphi method, the FDM has the following advantages: 1. Fewer surveys; 2. Each expert’s opinions are considered and integrated; 3. Fuzzy theory considers the expert’s uncertain and subjective opinions; 4. Reduced time and cost required for surveys [41].
FDM screening indicators generally include the following steps: establishing the model hierarchy and designing the expert questionnaire, collecting experts’ decision-making opinions, checking the degree of consensus, and applying the threshold value to screen the indicators [42]. This study analyzed experts’ opinions involving the double-triangular fuzzy number [20] and checked for a consensus using the gray area [43]. The four steps of the FDM to screen indicators in this study are:
Step 1: Establish a hierarchy and design a questionnaire.
The five-level hierarchy of the model indicators is shown in Table 4. The expert survey consists of two values: (1) the indicator’s “single value” indicates the quantitative value of its importance, and (2) the interval value for each indicator. The minimum of this interval value indicates the expert’s “conservatively perceived value (C)” for the quantitative score of the indicator, and the maximum means the expert’s “optimistically perceived value (O)” for it. The above values are all different integers ranging from 1 to 9, with larger values indicating greater importance [44].
Step 2: Establish double triangular fuzzy numbers.
The minimum , maximum
and geometric mean
of the “single value,” the minimum
, maximum
and geometric mean
of the “conservatively perceived value” and the minimum
, maximum
and geometric mean
of the “optimistically perceived value” given by each expert for each indicator i are calculated, and the corresponding triangular fuzzy number of “conservatively perceived value”
and that of the “optimistically perceived value”
for each indicator i are established [45], as shown in Fig 2.
Step 3: Check the degree of consensus reached by experts.
This step is to check how the two triangular fuzzy numbers overlap to determine if the experts can reach a consensus [43].
- Check if there is a gray zone. When
, it means that the two triangular fuzzy numbers do not overlap, and no gray zone exists, which indicates that experts’ opinions do not converge. Then conduct the next round of surveys, attaching the statistics of this round. When
, it indicates that they overlap and a gray area exists. Then calculate the check value for the gray zone to verify whether the experts reach a consensus.
- Calculate the check value for the gray zone. To assess if the experts’ opinions have converged, the fuzzy gray zone
is compared to the range
of the geometric mean of the “conservatively perceived value” and “optimistically perceived value.” If Mi−Zi > 0, it means that the experts reached a consensus. If Mi−Zi < 0, it indicates there is a gray area with no expert consensus. Therefore, it is necessary to consult the statistics on the “conservatively perceived value” and “optimistically perceived value” of indicators that do not converge to experts for reference, and another survey has to be carried out, repeating steps 1 to 3 until all the indicators reach consensus.
Step 4: Calculate the threshold value of the screening indicators.
When the importance of each indicator reaches convergence, the geometric mean of the “single value” and the “conservatively and optimistically perceived value” for each indicator is geometrically averaged to obtain its consensus value, Gi. The larger a Gi, the higher the consensus and importance of the indicator. The geometric mean of the “single value,” the “conservatively perceived value,” and the “optimistically perceived value” for all indicators is then geometrically averaged to determine the indicator screening threshold Ti. When the Gi<Ti, the indicator is deleted. If not, it is retained.
3.4 Stage 4: Determining weight by applying the AHP
The weight reflects an indicator’s importance in the mode, and its reasonableness directly affects the scientific robustness of student development assessment. The first author used the Analytical Hierarchy Process (AHP) to assign indicator weight in a third round of expert surveys from April 6 to 27, 2022. Saaty proposed the AHP in the mid-1970s, which is a multi-objective decision-making method that combines qualitative and quantitative analyses [47]. The Saaty scale is adopted, and the pairwise comparisons [48] are used to assign the student development weights. The AHP involves four steps:
Step 1: Establish a hierarchy.
The hierarchy structure divides a complex problem into indicators grouped by attributes to form different levels. The upper-level indicators govern the same-level indicators and guide some of the lower-level indicators. The structure divides decision-making objectives, factors, and objects into high, middle, and low levels based on their relationships.
Step 2: Create the pairwise comparison matrix.
The weight questionnaire is designed based on the indicator hierarchy. Experts make pairwise comparisons of same-level indicators, and the rating scale uses Saaty’s relative importance scale ranging from 1 to 9 [49]. The higher the number, the greater the importance of one indicator relative to another. The comparison results are organized to form a pairwise comparison matrix to obtain the eigenvectors and eigenvalues.
Step 3: Carry out a consistency test.
The judgment of expert consistency is based on the consistency ratio (CR), calculated by dividing the consistency indicator for the judgment matrix by the corresponding random matrix (RI). Saaty (1980) suggested that CI≤0.1 indicates that the pairwise comparison matrix is consistent. When CI>0.1, it indicates inconsistency. When CR<0.1, it indicates that its consistency is satisfactory. If CR≥0.1, the consistency test is not passed, and another round of expert surveys is carried out until it is consistent. The test formula is as follows:
(1)
(2)
Step4: Calculate indicator weight.
The eigenvector method is used to calculate the maximum eigenroot vector of each pairwise comparison matrix, which is normalized to obtain the weight of each indicator. Then, the local weight is calculated by averaging each indicator weight of all experts.
3.5 Selection of survey experts
The survey experts had to be familiar with and cooperate with this study. Zhong Zhengwei et al. believe a group of experts consisting of more than ten but less than 30 members makes the fewest group attribution errors and establishes the highest credibility [41]. Twenty-two experts surveyed in this study met the following two criteria.
Criterion 1: Closely related fields of this study. Selecting 22 experts based on our research characteristics requires both authoritative academic experts in the field and experts from the front line, in this case, including two student development research experts, six undergraduate vocational education talent cultivation research experts, two experts in career planning for college students, two vocational education policy development experts, four enterprise human resources experts, and four technical experts. To be chosen as an expert in this study, individuals had to have at least an associate senior title and ten years or more of experience in the field. Specific information on the experts in each area is shown in Table 5.
Criterion 2: Expert authority coefficient (Cr). The expert authority coefficient (Cr) is the arithmetic mean of the judgment coefficient (Ca) and familiarity coefficient (Cs), namely, indicates acceptable reliability. Ca represents the expert judgment basis [50]. In this study, experts used terms such as “practical experience,” “logical reasoning,” “domestic and international knowledge,” and “intuition” as judgment criteria (Table 6). Cs represents an expert’s familiarity with the problem, and its value ranges from 0 to 1 (1 = very familiar, 0.75 = familiar, 0.5 = relatively familiar, 0.25 = generally familiar, 0 = not very familiar). The closer Cr is to 1, the greater an expert’s authority and the higher the reliability of the survey results.
4. Data analysis and results
4.1 Analyzing experts’ authority coefficients
The authority coefficients of 20 experts who responded positively in the first round are between 0.7 and 1.0, with a mean of 0.91, which indicates that these experts are authoritative and that the survey results are reliable. The second and third rounds of expert surveys only surveyed the 20 experts who had responded positively in the first round.
4.2 FDM analysis results
The expert panel decided to conduct a second round of expert surveys for the 5th-level indicators since the 2nd- to 4th-leve indicators had a high degree of expert consensus and a well-established theoretical basis. The survey was carried out according to the operational steps of the FDM, and Microsoft Excel was used to analyze the survey data. The results are shown in Table 7. All of the indicators’ gray zone test values Zi are greater than 0, indicating the presence of gray areas. Their Mi−Zi > 0 means that experts’ opinions tend to be consistent, and the indicator is convergent.
The expert consensus value Gi is an essential factor in screening indicators. The higher it is, the greater its importance. As for the screening indicators, they are determined by the decision threshold Ti. The threshold will directly affect the number of screening indicators. Yao Kaichao (2022) believed that the threshold could be set to 6, excluding indicators with Gi below 6 [46]. Chen Wenliang (2021) took the arithmetic mean of all indicators Gi as the threshold and deleted indicators with a Gi value below the mean [51]. This study used the arithmetic mean of all indicators, 7.54, as the decision threshold (above 6) to reduce human intervention. As shown in Table 7, E1, E33, and E34 were deleted since their Gi values were below 7.54. The other 33 indicators were retained (Table 8), with a consensus value of 7.6 or higher, whereas the three deleted indicators had a value of 4.5 or less. The results indicates that academics, industry, and government departments have reached a high degree of consensus on the student development indicators, thus providing a convincing basis for constructing a student development model.
4.3 AHP analysis results
This study established a five-level student development model (Table 8). Researchers believe that assigning weights to too many levels and indicators makes pairwise comparisons complicated and laborious, requiring considerable patience by experts. Researchers, therefore, argued that methods or procedures should be optimized or simplified [46]. The expert panel made each 5th-level indicator equally crucial to the 4th-level indicator. Therefore, this study surveyed the weights of 2nd- to 4th-level indicators.
Pairwise comparison matrices were created based on the third round of expert survey data. They all passed the consistency test because they met the condition of CR<0.1. The pairwise comparison matrices created by Expert 1 are shown in Table 9.
Local weights, or relative weights, were obtained based on the pairwise comparison matrices. It is each indicator’s importance relative to its upper-level indicator. Based on the considerations of easy judgment and simple calculation, this study used the arithmetic mean of experts’ weights to obtain the local weight of the indicator and then calculated the global weight, also referred to as the absolute weight, which is the indicator’s weight on the same level. The results are shown in Table 10. The five-point Likert scale score was assigned to each 5th-level indicator (particularly much improvement = 5, a lot of improvement = 4, general improvement = 3, slight improvement = 2, almost no improvement = 1) to develop the student development assessment instrument.
The results of this study showed that the two 2nd-level indicators, cognitive and non-cognitive, are equally significant to student development. The three 3rd-level indicators are ranked in importance in terms of quality, ability, and knowledge development, indicating that experts pay more attention to students’ quality and ability development. In cognitive development, ability development is significantly more important than knowledge development. This result is consistent with socialist education with Chinese characteristics, prioritizing moral education, ability, and comprehensive development. Among the eight 4th-level indicators, the top five are value (D6), career development ability (D5), personal quality (D7), vocational quality (D8), and vocational ability (D4), which indicates that society places the greatest emphasis on the students’ value development, followed by the ability and quality requirements associated with the characteristics of undergraduate vocational education. The indicator importance ranking is generally consistent with that of the FDM screening indicators, thus verifying the validity of this study’s student development model.
5. Discussion and recommendations
Based on survey data from 20 experts in six fields, this study used the FDM and AHP to design a reliable student development model and assessment instrument that fills research gaps. The proposed model is comprehensive, pertinent, systematic, and scientific, as shown by its analysis.
Regarding comprehensiveness, the model depicted student development elements from five levels, three dimensions, and eight factors, clarified its constituents and ability observation points, and comprehensively and fully reflected the essential attributes of student development in undergraduate vocational education. In addition, the researcher and experts constructed the student development model from the perspective of the social and economic development requirements for high-quality technical and skilled talent and individual student development. This study followed the student development theory and divided it into cognitive and non-cognitive development, which have been well-respected among researchers. Other researchers focused more on students’ learning engagement or the influence of various external factors on student development [8, 35, 36]. Their indicators are more macroscopic and fewer in number, while the model presented in this study is rich in levels and comprehensive in indicators.
Pertinence is mainly manifested in the following ways. First, other researchers’ student development models include basic knowledge and abilities indicators that are somewhat universal [4, 8, 35, 36]. Comparatively, this study focused on the outstanding vocational and practical characteristics of undergraduate vocational education and used vocational quality, vocational ability, and career development ability as the 4th-level indicators. The 5th-level indicators included emergency handling ability, post-operation and problem-solving ability, transfer ability, conversion ability, and craftsmanship, which are essential abilities and qualities for students in the production line job groups. However, student development indicators of vocational colleges emphasize specific techniques and skills needed in a certain position or craftsmanship, and require very little theoretical knowledge, innovation, and developmental abilities. Indicators of general undergraduate universities focus on systematic and complete theoretical knowledge, strong dialectical thinking, and weak practical abilities. Student development indicators and their weights will vary across the different categories of universities due to the above differences. Second, it is worth noting that the model presented in our study is localized to be applied in evaluating student development in China and may not be appropriate for evaluating student development in other countries. Chinese culture is an ethical culture that places human development within the context of interpersonal relationships and values human virtues and cooperation. In comparison, Western culture emphasizes individual growth and is an individualist culture [52]. Such cultural differences inevitably lead to differences in student development indicators in different countries, and even the same indicator may be weighted inconsistently across countries.
For example, undergraduate vocational education in Germany and the United States starts earlier, so students already possessed post-practice and problem-solving abilities. They now emphasize cultivating students’ innovative spirit and creative ability [53]. However, China’s undergraduate vocational education has just begun emphasizing students’ basic vocational and career development abilities. Third, experts’ weights are consistent with China’s educational standard of “morality, ability, and knowledge” [54], prioritizing national conditions and fully embodying socialist universities with Chinese characteristics.
As far as systematicity is concerned, it is reflected in the indicator design, which considers the connections between indicators and levels. Each indicator performs its functions and roles and is intertwined with others to constitute an organic whole. Previous research only divided model dimensions but neglected their level of performance. This study examined the model indicators holistically, focusing on the systematic composition of student development and subdividing them into levels to show the entire pattern of student development in an all-round and multi-level way. The division into levels for each dimension corresponds to the student development stage and is well differentiated. For instance, ability development is divided into general, vocational, and career development ability in ascending order of difficulty, and quality development is subdivided into value, personal quality, and vocational quality from high to low, according to importance. The indicators were assigned weights according to their importance, and the model indicators were designed from a global perspective. The application of the FDM and AHP standardized the screening process and the assignment of weights to the indicators and allowed for the construction of the student development model indicators with systematic characteristics.
From the scientific perspective, the authority and consensus of the experts included in our study are high, and the results are reliable. This is also reflected in the indicator selection and design method. Regarding indicator selection, this study added two steps based on the literature before conducting the Delphi expert survey. First, the expert panel, consisting of six experts, was invited to discuss and analyze the indicators. Second, the preliminary indicators were determined after the indicators were refined through the expert survey, which included 20 experts, so repeated discussions and surveys improved the scientific rigor of the indicators. In terms of the indicator design method, the FDM overcame human subjective ambiguity in indicator analysis. The AHP was used to determine indicator weights by considering the model’s hierarchical nature and the interrelationships between levels. These two methods reduced indicator selection subjectivity and arbitrariness while also avoiding the problems of the simplification and superficiality of indicators caused by the singular use of qualitative or quantitative methods. The experts unanimously agreed that this study’s model and assessment instrument could be widely used in undergraduate vocational university student development.
Despite its contributions, this study also has some limitations. First, the experts have a high degree of authority and consistency in data analysis, and the designed indicators have a certain scientific nature. Nevertheless, our study is part of ongoing academic research, and the constructed assessment index system has not yet been empirically verified. Second, the subjects of this study are experts from different fields, and the number of experts per field was uneven. Their positions and opinions may also differ, although they all belonged to the decision-making group.
6. Conclusion
This student development model constructed in this study is China’s first set of indicators of an “undergraduate vocational university student development model.” Before this study, evidence of undergraduate vocational education student development was merely anecdotal. The academic rigor and transparency of our study process make a novel contribution to the vocational education and student development literature and thus can help other countries consider ways to transfer the findings to their context. Furthermore, our study’s five-level model is essential and actionable for undergraduate vocational education student development practice in China. Education authorities can use the model to assess the quality of talent cultivation at undergraduate vocational universities. Universities can use this instrument to assess student development, diagnose related problems, and collect empirical data to improve it. They can also optimize the allocation of educational and teaching resources based on the indicator weights to quickly cultivate the urgently needed talent and promote the high-quality development of China’s economy and society. It is expected that the government, authorities, and universities will make a coordinated effort to pilot the model in some universities, further improve it based on practice, and then establish a national standard for all undergraduate vocational universities as soon as possible.
This study marks the beginning of undergraduate vocational university student development research, and there is still a considerable need for further research. First, future researchers can use our proposed model to track student development data and assess student groups’ development levels and characteristics in different grades, disciplines, and universities to provide empirical data for optimizing the model and improving student development. Second, this study was carried out by Chinese experts and undergraduate vocational universities, and future research can apply the proposed research methods to other countries and universities by conducting similar surveys and comparing the survey results to learn from each other. Third, researchers can promote students’ outstanding development in terms of the critical indicators by conducting in-depth studies on the specific implementation of each key indicator. Fourth, in future research, the indicators can be further revised according to the actual situation of the application of the indicators to maximize their adaptation to actual needs and achieve the highest possible objectivity and accuracy. Fifth, future research can consider the number of survey experts in different fields and weigh their relative importance to make the survey results more scientific.
Supporting information
S1 Dataset. Data used in this study.
S1A, S1D and S1G are the first, second and third rounds of expert questionnaires respectively. S1C, S1E and S1H are the raw data of the first, second and third rounds of expert surveys respectively. S1B is the authoritative level data for the first round of expert surveys. S1F is the statistical analysis data of screening indicators in the second round of expert surveys. S1I is the statistical analysis data of indicator weight in the third round of expert surveys.
https://doi.org/10.1371/journal.pone.0301017.s001
(ZIP)
Acknowledgments
We are grateful to the authoritative experts in various fields for providing valuable data.
References
- 1. Meng FY, Xu H. A review of research on undergraduate vocational education. Explorations in Higher Vocational Education. 2020; 19 (06): 9–15.
- 2. Cui SQ, Yao CL, Zhang L. Objectives, followings, and dimensions evaluating the teaching of Civics in higher vocational university programs. China Vocational and Technical Education. 2023; (20): 72–77.
- 3. Yang LJ, He XL. College student development index: structure and level-analysis based on 2016 CCSS survey data. China Higher Education Research. 2018(12): 46–52.
- 4. Liu ZZ, Xiao XT, Yao D. A study on the dilemma of cultivating talents for undergraduate vocational education. Journal of Tonghua Normal University. 2023; 44(05):121–127.
- 5. Bowen HR. Investment in leaming. San Francisco: Jossey-Bass; 1977.
- 6.
Astin AW. Assessment for excellence: the philosophy and practice of assessment and evaluation in higher education. New York: Oryx Press; 1993.
- 7.
Pang B. Research on college student development risks. Nanjing: Nanjing University Press; 2017.
- 8.
He XL. Research on the level, change trajectory, and influencing factors of college student development index [master’s thesis]. Nan Jing (CN): Nanjing University of Posts and Telecommunications; 2019.
- 9. Kannan D, de Sousa Jabbour ABL, Jabbour CJC. Selecting green suppliers based on GSCM practices: Using fuzzy TOPSIS applied to a Brazilian electronics company. European Journal of Operational Research. 2014; 233(2): 432–447.
- 10. Ishizaka A, Siraj S. Are multi-criteria decision-making tools useful? An experimental comparative study of three methods. European Journal of Operational Research. 2018; 264(2): 462–471.
- 11. Etu E-E, Monplaisir L, Aguwa C, Arslanturk S, Masoud S, Markevych I, et al. Identifying indicators influencing emergency department performance during a medical surge: a consensus-based modified fuzzy Delphi approach. Plos one. 2022; 17(4): e0265101. pmid:35446857
- 12. Ocampo L, Ebisa JA, Ombe J, Escoto MG. Sustainable ecotourism indicators with Fuzzy Delphi Method–A Philippine perspective. Ecological indicators. 2018; 93: 874–888.
- 13. Padilla-Rivera A, do Carmo BBT, Arcese G, Merveille N. Social circular economy indicators: Selection through Fuzzy Delphi Method. Sustainable Production Consumption. 2021; 26: 101–110.
- 14. Sadeghi J, Oghabi M, Sarvari H, Sabeti M-S, Kashefi H, Chan D. Identification and prioritization of seismic risks in urban worn-out textures using Fuzzy Delphi Method. Environmental Engineering Management Journal. 2021; 20(6): 1035–1046.
- 15. Liu ZG, Ning L. Research on the evaluation indicator system of enterprise practice exercise for teachers in vocational colleges based on AHP. China Vocational and Technical Education. 2020; (17): 62–69.
- 16. Panchal S, Shrivastava AK. Landslide hazard assessment using analytic hierarchy process (AHP): A case study of National Highway 5 in India. Ain Shams Engineering Journal. 2022; 13(3): 101626.
- 17. Shen G, Wang W-L. Circlize package in R and Analytic Hierarchy Process (AHP): Contribution values of ABCDE and AGL6 genes in the context of floral organ development. Plos one. 2022; 17(1): e0261232. pmid:35061694
- 18. Tempa K. District flood vulnerability assessment using analytic hierarchy process (AHP) with historical flood events in Bhutan. Plos one. 2022; 17(6): e0270467. pmid:35749469
- 19. Chen C-W, Wang J-H, Wang JC, Shen Z-H. Developing indicators for sustainable campuses in Taiwan using fuzzy Delphi method and analytic hierarchy process. Journal of Cleaner Production. 2018;193:661–671.
- 20. Karam A, Hussein M, Reinau KH. Analysis of the barriers to implementing horizontal collaborative transport using a hybrid fuzzy Delphi-AHP approach. Journal of Cleaner Production. 2021; 321: 128943.
- 21. Li Q, Dong MM. Research on the remaining service life prediction model of existing residential buildings based on FDM-AHP. Journal of Safety and Environment. 2020; 20(03): 1132–1138.
- 22. Song KH, Tian SH, Peng QW. Knowledge, ability, and quality structure of applied talents and their cultivation. Higher Education Research. 2012; 33(07): 94–98.
- 23.
Ma Y. Research on the development of undergraduate vocational education in China [PHD thesis]. Tian Jing (CN): Tianjin University; 2015.
- 24. Guo GJ, Liu LM, Long W, Chen CJ, Fang JC, You XE. Research on the positioning and cultivation system of key abilities of students in vocational colleges and universities in China under the new situation. China Vocational and Technical Education. 2017(05): 22–28.
- 25. Li YZ, Chang J. Positioning, specification and system construction of high-quality applied talent cultivating. China Vocational and Technical Education. 2019(01): 45–49.
- 26. Zhao ZQ. An analytical study of the characteristics and logic of talent cultivating in undergraduate vocational education. The Guide of Science & Education. 2023(20): 4–6.
- 27.
Liu L. Research on the positioning of talent cultivation objectives of undergraduate higher vocational education [master’s thesis]. Hu Nan (CN): Hunan Normal University; 2016.
- 28. Fang ZQ. Undergraduate vocational education: Concept, development dynamics, and reform breakthroughs. Vocational and Technical Education. 2019; 40(13): 18–23.
- 29. Chen Q. Pilot study on undergraduate vocational education in China in the new era. Education and Careers. 2020(07): 19–25.
- 30. Wen BY, Yin HY. A study on talent cultivation in undergraduate vocational education: A case study of 14 undergraduate vocational education institutions in Jiangxi Province. Vocational Education Forum. 2020(04): 131–137.
- 31. Hu YH. Research on the specification requirements of talent cultivation for undergraduate vocational education in China. Vocational Education Form. 2020(04): 154–160.
- 32. Wang J, Huang LQ. The desirable direction, target characteristics, and path of talent cultivation in China’s undergraduate vocational education. Education and Careers. 2020(16): 20–26.
- 33. Astin AW. Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. Phoenix, AZ. Oryx. 1993.
- 34.
Chickering AW, Reisser L. Education and Identity. The Jossey-Bass Higher and Adult Education Series. ERIC; 1993.
- 35.
Zhao XY. Research on student development and its influencing factors in higher education based on student engagement theory [PHD thesis]. Tian Jin (CN): Tianjin University; 2013.
- 36. Guo LB. A study of college student development based on learning gains [master’s thesis]. Bei Jing (CN): Beijing University of Technology; 2014.
- 37.
Wang WJ. Analysis of the impact of extracurricular learning input on student development [master’s thesis]. Guang Xi (CN): Guangxi University; 2014.
- 38.
Zhuo N. Research on the problems of diagnosing and improving student development in vocational education colleges and universities [master’s thesis]. Hu Bei (CN): South-central University For Nationalities; 2019.
- 39. Yang LJ, He XL. College student development index: Concepts, architecture, and critical technologies. Higher Education Exploration. 2018(02): 109–114.
- 40. Dalkey N, Helmer O. An experimental application of the Delphi method to the use of experts. Management Science. 1963; 9(3): 458–467.
- 41. Chung C, Xu Y, Chen Y, Hsin J. Study on the construction of development indicators of recreational fisheries by the fuzzy Delphi method. Kaohsiung Norm Univ J Educ Soc Sci. 2017; 43: 85–98.
- 42. Hsu H, Wey W, Tsai P. The applications of Analytic Network Process to the priorities of interdependent housing projects selection. Archit. 2007; 62: 49–74.
- 43. Wang WM, Peng H-H. A fuzzy multi-criteria evaluation framework for urban sustainable development. Mathematics. 2020; 8(3): 330.
- 44. Naveed QN, Qureshi MRN, Tairan N, Mohammad A, Shaikh A, Alsayed AO, et al. Evaluating critical success factors in implementing E-learning system using multi-criteria decision-making. Plos one. 2020; 15(5): e0231465. pmid:32365123
- 45. Tseng Y-P, Huang Y-C, Li M-S, Jiang Y-Z. Selecting key resilience indicators for Indigenous community using the Fuzzy Delphi Method. Sustainability. 2022; 14(4): 2018.
- 46. Yao KC, Lai JY, Huang WT, Tu JC. Utilize Fuzzy Delphi and Analytic Network Process to construct consumer product design evaluation Indicators. Mathematics. 2022; 10(3): 397.
- 47. Saaty TL. Decision making with the analytic hierarchy process. International journal of services sciences. 2008; 1(1): 83–98.
- 48. Rodrigues D, Godina R, da Cruz PE. Key performance indicators selection through an analytic network process model for tooling and die industry. Sustainability. 2021; 13(24): 13777.
- 49. Wei W, Chang W. A study on selecting optimal product design solution using fuzzy Delphi method and analytic. J Des. 2007; 10: 59–76.
- 50. Chen ZY, Zeng Y, Tian FY. Developing a third-degree burn model of rats using the Delphi method. Scientific Reports. 2022; 12(1): 13852. pmid:36056098
- 51. Cheng WL. Integrating FDM and ISM to analyze the key design indicators of children furniture. J SHU-TE Univ. 2020; 22: 49–66.
- 52. Gong JP. A study of the manifestation of socio-cultural differences between Chinese and Western societies in cross-cultural communication. International Public Relations. 2023(01): 82–94.
- 53.
Xu XY. Research on talent cultivation objectives of vocational undergraduate education in China [master’s thesis]. Ji Lin (CN): Jilin HuaQiao University of Foreign Languages; 2023.
- 54. Su RP. Organic systematic of ideological and political education for female college students. Journal of China Women’s University. 2012; 24(02): 47–52.