https://orcid.org/0000-0002-5247-0173
Figures
Abstract
Fuzzy cognitive maps (FCMs) are a powerful tool for system modeling, which can be used for static and dynamic analysis. However, traditional FCMs are usually learned by gradient-based methods, and the adopted sigmoid nonlinear activation function frequently causes gradient saturation. These two shortcomings set a limit on the modeling accuracy. To overcome those problems, we propose in this paper a new FCM with two improvements. First, the rectified linear unit (ReLu) activation function is adopted to replace the sigmoid function. Second, a newly proposed quasi-oppositional bare bone imperialist competition algorithm (QBBICA) is used to learn the FCM. The improved FCM is used to predict the employment rate of graduates from Liren College, Yanshan University. Experimental results show that the improved FCM is effective in employment rate prediction.
Citation: Guo A (2022) ReLU-FCM trained by quasi-oppositional bare bone imperialist competition algorithm for predicting employment rate. PLoS ONE 17(8): e0272624. https://doi.org/10.1371/journal.pone.0272624
Editor: Lixiang Li, Beijing University of Posts and Telecommunications, CHINA
Received: January 28, 2022; Accepted: June 5, 2022; Published: August 18, 2022
Copyright: © 2022 Aihua Guo. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
Improving the employment rate and employment quality of university graduates has been an important task for China’s education authority and universities. Based on the employment status of graduates, the education authority can adjust relevant policies and the university’s faculty can adjust their education and training plans to enhance the graduates’ employability. To guarantee sufficient employment of graduates, the responsible decision-makers need to make adjustments ahead of time. Adequate supporting data is necessary to ensure that the decisions are made in a scientific manner. Hence, accurately predicting the employment rate of graduates becomes an important topic of research.
Since the employment rate of graduates is affected by many factors, accurate prediction of graduates’ employment rate remains a difficult task. In the past years, many researchers have paid much attention to the prediction of employment rate of graduates. In [1], Wang adopted a neural network with an adaptive learning back-propagation framework to construct an intelligent employment rate prediction model. In [2], Guo et al. developed a framework called Multi-major Employment Status to predict graduates’ employment status when bias exists. In the framework, a long short term memory (LSTM) with dropout was adopted as the predictor. In [3], an employment rate prediction model was constructed using the back propagation (BP) neural network based on employment rate data from 2002 to 2012. Usually, the BP neural network is trained using the gradient descent algorithm, which can be easily trapped in a local optimum. To overcome this drawback, Jiang et al. [4] proposed to use an artificial fish swarm algorithm (AFSA) to train a BP neural network for predicting university graduates’ employment rate. Zhang proposed to predict university graduates’ employment rate using regression analysis, and they constructed two regression models, i.e., the polynomial regression model and the multiple linear regression model [5]. To improve the prediction accuracy, a model that combines the B-mode relational degree and general induced order weighted averaging aggregation (GIOWA) operator was proposed in [6]. A decision tree model for employment prediction was presented in [7].
Despite much research progress in the prediction of graduates’ employment rate, there are still improvements to be made in the prediction accuracy. The reason for this problem is twofold. First, the employment rate data of a single university is not large enough. That is to say, one can not obtain enough samples to construct an artificial neural networks (ANN) based prediction model. As a consequence, the learned ANN model is under or over fit and the prediction accuracy decrease. Second, factors affecting the employment rate of graduates and the causality between these factors and the employment rate of graduates are still unknown. In other words, the employment rate data contains much uncertainties. The above mentioned neural networks and gray model based prediction methods are still weaker in handling uncertainty. Therefore, it is necessary to develop an alternative model that can handle different uncertainties and perform prediction with limited data.
Fuzzy cognitive maps (FCMs), which was first introduced by Kosko [8], are a powerful tool for modeling and simulating complex dynamic systems. In nature, an FCM is a directed graph consisting of a set of nodes and edges. The nodes stand for concepts, variables or factors in a system, and they are connected with each other through edges. The weights of the edges represent the degree of the interaction between these nodes. FCMs have several appealing characteristics in modeling the complex relationship between variables. One is that FCM is flexible and easy to be constructed. Second, FCM has powerful ability to handle uncertainties. Finally, FCM can be sufficiently trained with small samples. With these advantages, FCMs have been widely used to solve different scientific and engineering problems, for example, in [9], an enhanced FCM called E-FCMs is proposed to model the influence strength between brain activity associated with different regions of the brain. In [10], the FCM is applied to gene regulatory network reconstruction, and a preference-based evolutionary bi-objective algorithm was developed to lean the FCM. In [11], the FCM was used to predict the DNA-Binding residues in local segments of protein sequences. In [12], FCM was used to model the discontinuous behavior in a complex system. In [13], a higher-order FCM combined with sparse auto-encoder was proposed for time series predicting and a batch gradient descent method is adopted to learn the FCM. In [14], a fast and effective method based on constrained convex optimization was developed for FCM learning. In [15], FCM was used to control wheeled mobile robot.
Although FCMs have found successful applications in many fields, they still suffer from several drawbacks. First, FCMs adopt the sigmoid function as the nonlinear transfer function. Recent deep learning researches have revealed that the sigmoid function can lead to poor performance due to its saturation near 0 and 1, which limits its ability to learn nonlinear mappings. Second, gradient-based methods are traditionally used as the learning algorithms for FCMs. Gradient-based methods are sensitive to noise and can easily be trapped in a local optimum. Recently, several population-based methods have been proposed to learn FCMs due to their global searching characteristics. In [16], the real-coded genetic algorithm (RCGA) was used to learn FCMs, where the weights of FCMs to be learned were encoded as an individual of GA. Compared to GA, particle swarm optimization (PSO) has simpler evolutionary operations. Therefore, in [17], PSO was adopted to learn FCMs. The differential evolution (DE) [18], the asexual reproduction optimization (ARO) algorithm [19], and the imperialist competition algorithm (ICA) [20] have also been adopted to learn FCMs.
ICA is an effective evolutionary algorithm first proposed by Atashpaz-Gargari and his colleagues [21]. It shows great potential in solving different complex optimization problems, such as flexible job shop scheduling [22], neural network training [23], parameter optimal design of heat exchangers [24], controller parameter tuning [25], and node placement in wireless sensor networks [26]. However, like many other evolutionary algorithms, ICA is also plagued by the premature problem. That is to say, if ICA finds a local optimal solution and fails to jump out of it, ICA will converge to the local optimum, which is undesired. To enhance the search ability of ICA, several ICA variants have been proposed. These variants can be roughly classified into three classes. The first class is to change the assimilation strategy. For example, in [27], an ICA variant called AR-ICA was proposed, where the assimilation coefficients are dynamically adjusted based on the concept of attraction and repulsion to better balance the diversity and convergence of the algorithm. In [28, 29], a modified ICA was proposed, in which the colonies move to its imperialist using a velocity-position model like PSO algorithm. In [30], a bare-bone ICA (BB-ICA) was given. In BB-ICA, the new position of each colony is determined according to a Gaussian sampling. In [31], an ICA variant called Gbest-ICA was proposed via taking the difference between the colony’s current position and the global best position into account when colony moves towards its imperialist. An orthogonal ICA (O-ICA) was proposed in [32], where a new move direction orthogonal to the original move direction is added to the colonies’ movement formula. The other class of ICA variants is to modify the original ICA’s competition and revolution step. In [33], the chaotic map was used to generate the revolted colonies, while in [34], the GA mutation operator is used to generate new revolted colonies to replace some original colonies. In [35], a clone evolution operator is introduced into the competition phase. Apart from the above two classes of variants, some researchers hybrid ICA with other evolutionary algorithms to improve the performance of ICA. For instance, in [36], the hybrids of ICA and simplex method was proposed. In the hybrid algorithm, the simplex method is responsible for a local searching. In [37], the ICA is hybridized with simulating annealing (SA), in which the SA is used to improve the position of imperialist after assimilation step. In [38], ICA is mixed with pattern search algorithm, the results of ICA severing as initial position are further improved using pattern search.
With the above facts in mind, we develop in this paper an improved FCM in which two improvements are made. First, a ReLU function is adopted to replace the original sigmoid function to enhance the nonlinear mapping ability. Second, an improved ICA, which integrates the quasi-opposition learning and bare bone ICA, is proposed to learn FCMs’ weights. The proposed method is used to predict the employment rate of university graduates.
The rest of this paper is arranged as follows. Some basics about FCMs are provided in Section 2. The bare-bone imperialist competitive algorithm is detailed in Section 3. The proposed method for predicting the employment rate of graduates is presented in Section 4. The prediction results are shown in Section 4.4. Finally, the concluding remarks are given in Section 5.2.
2 ReLU-FCM
FCMs, as shown in Fig 1, can be viewed as a fuzzy version of Axelrod’s cognitive maps. The causality between the nodes in an FCM can be a fuzzy degree. Like cognitive maps, an FCM consists of a set of nodes and weighted edges. The nodes represent the concepts or variables of a system while the weighted edges denote the causality between these concepts or variables. Suppose there are n nodes in an FCM, denoted as {C1, C2, ⋯, Cn}. The weight of the edge from node Cj to Ci is denoted by wji, where i, j = 1, 2, ⋯, n. The value of wji lies in [−1, 1]. The value of weight wji explains the degree of the causality between the nodes, and the sign of weight wji denotes the influential direction of the connected nodes. Specifically, if wji > 0, it means that an increase of node Cj results in an increase of node Ci with strength wji and vice versa. If wji < 0, it means an increase of node Cj results in a decrease of node Ci with strength |wji| and vice versa. If wji = 0 means there is no relation between node Cj and node Ci, the weights of all edges in an FCM can be rewritten into a matrix form as follows:
(1)
In the FCM, each node at time t has a value, called activation or state value, denoted by ai(t). The state values of nodes vary with time. At time t + 1, the state value is calculated as:
(2)
where f(⋅) is the transfer function or activation function. Usually, the sigmoid function can be expressed as:
(3)
is selected as the transfer function. As stated before, the sigmoid function has several drawbacks that limit its nonlinear mapping performance. In this paper, the ReLU function is adopted to replace the sigmoid function. The ReLU function is defined as:
(4)
Denote a(t) = [a1(t), a2(t), ⋯, an(t)]T. Then (2) can be written into the following form:
(5)
In (5), the state value of each node at time t + 1 depends on the state values of all nodes at time t only. If the state value of each node at time t + 1 depends on the state values of all nodes at both time t and time t − 1, ⋯, t − k + 1, i.e.,
(6)
then, the FCM is called a high-order FCM. Eq (6) can be written in a matrix form as follows:
(7)
To enable the FCM to describe the complex dynamics of a system, we must determine its weights first. The weights can be determined in two ways, by using expert knowledge and by learning from actual data. In this paper, the weights of the FCM are obtained by learning. An improved ICA called BICA is developed to perform the learning.
3 Quasi-oppositional bare-bone imperialist competitive algorithm
3.1 Quasi-opposition learning
Opposition-based learning (OBL) was first introduced by Tizhoosh in 2005 [39] inspired by the Ying-Yang concept. OBL aims to find better solutions for machine learning tasks in an opposition direction. Since its introduction, OBL has been applied in several different fields such as neural network training and evolutionary algorithm construction [39]. OBL can be used to improve the performance of original methods. However, the opposition points frequently exceed the searching range and are therefore unable to produce higher performance. To mitigate this problem, in this paper, we focus on a variant of OBL, i.e., quasi-opposition based learning (QOBL). The related concepts are stated as follows.
Definition 1 (Opposite number) Let x ∈ [a, b] be a real number. Its opposite is defined as:
(8)
Definition 2 (Opposite point in the D-dimensional space) [39] Let x = [x1, x2, ⋯, xD] be a point in D-dimensional space and xi ∈ [ai, bi], i = 1, 2, ⋯, D. The opposition of x is denoted as , where each element of
is given as:
(9)
Definition 3 (Quasi-opposite number) Let x ∈ [a, b] be a real number. Its quasi-opposite number is defined as:
(10)
Definition 4 (Quasi-opposite point in the D-dimensional space) Let x = [x1, x2, ⋯, xD] be a point in D-dimensional space and xi ∈ [ai, bi], i = 1, 2, ⋯, D. The quasi-opposition of x is denoted as , where each element of
is given as:
(11)
Definition 5 (Opposition or quasi-opposition based optimization) Let x = (x1, x2, ⋯, xD) be a point in D-dimensional space and f(x) the fitness function. The opposition and quasi-opposition point of x is denoted as and
, respectively. If
or
is better than f(x), then point x is replaced with
or
; otherwise, point x is reserved.
3.2 Bare bone ICA (BBICA)
BICA involves the same main steps as basic ICA, including initialization, assimilation, imperialistic competition, and convergence. These steps are described in detail in the following sections.
3.2.1 Initialization.
In this step, the main task is to generate initial countries, and then classify them into imperialists and colonies according to their power. Furthermore, the colonies are probably assigned to each imperialist to form empires. Suppose a country is represented by a vector xi = [xi1, xi2, ⋯, xiD] and each element xij of vector xi lies in [lj, uj], which is randomly generated as:
(12)
where rand is a random number distributed in [0, 1]. To evaluate each country, the cost of each country is calculated as
, where
is called the cost function. Once N countries are generated, they will be classified into imperialists and colonies according to their power, i.e., the cost of each country. Usually, the first Nimp countries are selected as the imperialists and the remaining Ncol = N − Nimp countries are classified as colonies. Then, the colonies are assigned to each imperialist. The n-th imperialist occupies NCn colonies, which are calculated as:
(13)
where
(14)
is the normalized power of an imperialist and cn is the cost of the n-th imperialist. The NCn colonies together with the imperialist make up the n-th empire.
3.2.2 Assimilation.
After initialization, the colonies in each empire move towards the corresponding imperialists to increase the total power of the empire. This process is called the assimilation step. In the original ICA, the colonies move towards their corresponding imperialists as shown in Fig 2. However, the assimilation is an important step, which is responsible for the exploitation and exploration of the ICA. A good strategy should strike a good balance between exploration and exploitation. To this end, a Gaussian sampling technique is adopted as the assimilation strategy in the BICA. Specifically, the new position of a colony is determined as:
(15)
where
represents the j-th variable of
, i.e., the new position of the i-th colonies in an empire, and N(μj, σj) is a Gaussian distribution random number with mean μj and variance σj. The mean and variance are calculated as:
(16)
and
(17)
where
is the j-th variable of the imperialist position.
3.2.3 Imperialistic competition.
After assimilation, the power of each empire is enhanced. Then, the competition among empires begins. The competition leads more powerful empires to possess more colonies and the weaker ones to lose their colonies. The lost colonies from the weaker empires are reassigned to other more powerful empires via competition. Let the ℓ-th empire be the weakest. The probability of the n-th empire to possess the colonies is given as:
(18)
where NTCn = TCn − maxi{TCi}, i = 1, 2, ⋯, Nimp and TCi is the total cost of the i-th empire. Define a probability vector P as
(19)
and a probability vector R as
(20)
The difference vector between P and R is
(21)
The empire whose relevant index is maximum in D will occupy the weakest colony of the weakest empire.
3.2.4 Convergence.
After empire competition, all the empires will collapse, except the most powerful one, and all the colonies will belong to this empire. As a consequence, all the colonies along with the unique imperialist will be located in the same position and have the same cost. In such a condition, the competition is stopped and the algorithm is terminated.
3.3 Quasi-oppositional bare-bone ICA (QBBICA)
To further improve the global searching performance of BBICA, in this paper, quasi-opposition learning is combined with BBICA. The resulting algorithm is called QBBICA. The difference between BBICA and QBBICA lies in the initialization and assimilation steps.
3.3.1 Quasi-oppositional initialization.
In QBBICA, after randomly generating N countries according to Eq (12), another N corresponding quasi-oppositional countries are generated according to the concept of quasi-oppositional point (11). Then, we calculate the fitness values of the 2N countries and sort them in an ascending order. The first N countries are selected as the initial population.
3.3.2 Quasi-oppositional assimilation.
In QBBICA, after all the colonies in an empire have moved towards their corresponding imperialists according to Eq (15), a set of quasi-oppositional colonies are generated according to Eq (11). Similarly, we calculate the fitness values of new colonies and their corresponding quasi-oppositional colonies, and sort the values in an ascending order. The first N colonies are selected the final colonies in the empire. For complex optimization problems with many local optimum, exploring the information of oppositional point of current location may provide more help to find global optimum. The reasons are as follows. First, the oppositional point maybe result in a better function value, and can speed up the convergence of ICA algorithm. Second, the oppositional direction of current location maybe provide a better descent direction and thus has a large probability to escape the local optimal. Therefore, the oppositional operator can greatly improve the performance of ICA algorithm.
3.3.3 Computation complexity of QBBICA.
QBBICA mainly consists of four steps, i.e., initialization, assimilation, revolution and imperialistic competition. For a D dimension problem and QBBICA with N countries and the maximum iteration number itermax, the computation complexity for generating N initial countries and evaluate its fitness is O(N * D + N * nF), where nF is a number of function evaluation. The computation complexity of generating N quasi-oppositional countries and fitness evaluation is O(N * D + N * nF). Therefore, in the initialization step, the computation complexity of QBBICA is 2O(N * D + N * nF). In the iterative process, the computation complexity of assimilation step is, similarly, 2O(itermax * (N * D + N * nF)). The computation complexity of revolution step is O(itermax * (rr * N + nF * rr * N) and that of imperialist competitive is O(itermax * (Nimp + Nimp * Ncol)), where rr is the revolution rate and Nimp the number of imperialist, Ncol the number of colonies. Finally, the computation complexity of QBBICA is O(itermax * N * D * nF).
4 Predicting employment rate of graduates using proposed method
In this section, we describe in detail how our proposed method is used to predict the employment rate of graduates. Inspired by the work of Wojciech [40], we divide the prediction process into five steps. First, the historical employment rate series is transformed into a fuzzy time series (FTS). Second, the FTS is used to construct an FCM with the ReLU activation function. Third, the FCM is trained using QBBICA. Then, the trained FCM evolves into one or more steps to obtain a predicted fuzzy state. Finally, the predicted fuzzy state is defuzzified to obtain the numerical predicted values of employment rate.
4.1 Fuzzy time series
Let Y = {y(t), t = 1, 2, ⋯, n} be the historical time series of employment rate, and a set of membership functions. The membership value of each y(t) belonging to fuzzy set
is evaluated using
. Thus, the original time series y(t) is transformed into FTS F(t) = {f1(t), f2(t), ⋯, fm(t)}. In this paper, the triangle membership function is adopted.
4.2 Construction of FCM
After the original time series y(t) is transformed into FTS F(t), an FCM is constructed. At any time t, there are m membership values f1(t), f2(t), ⋯, fm(t) corresponding to y(t). The m membership values represent the state of the FCM at time t. Therefore, an FCM with m nodes is constructed, and the activation value of the i-th node equals to the membership value fi(t) at time t.
4.3 Training of FCM using QBBICA
After the FCM is constructed, the weights of FCM should be determined through training. In the literature, many learning algorithms have been developed for FCM learning, such as Hebbian-based, population-based and hybrid algorithms. See [41] for more details. In this paper, the proposed QBBICA is used to learn the FCM. The main steps are illustrated as follows.
- Step 1 Preparing employment rate data {y(t), t = 1, 2, ⋯, n}. The data is normalized into [0, 1]. The first 60% is used as training samples and the remaining as test samples. The employment rate data is normalized and then transformed into m fuzzy sets with the triangle membership function. Consequently, the m FTS fi(t) is obtained.
- Step 2 Constructing the FCM. According to the order k of the FCM and the node number m, an FCM with m nodes is constructed. The status value of FCM at time t is f1(t), f2(t), ⋯, fm(t).
- Step 3 Determining the algorithm parameters of QBBICA such as the number of countries, the number of empires, and the maximum number of iterations.
- Step 4 Solution representation. If the FCM has m nodes, then each weight matrix is a m × m matrix. Furthermore, if the order is k, then there are k weight matrices. Therefore, a vector with length k × m × m is used to represent each country in QBBICA.
- Step 5 Initialization. In the FCM, each weight lies in [−1, 1]. In this step, N vectors with length k × m × m in [−1, 1] are randomly generated. The r-th country is denoted as xr.
- Step 6 Calculating the fitness of each country. To evaluate the power of each country, their fitness is calculated. First, each vector xr is reshaped into k matrices with size m × m, which is denoted as
. Then, the fitness of each country xr is defined as:
(22) where
is the estimated state and ai(t) is the target state.
- Step 7 According the fitness values of the countries, the whole countries are classified into colonies and imperialists and assigned to their corresponding imperialists to form empires.
- Step 8 Perform assimilation on each empire as described in subsection 3.2.2.
- Step 9 Perform empire competition as described in subsection 3.2.3.
- Step 10 Repeat Step 7 and Step 8 until the convergence condition is met.
- Step 11 Output the best vector with the minimum fitness value. Reshaping the vector into weight matrices.
4.4 Predicting employment rate using FCM
After the FCM is trained, it is used to predict the employment rate. Suppose that the state value of the FCM at time t is {a1(t), a2(t), ⋯, am(t)}, where each ai(t) represents the degree of y(t) belonging to the i-th fuzzy set, it is a linguistic variable. After the weights of the FCM are determined, the FCM evolves into a step according to Eq (7) to obtain the predicted state values at t + 1, i.e., . The linguistic value
is transformed into a numerical value using the defuzzification technique. The predicted numerical value
is obtained as:
(23)
where mpi is the center of the i-th fuzzy set.
5 Results
5.1 Performance of QBBICA
Here, the optimization performance of the proposed QBBICA is validated using several benchmark functions. The selected benchmark functions are listed in Table 1. In those six functions, the Sphere function is unimodal and the others are multimodal. To show the advantage of the proposed QBBICA, we compare it with four other algorithms, i.e., GA, PSO, ICA, and BBICA. The parameter settings of these algorithms are as follows. For GA, the “tournament” selection, the “polynomial mutation” and the “Simulated Binary Crossover (SBX)” are adopted. For PSO, c1 = c2 = 2, and the inertia weight decreases linearly from wmax = 0.9 to wmin = 0.4. For DE, the crossover rate CR = 0.8, mutation factor F = 0.8, and “DE / rand-to-best / 1 / bin” strategy is adopted. The parameters for ICA, BBICA, and the proposed QBBICA are set as in [21], which are set as follows, the assimilation coefficient γ = 2, revolution rate equals to 0.3, the number of initial imperialist is 8. For each algorithm, the population size is 50 and the maximum number of iterations is 1000. All the algorithms independently run 30 times.
Table 2 shows the optimization results of benchmark functions with 10 dimension obtained by each algorithm, including the best, the worst and the mean fitness. It can be seen that our proposed QBBICA achieves better optimization results than the other four algorithms. To compare the convergence speeds of these algorithms, Fig 3 shows the fitness convergence curves of the algorithms. It is clear that our proposed QBBICA has a faster convergence speed than the other algorithms. To sum up, our proposed QBBICA not only has a stronger global search ability but also a faster convergence speed than the other algorithms. To compare the superiority of the proposed QBBICA, a Wilcoxon sign-rank-sum test had been conducted and the h values of the test are listed in Table 3. The h-value equaling to 1 indicates that QBBICA is better than other algorithm. From Table 3, one can see that the proposed QBBICA performs better than other algorithms except for PSO on sphere function. Furthermore, to demonstrate the computational efficiency, the average computation time of each algorithm during the 30 runs for each function are listed in Table 4.
(a) Sphere, (b) Rastrigin, (c) Griewangk, (d) Ackley, (e) Alpine, (f) Pathologic.
5.2 Predicting employment rate
In this section, experiments are conducted to predict the employment rate of graduates from Liren College, Yanshan University. The employment rate is the ratio of the number of the employed graduates to the total number of graduates. The employment rate data of Liren College from 2005 to 2020 is used in the experiments, which is listed in Table 5.
The first 60% of the employment rate data is used as the training sample and the remaining 40% as the test sample.
In the experiments, five triangle membership functions are adopted to transform the employment rate into an FTS. Thus, there are five nodes in the FCM. The order of the FCM is set to 2 in this paper. Thus, there are two weight matrices and the size of each matrix is 5 × 5. For the purpose of comparison, the RCGA, PSO, ICA, BBICA and the proposed QBBICA are implemented and used to train the ReLU-FCM and the original FCM, denoted as Sig-FCM. For the sake of convenience, the FCMs trained by these algorithms are denoted as RCGA-FCM, PSO-FCM, ICA-FCM, BICA-FCM, and QBBICA-FCM, respectively. In these algorithms, each individual is represented by a vector with length 50. Each algorithm has 200 individuals, and the maximum number of iterations of each algorithm is 1000. In addition, to reveal its effectiveness in employment rate prediction, the FCM based prediction methods are also compared back propagation neural network, linear regression model and grey GM(1,1) model.
To objectively reflect the performance of the FCMs trained by the different algorithms, root mean square error (RMSE), coefficient of determination R2, coefficient of efficiency Esn, and mean absolute error (MAE) are used as the indices for performance evaluation. They are defined as follows:
(24)
where yi and
are the real and predicted values, respectively, and N is the total number of data points used in the experiment.
Coefficient of determination:
(25)
where
and
are the mean values of the real and predicted sequences, respectively. The larger the R2 is, the more variability is explained by the model.
Coefficient of efficiency:
(26)
The larger the coefficient of efficiency is, the closer the predicted value is to the real value.
In the training phase, all the algorithms are run independently for 30 times. The employment rate data from 2005 to 2017 is used as the training set and that from 2018 to 2020 as the test set. The individual corresponding to the best fitness value in the 30 runs is taken as the final weight of the FCM. With the obtained weight, numerical prediction values are calculated using Eq (23). The prediction results from the training and test phases for ReLU-FCM and Sig-FCM are listed in Tables 6 and 7, and also, they are shown in Fig 4 for the sake of intuition. The performance indices are listed in Tables 8 and 9, respectively. It can be seen that the ReLU-FCM trained by QBBICA delivers better results than other FCMs in both prediction values and performance indices. The main reason for this is that QBBICA has a stronger search ability, which allows it to better balance exploration and exploitation. Therefore, QBBICA can find more appropriate weights of FCMs. Furthermore, the ReLU activation function has no saturation phenomenon, which enables ReLu-FCM to deliver better performance than Sig-FCM. On the other hand, to illustrate the computational efficiency of QBBICA and other referenced algorithms in FCM training process, the average cpu time in 30 runs is shown in Fig 5. Obviously, one can see that the average cpu time of expanded expended by QBBICA is less than ICA, BBICA, PSO and GA, but is competitive to DE. For a more straightforward comparison, the evolution curves of those algorithms are plotted in Fig 6. From Fig 6, it can be seen that QBBICA not only has a faster convergence speed but also a stronger search ability.
(a) ReLu-FCM, (b) Sig-FCM.
In order to demonstrate the advantage of FMC in prediction, the prediction results obtained by our proposed QBBICA-FCM is compared to those by BP neural network, linear regression (LR) model and grey GM(1,1) model, which are listed in Tables 10 and 11. From Table 10, one can see that the prediction value obtained by our QBBICA-FCM is more accurate than those by BP neural network, LR model and GM(1,1) model. Furthermore, the four performance indices of QBBICA-FCM are also better than those of BP, LR and GM(1,1). This proves that our proposed QBBICA-FCM can achieve more accurate prediction results. From the above experimental results and comparisons, one can draw a conclusion that our proposed method can give more accurate prediction results than the other methods.
6 Conclusion
In this paper, an improved FCM is proposed to predict the employment rate of university graduates. Two improvements are made in order to get more accurate prediction results. One is that the sigmoid activation function is replaced with the ReLu function. The other is that an improved ICA algorithm, called quasi-oppositional learning bare-bone ICA (QBBICA), is proposed to train the FCM. The QBBICA algorithm has a stronger search ability and can obtain more appropriate weights of the FCM. Experiments are conducted to predict the employment rate of graduates from Liren College, Yanshan University. Results show that our proposed method can give accurate prediction results.
References
- 1. Wang T. Intelligent employment rate prediction model based on a neural computing framework and human-computer interaction platform. Neural Computing and Applications. 2020;32(21):16413–16426.
- 2.
Guo T, Xia F, Zhen S, Bai X, Zhang D, Liu Z, et al. Graduate Employment Prediction with Bias. In: Proceedings of the 34-th AAAI Conference on Artificial Intelligence (AAAI),. vol. 34; 2020. p. 670–677.
- 3. Ma D, Yu Z, Liu Y. Research on application of BP neural network in predicting employment rate of college graduates. Journaal of Jiamusi University (Natural Science Edition). 2014;32(5):751–753.
- 4. Jiang Z, Yuan Z, Yan R. College students employment forecasting model based on IAFSA-BP parallel integrated learning algorithm. Value Engineering. 2019;38(19):232–234.
- 5.
Wang Y. Research on University Students’ employmnet Prediction model and Application based on Decision tree algorithm. Central China Normal University; 2018.
- 6. Pan W, Huang L. Combination forecast model based on B-mode relational degree and GIOWA operator. Statistics & Decision. 2018;(11):73–76.
- 7. Zhang W, Qianhan H. Prediction of the employment rate of graduates and research of the quality assessment. Computer Engineering & Science. 2009;31(5):141–143.
- 8. Kosko B. Fuzzy cognitive maps. International Journal of Man-Machine Studies. 1986;24(1):65–75.
- 9. Kiani M, Andreu-Perez J, Hagras H, Papageorgiou EI, Prasad M, Lin CT. Effective Brain Connectivity for fNIRS With Fuzzy Cognitive Maps in Neuroergonomics. IEEE Transactions on Cognitive and Developmental Systems. 2022;14(1):50–63.
- 10. Shen F, Liu J, Wu K. A Preference-Based Evolutionary Biobjective Approach for Learning Large-Scale Fuzzy Cognitive Maps: An Application to Gene Regulatory Network Reconstruction. IEEE Transactions on Fuzzy Systems. 2020;28(6):1035–1049.
- 11. Amirkhani A, Kolahdoozi M, Wang C, Kurgan LA. Prediction of DNA-Binding Residues in Local Segments of Protein Sequences with Fuzzy Cognitive Maps. IEEE/ACM Transactions on Computational Biology and Bioinformatics. 2020;17(4):1372–1382. pmid:30602422
- 12. Rickard JT, Aisbett J, Morgenthaler DG, Yager RR. Modeling of Complex System Phenomena via Computing With Words in Fuzzy Cognitive Maps. IEEE Transactions on Fuzzy Systems. 2020;28(12):3122–3132.
- 13. Wu K, Liu J, Liu P, Yang S. Time Series Prediction Using Sparse Autoencoder and High-Order Fuzzy Cognitive Maps. IEEE Transactions on Fuzzy Systems. 2020;28(12):3110–3121.
- 14. Lu W, Feng G, Liu X, Pedrycz W, Zhang L, Yang J. Fast and Effective Learning for Fuzzy Cognitive Maps: A Method Based on Solving Constrained Convex Optimization Problems. IEEE Transactions on Fuzzy Systems. 2020;28(11):2958–2971.
- 15. Amirkhani A, Shirzadeh M, Shojaeefard MH, Abraham A. Controlling wheeled mobile robot considering the effects of uncertainty with neuro-fuzzy cognitive map. ISA Transactions. 2020;100:454–468. pmid:31916988
- 16. Stach W, Pedrycz W, Kurgan LA. Learning of Fuzzy Cognitive Maps Using Density Estimate. IEEE Transactions on Systems Man & Cybernetics, Part B, Cybernetics. 2012;42:900–912. pmid:22345544
- 17.
Parsopoulos KE, Papageorgiou EI, Groumpos PP, Vrahatis MN. A first study of fuzzy cognitive maps learning using particle swarm optimization. In: The 2003 Congress on Evolutionary Computation, 2003. CEC’03. vol. 2; 2003. p. 1440–1447 Vol.2.
- 18.
Ighravwe DE, Mashao D. Development of a Differential Evolution-based Fuzzy Cognitive Maps for Data Breach in Health-care Sector Fuzzy Cognitive Maps for Data Breach. In: 2019 IEEE AFRICON; 2019. p. 1–5.
- 19. Salmeron JL, Mansouri T, Moghadam MRS, Mardani A. Learning Fuzzy Cognitive Maps with modified asexual reproduction optimisation algorithm. Knowledge-Based Systems. 2019;163:723–735.
- 20. Ahmadi S, Forouzideh N, Alizadeh S, Papageorgiou EI. Learning Fuzzy Cognitive Maps using Imperialist Competitive Algorithm. Neural Comput Appl. 2015;26(6):1333–1354.
- 21.
Atashpaz-Gargari E, Lucas C. Imperialist competitive algorithm: An algorithm for optimization inspired by imperialistic competition. 2007 IEEE Congress on Evolutionary Computation, CEC 2007. 2007; p. 4661–4667.
- 22. Li M, Lei D. An imperialist competitive algorithm with feedback for energy-efficient flexible job shop scheduling with transportation and sequence-dependent setup times. Engineering Applications of Artificial Intelligence. 2021;103:104307.
- 23. Ahmadi N, Akbarizadeh G. Iris tissue recognition based on GLDM feature extraction and hybrid MLPNN-ICA classifier. Neural Computing and Applications. 2020;32:2267–2281.
- 24. Yousefi M, Darus AN, Mohammadi H. An imperialist competitive algorithm for optimal design of plate-fin heat exchangers. International Journal of Heat and Mass Transfer. 2012;55(11-12):3178–3185.
- 25. Elsisi M. Design of neural network predictive controller based on imperialist competitive algorithm for automatic voltage regulator. Neural Computing and Applications. 2019;31:5017–5027.
- 26. Barkhoda W, Sheikhi H. Immigrant imperialist competitive algorithm to solve the multi-constraint node placement problem in target-based wireless sensor networks. Ad Hoc Networks. 2020;106:102183.
- 27. Afonso LD, Mariani VC, dos Santos Coelho L. Modified imperialist competitive algorithm based on attraction and repulsion concepts for reliability-redundancy optimization. Expert Systems with Applications. 2013;40(9):3794–3802.
- 28. Rabiee A, Sadeghi M, Aghaei J. Modified imperialist competitive algorithm for environmental constrained energy management of microgrids. Journal of Cleaner Production. 2018;202:273–292.
- 29. Idoumghar L, Chérin N, Siarry P, Roche R, Miraoui A. Hybrid ICA–PSO algorithm for continuous optimization. Applied Mathematics and Computation. 2013;219(24):11149–11170.
- 30. Chen CH, Chen WH. Bare-bones imperialist competitive algorithm for a compensatory neural fuzzy controller. Neurocomputing. 2016;173:1519–1528.
- 31.
Yang SD, Yi YL, Shan ZY. Gbest-guided Imperialist competitive algorithm for global numerical optimization. Proceedings—2012 International Conference on Computer Distributed Control and Intelligent Environmental Monitoring, CDCIEM 2012. 2012; p. 352–355.
- 32. Kaveh A, Talatahari S. Optimum design of skeletal structures using imperialist competitive algorithm. Computers & Structures. 2010;88(21):1220–1229.
- 33. Mortazavi A, Khamseh AA, Naderi B. A novel chaotic imperialist competitive algorithm for production and air transportation scheduling problems. Neural Computing and Applications. 2015;26(7):1709–1723.
- 34. Goldansaz SM, Jolai F, Anaraki AHZ. A hybrid imperialist competitive algorithm for minimizing makespan in a multi-processor open shop. Applied Mathematical Modelling. 2013;37(23):9603–9616.
- 35. Yin Z, Gao Q. A Novel Imperialist Competitive Algorithm for Scheme Configuration Rules Extraction of Product Service System. Procedia CIRP. 2019;80:762–767.
- 36. Peri D. Hybridization of the imperialist competitive algorithm and local search with application to ship design optimization. Computers & Industrial Engineering. 2019;137:106069.
- 37. Zandieh M, Khatami AR, Rahmati SHA. Flexible job shop scheduling under condition-based maintenance: Improved version of imperialist competitive algorithm. Applied Soft Computing. 2017;58:449–464.
- 38. Moazzami M, Morshed MJ, Fekih A. A new optimal unified power flow controller placement and load shedding coordination approach using the Hybrid Imperialist Competitive Algorithm-Pattern Search method for voltage collapse prevention in power system. International Journal of Electrical Power & Energy Systems. 2016;79:263–274.
- 39.
Tizhoosh HR. Opposition-Based Learning: A New Scheme for Machine Intelligence. In: International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06). vol. 1; 2005. p. 695–701.
- 40. Stach W, Kurgan LA, Pedrycz W. Numerical and Linguistic Prediction of Time Series With the Use of Fuzzy Cognitive Maps. IEEE Transactions on Fuzzy Systems. 2008;16(1):61–72.
- 41. Papageorgiou EI. Learning Algorithms for Fuzzy Cognitive Maps—A Review Study. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). 2012;42(2):150–163.