Leuconostoc Mesenteroides Growth in Food Products: Prediction and Sensitivity Analysis by Adaptive-Network-Based Fuzzy Inference Systems

Background An adaptive-network-based fuzzy inference system (ANFIS) was compared with an artificial neural network (ANN) in terms of accuracy in predicting the combined effects of temperature (10.5 to 24.5°C), pH level (5.5 to 7.5), sodium chloride level (0.25% to 6.25%) and sodium nitrite level (0 to 200 ppm) on the growth rate of Leuconostoc mesenteroides under aerobic and anaerobic conditions. Methods The ANFIS and ANN models were compared in terms of six statistical indices calculated by comparing their prediction results with actual data: mean absolute percentage error (MAPE), root mean square error (RMSE), standard error of prediction percentage (SEP), bias factor (Bf), accuracy factor (Af), and absolute fraction of variance (R 2). Graphical plots were also used for model comparison. Conclusions The learning-based systems obtained encouraging prediction results. Sensitivity analyses of the four environmental factors showed that temperature and, to a lesser extent, NaCl had the most influence on accuracy in predicting the growth rate of Leuconostoc mesenteroides under aerobic and anaerobic conditions. The observed effectiveness of ANFIS for modeling microbial kinetic parameters confirms its potential use as a supplemental tool in predictive mycology. Comparisons between growth rates predicted by ANFIS and actual experimental data also confirmed the high accuracy of the Gaussian membership function in ANFIS. Comparisons of the six statistical indices under both aerobic and anaerobic conditions also showed that the ANFIS model was better than all ANN models in predicting the four kinetic parameters. Therefore, the ANFIS model is a valuable tool for quickly predicting the growth rate of Leuconostoc mesenteroides under aerobic and anaerobic conditions.


Introduction
Growth prediction models are now widely used informative tools for rapid and cost-effective assessment of microbial growth for product development, risk assessment, and education purposes [1]. In recent studies of shelf life in food products, microbiologists have used predictive models to forecast spoilage caused by the growth of micro-organisms. Despite the major technological advances in the food industry in recent years, fungal spoilage of food commodities remains a major cause of economic losses for food producers and an important health concern for regulatory agencies. Therefore, improved understanding of fungal growth in foods, particularly those factors associated with new manufacturing processing and packaging techniques, is urgently needed [2]. Fungi degrade the organoleptic properties of foods by producing visible mycelium, and fungal contamination is often implicated in off-flavor food products. In addition to the economic effects of consumer rejection, the diminished nutritional value and, more importantly, the production of potentially carcinogenic toxic metabolites, pose a public health risk [3]. Improvements in food quality and safety require the development of appropriate fungal growth prediction tools. For many years, research in predictive microbiology has focused on food-borne pathogens whereas models for predicting growth in filamentous fungi have received relatively less attention [4]. Recently, however, the situation has changed, and the literature now shows a growing number of studies of models for this purpose [5][6][7].
Leuconostoc mesenteroides (LM) is a common spoilage microorganism in cooked meat products. These bacteria can alter food products by fermentation of sugars, which forms lactic acid. By contributing to slime mold formation and CO 2 production, they also degrade the smells and flavors of food products. Because the resulting sensorial qualities of the product can make it unaccept-able for consumption [8], spoilage caused by these microorganisms is potentially a major cause of economic loss in the food industry. Therefore, a tool is needed for predicting the growth capacity of this microorganism to multiply in a food product under conditions that typically occur while processing, preserving, storing and distributing foods. An effective tool for predicting shelf life would help to reduce economic losses from deterioration of food. In an earlier study of tools for predicting LM growth rates under aerobic and anaerobic conditions, Zurera-Cosano et al. [9] used response surface methodology (RSM) to compare the combined effects of different temperatures, pH levels, sodium chloride levels and sodium nitrite levels on the accuracy of LM growth rate predictions under aerobic and anaerobic conditions. The RSM models showed potential use for estimating shelf life in food products. However, a subsequent study by Garcia-Gimeno et al. [10] showed that an artificial neural network (ANN) model was more accurate than RSM for predicting LM growth given similar environmental conditions.
A literature review shows that most studies in this line of research have used ANN models for predicting the growth of spoilage microorganisms in food products [7] because ANNs can handle high-level nonlinearities, numerous parameters and missing information [11][12][13][14][15]. Hajmeer et al. [16] developed an ANN model of Shigella flexneri growth and reported that it outperformed regression equations in terms of mean absolute percentage error (MAPE) and coefficient of determination. Again, however, a noted limitation of the ANN model was its high complexity. Geeraerd et al. [17] reported that an ANN model was superior to conventional Table 1. Average values for observed (OBS) growth rate (G r , h 21 ) and average values estimated by ANN [10] and by ANFIS models of Leuconostoc mesenteroides growth under aerobic conditions during model development with the training data set [10]. T  microbiological models in terms of accuracy in predicting the effects of temperature, pH and NaCl on microbial activity. Jeyamkondan et al. [18] reported that, for predicting growth in Aeromonas hydrophila, S. flexneri and Brochothrix thermosphacta, ANN models were superior to statistical models in terms of root mean square error (RMSE), mean relative percentage error (MRPE) and MAPE. Lou and Nakai [19] applied response surface methodology, a Cerf model and an ANN model in a study of the effects of temperature, water activity and pH on the thermal inactivation of Listeria monocytogenes. Again, the network-based approach proved superior in terms of RMSE and coefficient of determination. Lou and Nakai [20] further showed that ANN was more accurate than response surface methodology in a study of the effects of temperature, water activity and dissolved CO 2 concentration on the kinetic parameters of Lactobacillus sake. In Garcia-Gimeno et al. [21], another comparison between ANNs and RSM models for predicting growth rates in L. plantarum and E. coli showed that, although the ANN models had higher complexity, they had lower standard error of prediction percentage (SEP) terms compared to the statistical models. Panagou and Kodogiannis [7] compared ANN methods and polynomial methods of modeling the joint effects of water activity, pH level and temperature to predict the maximum growth rate of ascomycetous fungus Monascus ruber.
Comparisons of six statistical indices, i.e., coefficient of determination, RMSE, MRPE, MAPE, SEP, bias factor (B f ), and accuracy factor (A f ), confirmed that, for modeling microbial kinetic parameters, ANNs are an acceptable alternative to polynomial methods.
For describing relationships between different combinations of inputs and outputs such as those that must be determined for accurately predicting growth in spoilage microorganisms, ANN is currently the most widely used technique. A recent literature review shows that the use of adaptive-network-based fuzzy inference system (ANFIS) [22,23] for such purposes is relatively rare. Jang [22] implemented ANFIS in an adaptive fuzzy neural network framework by using a hybrid learning procedure combining back-propagation gradient descent and least square methods. Since the membership functions in the resulting fuzzy inference system were iteratively adjustable according to a given training set of inputs and outputs, the ANFIS could effectively map input-output relationships according to both human knowledge (in the form of fuzzy if-then rules) and stipulated input-output data pairs [24,25]. In the current study, ANFIS was used to model the relationship between predicted and actual LM growth rates under various conditions. Therefore, this study evaluated the accuracy of ANFIS in predicting LM growth rates under aerobic and anaerobic conditions. The ANFIS and ANN models were then compared in terms of their accuracy in predicting LM growth under varying experimental conditions, including temperature, pH, salt, nitrite concentrations, and under aerobic and anaerobic conditions. Tables 1 and 2 define the four parameters used to  compare the ANFIS and ANN models [10] for predicting LM growth under aerobic and anaerobic conditions. The ANFIS model was trained using Gaussian membership functions. Some experimental results obtained by ANFIS method were also compared with those obtained by ANN methods in an earlier study by Garcia-Gimeno et al. [10]. Finally, sensitivity analyses were performed to identify the environmental factors that had the largest effects on the accuracy of the predictions of LM growth rate under aerobic and anaerobic conditions.

ANFIS architecture
The ANFIS multilayer feed-forward network of nodes and directional links combines the learning capabilities of an ANN with the reasoning capabilities of fuzzy logic. The ANNs and fuzzy inference systems (FISs) are complementary technologies in the design of adaptive intelligent systems. The ANNs, which learn from scratch by adjusting the interconnections among neurons, are noted for their generalization capability. That is, a properly trained ANN can correctly match a set of new input data to output data. The FIS is a popular computing framework based on fuzzy set theory, fuzzy if-then rules, and fuzzy reasoning. For a given set of if-then rules, an FIS can perform nonlinear mapping from its input space to its output space [22][23][24][25].
By storing its essential components in a knowledge base, fuzzy reasoning enables an FIS to infer an overall output value according to human expertise. However, no systematic method of transforming the experience and knowledge of human experts to an FIS knowledge base has been developed. To build an FIS, the fuzzy sets, fuzzy operators and knowledge base must be specified. To construct an ANN for an application, the architecture and learning algorithm must be specified. The homogenous structure of an ANN complicates the extraction of structured knowledge from the weights of interconnections between its Table 3. Average values for observed (OBS) growth rate (G r , h 21 ) and average values estimated by ANN [10] and by ANFIS models of Leuconostoc mesenteroides growth under aerobic conditions using testing data set [10].  neurons. The a priori knowledge of human experts is often needed to solve practical problems. However, encoding prior knowledge into an ANN is difficult. The ANFIS combines the advantage of FIS, i.e., its learning capability, with the advantage of ANN, i.e., its formation of a linguistic rule base. To obtain an ANFIS that can share data structures and knowledge representations, ANN and FIS are typically integrated by using a special ANN architecture to represent FIS [22][23][24][25].
The hybrid network structure of ANFIS can achieve prediction capabilities superior to those of ANN alone and fuzzy logic techniques alone. By analyzing mapping relationships between input and output data, ANFIS optimizes the distribution of membership functions by using a back-propagation gradient descent algorithm either alone or combined with a least-squares method [22]. The ANFIS uses fuzzy if-then rules involving premise and consequent parts of a Sugeno-type fuzzy inference system [22]. Figure 1 shows how the description of this system can be simplified as an inference system of inputs m and n and output f [22]. The corresponding ANFIS architecture is also shown. The five-layer system ANFIS architecture includes a fuzzification layer (Layer 1), a production layer (Layer 2), a normalization layer (Layer 3), a de-fuzzification layer (Layer 4), and a total output layer (Layer 5) [22].
In the first layer, which is the fuzzification layer, m and n are the inputs of nodes A 1 and A 2 and nodes B 1 and B 2 , respectively. Each node i in this layer is a square node (adaptive node A i and B i , i~1, 2). The linguistic labels used in fuzzy theory to divide membership functions are A 1 (small), A 2 (large), B 1 (small) and B 2 (large). The Gaussian membership relationship between the output and input functions of this layer can be expressed as and where O 1,i and O 1,j denote the output functions, m A i m ð Þ and m B j n ð Þ denote the Gaussian membership functions; parameters c A i and c B j (the center) and parameters a A i and a B j are represented by and j~1, 2,) and denote the nonlinear parameters of the premise part, which are the width of the Gaussian membership function of the ith implication of input variables m and n, respectively. The second layer, which is the production layer, consists of two circular nodes labeled P, which multiply the incoming signals. The resulting product is used to represent the firing strength of a rule. In this layer, the product operator that performs a generalized AND can be used as the node function. Outputs W 1 and W 2 are the weight functions of the next layer. The output of this layer is the product of the input signal, which is defined as follows: where O 2,i denotes the output of Layer 2. The third layer is the normalization layer. Each node in this layer is a circular node labeled N. The ith node calculates the ratio of the firing strength of the ith rule to the sum of the firing strengths of all rules. Its function is to normalize the weight function by applying the following process: where O 3,i denotes the Layer 3 output.  The fourth layer is the de-fuzzification layer. Each node i in this layer is a square node (adaptive node, i~1, 2). The output equation is where W W i is the output of Layer 3 and where p i , q i , and r i represented by p i , q i , r i f g(i~1, 2) denote the linear parameters or the so-called consequent parameters of the node. The de-fuzzification relationship between the input and output of this layer can be expressed as the output of the ith rule as follows: where O 4,i denotes the Layer 4 output. The fifth layer is the total output layer. The single node in this layer is a circular node labeled S, which computes the overall output by summing all incoming signals. The output of this layer is the total of the input signals, which represents the predicted growth rate of LM. The results can be written as where O 5,i denotes the Layer 5 output. Figure 1 further shows the typical if-then rules in a first-order Sugeno fuzzy model [24,25]: In the ANFIS architecture, the nonlinear parameters a Ai , a Bj , c Ai , c Bj È É of the premise part and the linear parameters p i , q i , r i f gof the consequent part can be trained by using a hybrid learning procedure combining back-propagation gradient descent with least square methods. Figure 2 is a flowchart showing the growth rates predicted by ANFIS. The training data are used to train the nonlinear parameters a Ai , a Bj , c Ai , c Bj È É of the premise part and the linear parameters p i , q i , r i f gof the consequent part. The number of membership functions is set for each input parameter until prediction performance is satisfactory. After the training results are obtained by this process, the testing data are input into the trained ANFIS model to obtain the testing results. The following sections present the details of the input-output relationships in each ANFIS layer. Note that the system output is the weighted sum of the results of the rules. The number of fuzzy sets depends on the number of Layer 1 nodes. However, the number of Layer 4 dimensions determines the number of fuzzy rules used in the ANFIS architecture. Therefore, the number of Layer 4 dimensions indicates the complexity and flexibility of the ANFIS architecture. The number of fuzzy rules in an ANFIS is analogous to the number of neurons in an ANN [25].
Like ANNs, an ANFIS network can be trained by supervised learning by using a specified output to provide a target output. The forward pass of the hybrid algorithm of the ANFIS moves the node outputs forward to Layer 4. The consequent linear parameters p i , q i , r i f gare then obtained by least-squares method [22,25]. In the backward pass, the error signals propagate   [22,25]. That is, the premise nonlinear parameters a Ai , a Bj , c Ai , c Bj È É and the consequent linear parameters p i , q i , r i f g are trained in the ANFIS.

Evaluation criteria
As described in the literature, the criteria used to compare fitting and prediction accuracy between the ANFIS model and the ANN model were MAPE, RMSE, SEP, B f , A f , and the absolute fraction of variance (R 2 ). The MAPE indicates the relative absolute percentage deviation in experimental values where a lower value implies a better correlation. The equation for calculating MAPE is whereÑ N is the number of the points in the data set, y i is the observed value, andŷ y i is the prediction value. The RMSE is given by Although unlikely, an RMSE of 0 indicates the best possible fit between predicted and actual values. The SEP(%) is the relative typical deviation in mean prediction values. The advantage of SEP(%) compared to other error measures is that SEP(%) is calculated independently of the magnitude of the measurements. The equation for calculating SEP(%) is where y y is the mean of observed values. To evaluate the predictive capacity of the proposed model, the following formulas were used to calculate RMSE and SEP(%)together with B f and A f [26] : and A f~1 0 The equation for R 2 is An R 2 value of 1 indicates a very good fit whereas a value approaching 0 indicates a poor fit.

Sensitivity analysis of ANFIS output
Sensitivity analysis was performed with ANFIS learning disabled so that network weights would not be affected [27]. The first input varies between its mean plus or minus a user-    defined number of standard deviations whereas all other inputs are fixed at their respective means. The ANFIS output is computed and recorded as the percent change above and below the mean channel output. This process is repeated for each input variable [28]. In this so-called 'sensitivity' analysis to determine the relative importance of input variables for determining output, 0 indicates a  variable that does not affect prediction, and 1.0 indicates a field that completely dominates the prediction [28].
To compare the robustness of the ANFIS and ANN models, bootstrap method [29] was used to minimize problems resulting from insufficient data. Five bootstrap data sets derived from the original data set with different numbers of bootstrap samples (30,60,90, 120 and 150) were prepared under both aerobic and anaerobic conditions. At this step, the goal was to determine the appropriated number of bootstrap samples needed for model evaluation. Tables 7 and 8 show the means and standard deviations in each variable for five sets of bootstrap data.
Bootstrap resampling has been widely used in applied statistics in the past three decades [30] since the bootstrap method was first introduced by Efron in 1979 [29]. Boostrapping is a Monte Carlotype data augmentation method of resampling with replacement that can be used with observed data. While Monte Carlo techniques usually generate fictitious data, bootstrap resamples in which the originally observed values are replaced can be used to generate multiple bootstrap samples for use as a proxy for the actual independent sample. Each bootstrap sample is a random sub-sample (of a size equal to that of the original sample) in which the actually observed values are replaced. The original sample is considered a ''virtual population'', and the sample is repeatedly duplicated. The procedure can be repeated as needed [30].

Results
First, the ANFIS was trained using 30 data sets (Tables 1 and 2) selected from the 58 data sets obtained in the experiments. After training was completed, another 28 data sets were then used to verify its accuracy in predicting growth rates (Tables 3 and 4). The prediction results are further analyzed and discussed below. Figure 3 shows the fuzzy rule architecture of an ANFIS with Gaussian membership function. The four inputs (temperature, pH, NaCl, and NaNO 2 ) and one output (growth rate) of the ANFIS Figure 7. Actual values for growth rate of Leuconostoc mesenteroides compared with values predicted by ANFIS model and by ANN model under anaerobic conditions. Under anaerobic conditions, the growth rates predicted by ANFIS models in (A) the training data set and in (B) the testing data set were closer to the 45u line compared to those predicted by the ANN models. In other words, the predictive accuracy of the ANFIS models was higher than that of ANN models under anaerobic conditions. doi:10.1371/journal.pone.0064995.g007 model were designed using MATLAB Fuzzy Logic Toolbox [31]. Each input divides three Gaussian membership functions. Therefore, the architecture in Figure 3 includes 81 fuzzy rules. The ANFIS was trained using 30 sets of experimental data in 100 learning cycles. For each input in the architecture, the Gaussian membership function can be divided into small, medium and large areas. Figure 4 shows the final Gaussian membership functions of the four inputs derived by training under aerobic and anaerobic conditions, respectively. Tables 5 and 6 show the prediction results obtained by the ANFIS models under aerobic and anaerobic conditions, respectively, and Figures 5, 6, 7 and 8 show the training and testing data sets. The MAPEs obtained under aerobic conditions are 0.18 for the training data set and 0.27 for the testing data set, and those obtained under aerobic conditions are 0.39 for the training data set and 0.14 for the testing data set. The RMSEs obtained under aerobic conditions are 0.001 for the training data set and 0.002 for the testing data set, and those obtained under anaerobic conditions are 0.002 for the training data set and 0.001 for the testing data set. The SEPs obtained under aerobic conditions are 0.58 for the training data set and 0.67 for the testing data set, and those obtained under anaerobic conditions are 0.81 for the training data set and 0.44 for the testing data set.
Comparisons of performance indices for predictions of the 30 patterns on which the models were trained showed that the ANFIS was more accurate than the ANN. Specifically, the ANFIS models were superior in terms of MAPE, RMSE and SEP (Tables 5 and  6). For the 28 testing data sets, comparisons of model performance again confirmed the superior performance of the ANFIS models, considering the biological variability associated with the experiment. Additionally, measurements of bias and accuracy in the ANFIS models approached unity in all experiments, which indicated good agreement between observations and predictions. The performance difference between the two examined models  . Three-layer feedforward ANN. The input layer includes nodes for four inputs (temperature, pH, NaCl, and NaNO 2 ). In the hidden layer, three nodes transfer data to the output layer via a separate weighted logistic transfer function. In the output layer, one node (growth rate) transfers data via a separate weighted identity transfer function to obtain the predictive output. doi:10.1371/journal.pone.0064995.g009 was also graphically depicted in plots of bias (observed vs. predicted growth rate) and residual bias for all data sets. Generally, the predictions derived by ANFIS models were closer to the line of equity compared to the ANN models ( Figures 5 and 7), which indicated the better fit of the ANFIS models. In both the ANFIS and ANN models, residuals were also symmetrically distributed    around 0 with no systematic tendency to appear on the positive or negative sides of the graph (Figures 6 and 8). The narrower spread of residual values obtained by the ANFIS models in all data sets also indicated their superior prediction performance.
To compare model robustness, five bootstrap data sets under aerobic and anaerobic conditions were applied and compared. Tables 7 and 8 show the means and standard deviations of each variable for the five bootstrap data sets containing 30, 60, 90, 120 and 150 samples. Tables 9 and 10 show that, in all five bootstrap data sets, characteristics represented by mean values of four inputs and one output did not significantly differ from those in the original data set (analysis of variance (ANOVA) test using SPSS 14.0, P.0.05) under aerobic and anaerobic conditions, respectively, which confirmed the reliability of the data selection. Therefore, the 30-bootstrap data set was selected to construct the ANN model and the ANFIS model with 10-fold cross-validation. Figure 9 shows the three-layer feedforward ANN used for comparison. Both the ANN model and the ANFIS model had a four-input (temperature, pH, NaCl, and NaNO 2 ) node layer. A weighted logistic transfer function was used to transfer these inputs to a three-node hidden layer. A separate weighted identity transfer function was then used to transfer data from the hidden layer to the one-node (growth rate) output layer. The ANN model was constructed with STATISTICA 10.0 software (StatSoft, Tulsa, OK). The prediction results are further analyzed and discussed below. Tables 11 and 12 show the prediction results. Under aerobic conditions, the averages and standard deviations of the MAPEs were 0.347 and 0.117 in the ANFIS models and 11.041 and 2.576 in the ANN models, respectively. Under anaerobic conditions, the averages and standard deviations of the MAPEs were 0.382 and 0.216 in the ANFIS models and 12.325 and 1.925 in the ANN models, respectively. Under aerobic conditions, the averages and standard deviations of the RMSEs were 0.0016 and 0.0007 in the ANFIS models and 0.035 and 0.008 in the ANN models, respectively. Under anaerobic conditions, the averages and standard deviations of the RMSEs were 0.0016 and 0.0009 in the ANFIS models and 0.036 and 0.006 in the ANN models, respectively. Under aerobic conditions, the averages and standard deviations of the SEPs were 0.564 and 0.214 in the ANFIS models and 0.889 and 0.026 in the ANN models, respectively. Under anaerobic conditions, the averages and standard deviations of the SEPs were 0.569 and 0.315 in the ANFIS models and 14.935 and 2.821 in the ANN models, respectively.
The performance comparisons confirmed that the ANFIS models were superior in terms of MAPE, RMSE and SEP (Tables 11 and 12), considering the biological variability associated with the experiment. That is, the results shown in Tables 11 and  12 are consistent with those shown in Tables 5 and 6, respectively, which again confirms the superior accuracy of the ANFIS for predicting LM growth rates under aerobic and anaerobic conditions. Figures 10, 11 12 and 13 show the results of sensitivity analyses of 30-boothstrap data set under both aerobic and anaerobic conditions. In the ANFIS model, the sensitivity values for temperature, NaCl, pH and NaCO 2 were 0.88, 0.13, 0.07 and 0.04 under aerobic conditions and 0.46, 0.32, 0.19 and 0.25 under anaerobic conditions, respectively. In the ANN model, the sensitivity values for temperature, NaCl, pH and NaCO 2 were 0.2, 0.06, 0.04 and 0.02 under aerobic conditions and 0.52, 0.16, 0.09 and 0.16 under anaerobic conditions, respectively. That is, the ANFIS and ANN models showed similar results in sensitivity analyses. In terms of effect on the growth rate of LM under aerobic and anaerobic conditions, the most influential (sensitive) param-eters in both the training and testing data sets were temperature and, to a lesser extent, NaCl. Tables 5 and 6 compare the statistical data for the two models  with the original data set, and Figures 5, 6 7  Notably, although R 2 is a common criterion for comparing statistical models [32], it assumes a normally distributed error that is independent of the mean value. However, since error distributions are unknown when predicting microbial growth, this term must be used cautiously, particularly in nonlinear regression models [33]. Hence, RMSE values were used for further comparisons of model performance. Under aerobic conditions, the ANFIS model showed better RMSE values (0.001 for the training data set and 0.002 for the testing data set) compared to the ANN model (0.019 for training data set and 0.067 for the testing data set) ( Table 5). Under anaerobic conditions, the ANFIS model also showed better RMSE values (0.002 for the training data set and 0.001 for the testing data set) compared to the ANN model (0.008 for training data set and 0.053 for the testing data set) ( Table 6). The RMSE, which is calculated by comparing desired and actual output values, which are then averaged across all data, index provides an estimate of goodness of fit in statistical models and indicates the long-term consistency of a model [33]. The lower RMSE values in the ANFIS model in comparison with the ANN model confirmed its superior accuracy in predictions made without previous training.

Discussion
Like A f , MAPE indicates the average deviation from the observed value [26]. In this study, the MAPE results were in good agreement with the A f values estimated for all data sets. In the training data set under aerobic conditions, for instance, the A f values obtained by the ANN and ANFIS models were 1.03 and 1.00, respectively (Table 5), and their average deviations in predicted and actual growth rates were 3% and 0%, respectively. These values were highly consistent with the MAPE values of 3.06% and 0.18% obtained by the ANN and ANFIS models, respectively, in the training data set. Under aerobic conditions, the ANN and ANFIS models obtained A f values of 1.17 and 1.00, respectively (Table 5), and their average deviations in predicted and actual growth rates were 17% and 0%, respectively. Again, these values were closely approximated the MAPE values of 15.69% and 0.27% obtained by the ANN and ANFIS models, respectively, in the testing data set. The relevant figures again confirmed the better performance of the ANFIS models.
The accuracy factor is similar to the B f statistic, which was also introduced by Ross [26]. In this case, a B f value greater than 1 indicates that the model overestimates growth rate and is thus a 'fail-dangerous' model whereas a value less than 1 indicates that the model underestimates growth rate and is thus a 'fail-safe' model. For example, the accuracy of the growth rate estimates was confirmed by B f values of 1.00 and 1.00 under aerobic and anaerobic conditions, respectively, in the ANFIS model and by B f values of 0.99 and 1.00 under aerobic and anaerobic conditions, respectively, in the ANN models (Tables 5 and 6). These data indicate that the ANFIS model provides a relatively more accurate estimate of growth rate compared to the ANN model, which tends to underestimate growth rate.
The SEP index indicates the relative deviation in mean prediction values. A notable advantage of the index is that its calculation is independent of the magnitude of the measurements [10]. For both training and testing data sets (Tables 5 and 6), the SEP indices were again better in the ANFIS models (range, 0.44-0.81) than in the ANN models (range, 3.66-23.27). Tables 11 and 12 show the evaluation criteria for ANFIS and ANN models with 10-fold cross-validation of the 30-bootstrap data set. The tables also show that the ANFIS models were better than the ANN models in predicting the four kinetic parameters.
The ANFIS and ANN models offer an interesting option for defining the sensitivity and relative importance of inputs ( Figures 10, 11, 12 and 13). The sensitivity analysis in this experiment revealed the important effects of temperature on LM growth rates under aerobic and anaerobic conditions. In earlier works, Zurera-Cosano et al. [9] obtained similar results in RSM models of LM growth whereas Panagou and Kodogiannis [7] reported similar results in an ANN model of Monascus ruber growth.
In conclusion, this study confirmed that, compared to ANN models, ANFIS architectures provide better accuracy in predicting the growth rate of LM based on input data for temperature, pH, NaCl and NaNO 2 under both aerobic and anaerobic conditions. The statistical indices and graphic plots confirmed the superior performance of the ANFIS models in both training and testing data sets. Sensitivity analyses of the ANFIS models revealed that the most influential (sensitive) factors in the growth rate of LM were temperature and, to a lesser extent, NaCl. As ANFIS is mainly applied in predictive microbiology, the findings that ANFIS models are effective for predicting the kinetic parameters of fungi indicate their good potential use as an alternative to ANNs in this field. Future studies of differential evolution [34] in the ANFIS architecture may evaluate the effects of training on nonlinear parameters of the premise part and on linear parameters of the consequent part. Hopefully, improving the accuracy of the model will enhance its potential applications in predictive microbiology.