Investigation of ANN architecture for predicting shear strength of fiber reinforcement bars concrete beams

In this paper, an extensive simulation program is conducted to find out the optimal ANN model to predict the shear strength of fiber-reinforced polymer (FRP) concrete beams containing both flexural and shear reinforcements. For acquiring this purpose, an experimental database containing 125 samples is collected from the literature and used to find the best architecture of ANN. In this database, the input variables consist of 9 inputs, such as the ratio of the beam width, the effective depth, the shear span to the effective depth, the compressive strength of concrete, the longitudinal FRP reinforcement ratio, the modulus of elasticity of longitudinal FRP reinforcement, the FRP shear reinforcement ratio, the tensile strength of FRP shear reinforcement, the modulus of elasticity of FRP shear reinforcement. Thereafter, the selection of the appropriate architecture of ANN model is performed and evaluated by common statistical measurements. The results show that the optimal ANN model is a highly efficient predictor of the shear strength of FRP concrete beams with a maximum R2 value of 0.9634 on the training part and an R2 of 0.9577 on the testing part, using the best architecture. In addition, a sensitivity analysis using the optimal ANN model over 500 Monte Carlo simulations is performed to interpret the influence of reinforcement type on the stability and accuracy of ANN model in predicting shear strength. The results of this investigation could facilitate and enhance the use of ANN model in different real-world problems in the field of civil engineering.


Introduction
In aggressive environments, the load-bearing capacity of reinforcing bars in the concrete structure can be seriously declined due to steel corrosion. Accordingly, the performance of reinforced concrete structures could be reduced [1][2][3]. To prevent this phenomenon, numerous solutions have been proposed to avoid steel corrosion in reinforced concrete structures, such as increase the concrete cover layer to protect the reinforcements, the use of high-performance concrete (HPC), or waterproof paint [4]. However, these solutions lead to an increase in cost structures. Some investigations have shown that the corrosion resistance of reinforced concrete could be increased by using fiber-reinforced polymer (FRP) bars to replace traditional steel [3,[5][6][7]. Various types of FRP could be used, such as non-metallic glass fiber-reinforced polymers (GFRP), carbon fiber-reinforced polymer (CFRP), basalt fiber-reinforced polymer (BFRP), or aramid fiber-reinforced polymer (AFRP). Indeed, FRP bars have many advantages, such as good mechanical properties, small self-weight, easy transportation, installation, nonconductive, non-magnetic, and low heat conductivity [3,[8][9][10][11][12]. Therefore, FRP bars have been proposed as longitudinal and shear reinforcements for different types of concrete structures exposed to various aggressive environments, mostly sewage water, seawater. . .[9, [13][14][15]. Numerous experimental investigations have shown that the general flexural theory of reinforced concrete structures is also valid for FRP reinforced concrete beams [16]. However, the material properties of FRP are significantly different compared with that of steel reinforcement. Especially, the elastic modulus of FRP is lower than that of steel reinforcement, so that the shear behaviors of FRP bar-reinforced concrete beams such as shear strength, deformation, and crack width are different from those of traditional steel reinforcement concrete [17,18]. Moreover, the FRP bars have only linear elastic behavior without any plasticity until failure [19]. In addition, the general flexural theory of reinforced concrete structures is moderately applied for concrete beams using FRP as flexural reinforcement [20,21], but using FRP as shear reinforcement raise the complexity of mechanism behavior of concrete beams. Therefore, it is difficult to apply the existing shear strength prediction models of reinforced concrete beams to estimate FRP reinforced concrete beams. Until now, many theoretical and experimental studies have been carried out to predict the shear strength of FRP reinforced concrete beams [3, 16,[22][23][24][25]. Most investigations have tried to come up with simple predictive equations based on different shear mechanisms to facilitate the FRP bars design in concrete structures [2,26]. However, the accuracy of these predictive equations seems to be limited. In fact, these equations are empirically developed by different experimental results, where each one is performed for a specific case study, such as varying the geometrical variables of beams or investigating the feasibility of using one specific type of reinforcement. Therefore, these equations could hardly be universal for predicting the shear strength of different FRP reinforced concrete beams [27]. In 48], and earth sciences [49][50][51][52][53] have been favorably resolved by applying ANN. Using machine learning techniques, the shear strength prediction of FRP concrete beams has also been the subject of some investigations in the literature. Nehdi et al. [2] have used a genetic algorithm approach and 168 experimental results to propose shear strength design equations for FRP reinforced concrete beams. In the investigation of Abbasloo et al. [54], a rule-based method has been used to predict the shear strength of FRP reinforced concrete without shear reinforcement. The authors have collected 176 experimental results from the literature and show the robustness of the machine learning approach against the shear strength design equations in both aspects of accuracy and reliability. These investigations indicate that machine learning algorithms are powerful numerical tools that can resolve numerous complex relationships between different components and optimize them to acquire the targeted mechanical properties, such as the shear strength of FRP reinforced concrete. Therefore, the primary purpose of this investigation is to propose an efficient model to increase the shear strength prediction accuracy of FRP concrete beams.
In this work, the performance of ANN model is investigated to predict the shear strength of FRP reinforced concrete. Although the most effective machine learning algorithm is ANN model, its performance depends strongly on the selection of ANN architecture. Therefore, this investigation simultaneously performs the determination and optimization of the ANN architecture for better prediction of the shear strength of FRP reinforced concrete beams. To achieve this goal, numerous experimental data from the literature are collected and randomly divided into two parts: the training part (70% of data) and the testing part (30% of data). Moreover, Monte Carlo simulations (MCS) are also performed to verify the convergence and feasibility of the proposed model. Thanks to Monte Carlo simulations, a minimal number of simulations that ensure the reliability of prediction results are determined. The best ANN architecture is derived and used to predict the shear strength of FRP reinforced concrete beams with the help of three statistical measurements, including the Coefficient of Determination (R 2 ), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE). Finally, the prediction capability of the best ANN architecture is investigated in function of different types of FRP reinforcements.

Database construction
In this study, 125 experimental results have been collected from 15 experimental works [16,[55][56][57][58][59][60][61][62][63][64][65][66][67][68] to construct the database (Table 1). Summary and source information for the database are also summarized, including the amount of data and the corresponding percentage. It is worth noticing that all the research works include FRP concrete beams with both flexural and shear reinforcement. Based on the above database, each sample consists of a vector of dimension 1 × 9, including three groups of inputs. The first group consists of beams' characteristics, such as beam width, effective depth, the ratio of the shear span to the effective depth, and compressive strength of concrete. The second group consists of flexural reinforcement parameters such as longitudinal FRP reinforcement ratio and modulus of elasticity of longitudinal FRP reinforcement. The last group consists of shear reinforcement parameters, namely the FRP shear reinforcement ratio, the tensile strength of FRP shear reinforcement, and modulus of elasticity of FRP shear reinforcement. The output vector is of dimension 1 × 1 and consists of the value of the shear strength of FRP beams with flexural and shear reinforcement. Statistical information of the input and output variables used in this study are presented in Table 2, including the mean values, along with the minimum, maximum, standard deviation (StD), median, skewness values (StD).
In the presented database, concrete beams with four types of FRP reinforcement are used, namely carbon fiber reinforced polymer (CFRP), glass fiber reinforced polymers (GFRP), aramid fiber-reinforced polymer (AFRP), and vinyl fiber-reinforced polymer (VFRP). Besides, 27 samples also use steel as flexural reinforcement. A summary of the reinforcement type used in this study is presented in Table 3 in separating samples with flexural reinforcement, shear reinforcement, and a combination of flexural reinforcement and shear reinforcement. Finally, the beam test diagram is illustrated in Fig 1.

Artificial neural network
An artificial neural network (ANN) is an information processing model simulated according to the information processing way of biological neuron systems. It is made up of a large number of neurons that are interconnected through connections to solve a particular problem. The ANN network structure is made up of three or more layers, depicted in Fig 2, consisting of (i) an input layer that is the leftmost layer of the network, representing the input parameters, (ii) an output layer is the rightmost layer of the network, representing the results achieved, and (iii) one or more hidden layers denoting the logical inference of the network. The input layer contains the information x i (i = 1,2,.., n) from the original data. The values x i are then multiplied by a weighted value w 1 jk (j = 1,2,. . ., N) where k = 1,2,.., n. The value S n k¼1 w 1 jk x k þ y 1 j becomes the input of the hidden layer, and continues to be multiplied by another weighted value w 2 ij (i = 1,2,. . ., m), becoming the input of the output layer. The values y 1 j represent the bias associated with each neuron in the hidden layer.

PLOS ONE
Predicting shear strength of fiber reinforcement bars concrete beams Finally, the output value is obtained as S L j¼1 w 2 ij ðS n k¼1 w 1 jk x k þ y 1 j Þ þ y 2 i . The learning process of the neural network corresponds the process in which the network continuously changes the values w 1 jk and w 2 ij in order to change the output value y i , until the y i value meets the requirements (to reach a certain error compared with the target output). The back-propagation algorithm (BPNN) is widely used to train the neural network [69]. The BP algorithm is a slope reduction technique that minimizes errors for a particular training pattern in adjusting the weight by a small amount each time (or each epoch, iteration). Nerve cells in each layer are linked to the front and rear nerve cells with each associated weight, as presented above. However, the standard backpropagation algorithm typically has a slow convergence rate [70]. One of the generalization-type methods, such as the Bayesian Regularization (BR) [71,72] is commonly used to avoid such a convergence problem and obtain a lower mean square error between the output of ANN and the target [73].
Bayesian regularization is a network training function that updates the weight and bias values according to Levenberg-Marquardt optimization. It minimizes a combination of squared errors and weights, then determines the correct combination to produce a network with high prediction accuracy. From a Bayesian point of view, the regularization corresponds to a prior probability distribution over free parameters w of the model. Using the notation of MacKay [71], the regularized target function can be written as: where E D is the sum of squared errors, E w is the sum of square of the network weights, α and β are target function parameters. In ANN algorithm, the weights are considered random variables, and therefore the density function is written according to Baye's rules [74] as follow: where w is the vector of network weights, D is the data vector, and M is the neural network model. The optimization of regularization parameters α and β demands for solving the Hessian matrix of F(w) at the minimum point wMP. This technique can reduce the potential to arrive at a local minimum, thus increasing the network's generalizability.

Monte Carlo simulation
Numerical prediction models involving Monte Carlo simulation can explain the output results' variation through statistical analysis. The Monte Carlo method is potent to calculate the influence of the input variability on the output results using numerical AI models [40,44,75,76].
In this study, the objective of the Monte Carlo method is to randomly repeat simulations, taking into account the variability in the input space, then calculate the corresponding output through a machine learning model [77]. The robustness of Monte Carlo simulation and sensitivity of input variables can be evaluated through statistical performance criteria of the output results. The statistical convergence of Monte Carlo simulation has been carried out using the normalized convergence criteria as follow [78][79][80]: where Y is the mean value of the considered variable Y, and S is the number of Monte Carlo simulations. For statistical analysis, the convergence function provides efficient information of the computational time and reliability of the predicted results.

Performance criteria
To evaluate the effectiveness of the proposed ANN model, several evaluation criteria are proposed, including mean absolute error (MAE), root mean square error (RMSE), and the coefficient of determination (R 2 ). Precisely, MAE represents the average amplitude of the model error but does not indicate the biased trend of the model output and actual values. When MAE = 0, the model value completely coincides with the actual value, thus the model could be considered as "ideal". The values of MAE are in the range of (0; +1). Besides, root mean square error (RMSE) is one of the fundamental criteria and is commonly used to evaluate the predictive modeling results. It is common to use the RMSE to denote the mean magnitude of the error. In particular, RMSE is very sensitive to large error values. Therefore, the closer the RMSE is to the MAE, the more stable the model error. Criteria MAE, RMSE do not indicate the deviation between the model's output value and the actual value, and in the range of (0; +1). R 2 is the coefficient of determination that represents the suitability of the data with the algorithm, and in the range of (0; 1). The R 2 values close to 0 present the model's poor performance, whereas the values close to 1 show good model accuracy. These values are represented by the following equations: where N is the number of samples, M is the target value, and Q is the measured value.

Methodology flow chart
In this work, the methodology in the modeling process of the shear resistance of FRP beams includes four main steps, namely: Step 1: Data preparation for training the ANN model. In this step, a database of 125 experimental results is collected for the purpose of predicting the shear strength of FRP concrete beams with flexural and shear reinforcement. ANN model is built with 9 input parameters, as stated in the previous section. The data set is randomly divided into two parts: the training data set consisting of 70% of data (88 samples) and used to train the ANN model, and the testing data set consisting of the remaining 30% data (37 samples) to validate the trained ANN model.
Step 2: Train the model. In this step, BR algorithm is used to train and select the optimal ANN weights and bias associated with each neuron in the architecture. A training data set of 88 test samples, randomly taken from the database, were used for this purpose.
Step 3: Model evaluation. In this step, the trained ANN model is evaluated using the testing data set, including the remaining 37 test samples. The performance of the model is assessed by three statistical criteria, namely R 2 , RMSE, and MAE.
Step 4: Sensitivity analysis. This process is performed thanks to Monte Carlo simulation with the aim of evaluating the effect of each type of FRP bar on the performance of the ANN model in predicting the shear strength of FRP beams.
In summary, a schematic diagram of the methodology is shown in Fig 3. Table 4 summarizes the ANN parameters used in this study, and related information on Monte Carlo simulations. For the ANN and Monte Carlo simulation modeling, MATLAB programming language is used. Nothing that two cases of the hidden layer, consisting of 1 hidden layer and 2 hidden layers, are considered in this study. The number of neurons in each hidden layer is varied, ranging from 10 to 20 neurons, in order to cover the range of neuron as suggested in previously published works, such as in Neville (1986) [81], Gallant (1993) [82], Nagendra (1998) [83], Kannellopoulas (1997) [84]. Under the random sampling effect, the convergence of ANN model was investigated with a total of 66,000 simulations, including 121 architectures for the case of 2 hidden layers and 11 architectures for those with one hidden layer. Investigation on the convergence of results is crucial in Monte Carlo simulation, aiming at determining: (i) the appropriate number of Monte Carlo simulations, and (ii) the reliability of the simulation results. Fig 4 shows the convergence of results for all ANN architectures performed in this study. It is observed that the convergence is assured for both training, testing datasets for all cases over 500 simulations (Fig 4). Fig 4A and 4B show the normalized convergence results of R 2 . It can be seen that a fluctuation of 1% around the mean value is obtained after about 300 simulations, whereas with the same number of Monte Carlo simulations, RMSE and MAE vary about 2% of the mean values (Fig 4C-4F). Overall, the results are stable around the average values, normalized to 1 when the number of Monte Carlo simulations is 500. Thus, it can be stated that the reliable results obtained by the proposed ANN model with 1 and 2 hidden layers are converged after 500 simulations, under the random sampling effect of data. In the next step, the optimization process of different ANN architectures is performed.

ANN architectures with one hidden layer.
In this section, the performance of all the ANN architectures is evaluated via statistical criteria such as the mean and standard deviation values of R 2 , RMSE, MAE. The evaluation is carried out for both the training and testing parts. Fig 5 shows the performance of ANN in function of the neuron in the hidden layer,  varying from 10 to 20, regarding the mean and StD values of R 2 , RMSE, and MAE for the training and testing parts. It can be seen that the case of 17 neurons exhibits the best prediction results. In fact, the values of R 2 are the highest compared with the other 11 cases, with the values of 0.975 and 0.8886 for the training and testing parts, respectively (cf. Fig 5A and 5B). Similarly, the values of RMSE and MAE for this architecture are also lower compared with the remaining cases, along with better standard deviation values (cf. Fig 5C-5F). Precisely, the average values of RMSE are 13.38 and 33.77 for the training, testing datasets, respectively. The average values of MAE are 9.75 and 24.84 for the training, testing datasets, respectively. It is worth noticing that the testing part's performance is critical when evaluating the prediction capability of a model. The criteria associated with the testing dataset reflect the ability of a model to predict new data, which are not considered in the training phase of a given model. Overall, 17 neurons in the hidden layer generate the best ANN architecture with one hidden layer.

ANN architectures containing two hidden layers.
Concerning the ANN architecture with 2 hidden layers, various structures are investigated, as shown in Figs 6 and 7 for both the training and testing datasets, respectively. With respect to the training dataset, it is observed that a higher number of neurons in the second hidden layer produces higher prediction accuracy (i.e., R 2 of 0.95, RMSE of 14, and MAE of 12). An optimal zone is observed to achieve the highest accuracy, regardless of the number of neurons in the first hidden layer. This is an excellent example to demonstrate that an appropriate ANN structure should be determined before performing any further simulations. Investigation of the effect of neurons in the two hidden layers is performed via standard deviation plots of the three error criteria (Fig 6B, 6D and 6F). The training part exhibited small standard deviation values, in general, with a higher number of neurons in both hidden layers (i.e., above 15 neurons).
Similarly, the testing part results are in good agreement with those of the training part, where the higher number of neurons in the second hidden layer is required to obtain better prediction accuracy (cf. Fig 7). However, it is clearly observed that the prediction accuracy of ANN structure with 2 hidden layers is lower than that containing 1 hidden layer. Indeed, the highest values of R 2 are 0.7696 and 0.8886 for the cases of ANN using 2 and 1 hidden layer, respectively. Besides, the lowest values of RMSE and MAE are 51.60 and 36.50 for ANN structure using 2 hidden layers, whereas those values are 33.77 and 24.84 for ANN using 1 hidden layer, respectively.
Overall, it can be stated that ANN structure with 1 hidden layer and 17 neurons is the most effective predictor for the problem, and thus, it can be used for further investigation.

Prediction performance of typical ANN architecture
This section is dedicated to the presentation of typical results related to the best ANN architecture containing 1 hidden layer with 17 neurons, where the R 2 value of the testing part is highest. Fig 8 shows    Finally, several previously published models and the corresponding accuracy are given for comparison purposes (cf. Table 5). It could be seen that the proposed ANN model in this study exhibits good accuracy compared with other works. The accuracy of the best model (R 2 = 0.960) is slightly inferior to the accuracy obtained by the Multivariate adaptive regression splines (MARS) model in the work of Abbasloo et al. [54] (R 2 = 0.974). However, the authors used a smaller amount of samples (112 data) than this work (125 data). This might be the reason for a small difference in prediction accuracy. Second, the present study uses a smaller number of inputs (9 inputs) than in the work of Abbasloo et al. [54] (12 inputs). Without affecting too much the prediction accuracy, the reduction of 3 inputs is not significantly pronounced with a small dataset, but might be of great interest while considering a bigger dataset or industrial design context.

Analysis of prediction accuracy based on reinforcement type
In this section, the prediction capability of the best ANN architecture (i.e., ANN-9-17-1) is analyzed considering different types of FRP reinforcements. The performance of the best ANN architecture in function of flexural reinforcement type is investigated by mean and StD values of R 2 (Fig 10A), mean and StD values of RMSE (Fig 10B), and mean and StD values of MAE ( Fig 10C) for 500 Monte Carlo simulations. In fact, the accuracy of ANN model in predicting the shear strength is strongly dependent on the types of flexural reinforcement, namely steel, AFRP, CFRP, and GFRP. It is observed that the shear strength prediction by the proposed ANN architecture for AFRP has the lowest accuracy comparing with other types. In this case, the mean value of R 2 is the lowest (i.e., 0.90), and those of RMSE (i.e., 24.40), MAE (i.e., 18.13) are the highest. Besides, the prediction accuracy remains similar compared with other cases of flexural reinforcement. It is worth noticing that the prediction accuracy of ANN for AFRP type flexural reinforcement is the lowest, even though the AFRP flexural reinforcement type possesses the highest number of samples (41 samples as indicated in Table 3). This might come from the possible interaction between both types of reinforcement (flexural and shear), or an inappropriate range of inputs and output compared with those of the full dataset. Thus, further investigation of this behavior should be considered. Fig 11 shows the performance of the best ANN architecture containing 17 neurons with 1 hidden layer in function of shear reinforcement type. Similar to the previous investigation, four types of reinforcement, including VFRP, AFRP, CFRP, and GFRP, significantly influence the ANN model accuracy in predicting the shear strength. The mean value of R 2 in predicting the shear strength for the case of VFRP is the lowest (Fig 11A). However, the mean values of RMSE and MAE are the smallest (cf. Fig 11B and 11C), and the StD values, in this case, are significantly higher compared with other types of shear reinforcement. It could be concluded that the proposed ANN model is sensitive in predicting such type of reinforcement, which might come from the insufficient number of samples (only 10 samples, see Table 3). Moreover, the mean value of R 2 in predicting the shear strength using AFRP, CFRP, and GFRP shear reinforcement is higher than that of VFRP, but the mean values are higher than those with AFRP, CFRP, and GFRP. In this case, it is difficult to evaluate the model's accuracy based on the mean values of these statistical measurements. However, based on the StD values of R 2 , RMSE, and MAE, it could be stated that the model accuracy is very sensitive in predicting VFRP as shear reinforcement.

Conclusion
In spite of numerous investigations on the shear strength prediction of FRP concrete beam, the performance of prediction could still be enhanced from more in-depth investigations. In this paper, the current investigation displays a simple but efficient model to use an ANN architecture for predicting the shear strength of FRP concrete beam with both flexural and shear reinforcements. Two cases of the hidden layer number containing various numbers of neuron are proposed to find out the best architecture of ANN model. After the performance analysis of different architectures, the best architecture of ANN model is proposed, where the robustness is supported by the presence of a random dataset splitting over 500 Monte Carlo simulations. Statistical results assessment of Monte Carlo simulations is derived to validate the