3.3.1 Data preprocessing.

Data preprocessing is a process aimed at converting redundant, missing, excessive, and inconsistent data into a form amenable for model training. This phase involves several steps to improve the usability of the raw data and boost model efficiency. Python libraries such as NLTK and Keras have been utilized to aid in preprocessing tasks like stopword elimination, converting text to lowercase, and tokenization.

3.3.2 Word embedding techniques.

In natural language processing, one important step is to convert text data into vectors so that machine learning and deep learning models can be trained. Word embedding methods have gained popularity in recent years for their use in predictive models. In a study on identifying fake news, three word embedding techniques were used: fastText, fastText-subword, and GloVe. These techniques play a crucial role in the process of natural language processing.

3.3.3 fastText.

Word embedding in vector form has been applied to different tasks within natural language processing (NLP). In general, word embedding that has been pre-trained on sources like Wikipedia and Google News predicts word context without supervision. They believe that the words are in proximity and share similar contexts [27, 28]. The fastText method was developed by Facebook’s FAIR Research Lab. FastText should be considered for vector representation because it has a word difficulty detection tool. Moreover, the system uses morphological information and vectorization from 2 million words, with 600 billion tokens in 300 dimensions. This ability allows for enhancing text classification results by effectively generalizing them. The fastText word embedding vectors are created by adding n-grams together, allowing for the creation of vectors for words that are not well-known.

3.3.4 fastText subword.

The use of fastText subwords in training benefits from the sharing of common word roots. With 300 dimensions and 2 million word vectors (600 billion tokens), it incorporates subword information during training [29, 30]. Millions of vectors trained on common tokens or roots make up the fastText subword. It offers support by dissecting words into smaller parts or combining smaller parts into a single word. For example, “for” and “give” can be expressed independently and then joined to obtain dictionary-level representation by using the combination “forgive”. Character-level embedding is beneficial for accurately representing misspelled and slang terms.

3.3.5 GloVe.

This word embedding technique creates vectors using an unsupervised learning algorithm. GloVe contains 840 billion crawl words and has 300 dimensions. As its name suggests, It records the corpus’s global and local characteristics [31, 32]. It is context-independent and organizes words based on their semantic similarity, displaying the linear substructure of the word vector space. The worldwide lexicon of word co-occurrence is used to train it. GloVe is a logarithmic bilinear model that makes inferences about the meaning of words based on how likely it is that they will appear together. The factorization method creates a word context matrix.

3.3.6 Combining features.

This research work combines three popular word-embedding techniques(fastText, fastText-subword, and GloVe). These word-embedding techniques are independently taught to assist the DL models in improving predictive results. A DL model is trained using three word-embedding approaches separately at first, and subsequently by combining them at a ratio of 34% for the fastText subword and 33% for fastText and Glove. The illustration of combining features is shown in Fig 1. In the diagram, Red circles (o) represent the FastText embeddings, Green squares (s) represent the FastText-Subword embeddings, Blue triangles (∧) represent the GloVe embeddings, and Black crosses (x) represent the combined embeddings. The steps that are involved in combining features are:

1. Load pre-trained embeddings: Ensure all embeddings are loaded correctly.
2. Normalize embeddings: Make sure all embeddings have the same dimensions.
3. Combine embeddings: Compute the weighted average of the embeddings for each word.
4. Handle missing words: Provide a default embedding for words not present in all embeddings.
5. Save the combined embeddings: Store the final embeddings for future use.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Illustration of word embedding fusion.

https://doi.org/10.1371/journal.pone.0314174.g001

combined_embeddings = {}
alpha, beta, gamma = 0.33, 0.33, 0.34

For word in fasttext_embeddings:
    if word in fasttext_subword_embeddings and word in glove_embeddings:
        combined_embedding = (alpha * fasttext_embeddings[word] +
                              beta * fasttext_subword_embeddings[word] +
                              gamma * glove_embeddings[word])
        combined_embeddings[word] = combined_embedding
    # Handle words not present in all embeddings
    elif word in fasttext_subword_embeddings:
        combined_embedding = (alpha * fasttext_embeddings[word] +
                              beta * fasttext_subword_embeddings[word] +
                              gamma * np.zeros(embedding_dim))
        combined_embeddings[word] = combined_embedding
    elif word in glove_embeddings:
        combined_embedding = (alpha * fasttext_embeddings[word] +
                              beta * np.zeros(embedding_dim) +
                              gamma * glove_embeddings[word])
        combined_embeddings[word] = combined_embedding
    else:
        combined_embedding = alpha * fasttext_embeddings[word] + beta
        * np.zeros(embedding_dim) + gamma * np.zeros(embedding_dim)
        combined_embeddings[word] = combined_embedding

# Repeat similar steps for words in fasttext_subword_embeddings
and glove_embeddings that were not in fasttext_embeddings

3.3.7 Dimensionality reduction.

Using various word embedding methods together results in a higher quantity of features and repetitions, which adds extra workload to the classifier training process. Using a suitable method for selecting features can address this issue [33]. PCA is a popular tool for classification jobs, as it reduces the number of characteristics by using a linear transformation. Following PCA reduction, the dataset contains features that were present in the original data, and the covariance matrix is used to determine the principal component. Using a pool of 3400 features, PCA is used in this study to select significantly important 1500 features for the experiments.

3.3.8 Modeling methods.

Big data presents a variety of computational issues that CNN overcomes by using nonlinear-activation, dropout, pooling-layers, and filters [34, 35] to obtain complex information [36, 37]. This model training is more effective because it is completed end-to-end. Semantic information is encoded at the end of the model using fully connected layers. It is a network that advances by filtering the layer’s output before identifying features. Convolutional layers, pooling layers, a flattened-layer, dropout, activation functions, and the dense layer are the main components of the CNN model. Convolutional layers first extract features, which are then sent to fully connected layers for further analysis. To minimize overfitting risks, the number of features detected by convolutional layers is curtailed by the pooling layer. Pooling can be executed using either a max or average layer; with max-pooling selecting different features compared to average pooling. The flattened layer converts the data into an array prior to its passage to the fully connected layer. The rectified-linear-unit(ReLU) activation function is utilized in model development: (1)

‘y’ is the output activated, while ‘i’ is the input given. Convolutional layers are utilized to find the patterns in the dataset with the help of weights applied to kernels during training. Now 1D-CNN model shows great results with text classifications as in our case. The binary cross-entropy is utilized to calculate loss during training. Its formula is: (2)

‘i’ denotes class labels, log refers to the natural log, and p signifies the predicted probability. Given that CNN is a variant of the back propagation technique, the output’s error function is sigmoid [38]. The CNN model produces three neurons as output for every instance of the target class. CNN is considered a strong model for classifying tasks within the medical domain. Many researchers have made use of CNN for different classification purposes like identifying lung diseases, segmenting brain tumors, and analyzing chest X-rays. Prior research has also looked at CNN for applications like text summarization, text sentiment analysis, and text report classification. CNN is being used to identify eye diseases that could potentially lead to vision loss through analysis of medical records. The proposed methodology for recognizing Fake news is depicted in Fig 2. The optimized CNN model in this research project has been tailored in terms of layer count, neuron count, and choice of optimizer. Table 3 contains specifics of the CNN architecture. Besides CNN, other popular ML models like RF, SGD, LR, and LSTM have also been utilized for comparison in this process. Fine-tuning various hyperparameters optimizes the performance of these models, with Table 3 providing a list of the specific parameters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Proposed fake news stance detection architecture.

https://doi.org/10.1371/journal.pone.0314174.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. The learning models’ layer structure and hyperparameters.

https://doi.org/10.1371/journal.pone.0314174.t003

3.4.1 Random Forest(RF).

RF classifier is one of the most promising ML techniques for detecting fake news [39]. To improve the model’s overall classification accuracy and robustness, a lot of decision trees are combined using an ensemble learning technique called RF. Then, the RF algorithm creates multiple versions of decision trees from an individual subset of features on a random subset of input training data. For predicting, these trees will make their predictions at the final node. The final prediction in such an ensemble method would be determined by taking a majority vote or averaging out the projections of the individual trees. Several advantages can be derived from using RF for FND. First, RF’s ensemble nature will enable it to identify intricate non-linear correlations between the target variable and the input characteristics, if characteristics of fake news are not easily identified through some simple rules or linear models. Second, it gives a measure of feature importance that helps find the most discriminative characteristics to ascertain differentiation between genuine and fake news, hence helping in further feature engineering and model interpretability [40]. Besides, RF trains many decision trees on random subsets of the data, so this shall be less prone to overfitting a single decision tree model and thus generalizes the model’s performance on unseen data. Moreover, RF can operate with a wide variety of data: numerical, categorical, and textual features. Thus, it becomes versatile with the wide range of inputs usually found when performing tests to detect fake news. To classify news stories as authentic or fake, performance has been excellent while using an RF model with features that reflect textual, metadata, and context information.

3.4.2 Logistic Regression(LR).

The other prominent ML algorithm within FND is logistic regression. LR can be conceptualized as a supervised learning approach, specially adapted for binary classification problems, such as classifying news stories as fake or genuine [41]. The Logistic Regression model estimates the probability of events based on input variables, predicting whether a news item is real or fake in this case. It learns coefficients that characterize the relationship of input variables with the target variable and uses these to make predictions about class real or fake on new unseen data. Some key reasons logistic regression can be used in FND involve its interpretable fashion since the coefficients of the model provide information on both the strength and direction of the association between the target variable and the input features, its ability to accommodate a wide range of features that include numerical, categorical, or even textual data, and lastly, probabilistic output, which might help set up threshold or confidence levels for appropriate classification. In addition, logistic regression models are typically resilient to highly connected cases of overfitting; they can also be huge with large datasets [42]. Hence, they can be used in real-world applications which involve FND. Several researchers used Logistic Regression models trained on mixed features, including linguistic characteristics, source credibility, and social engagement patterns, and obtained promising results in grading news articles as real or fake. However, the thing to note here is that logistic regression can only be practical to a certain extent it might miss out on many complex, nonlinear relationships within the data in such scenarios, more advanced ML algorithms would draw a line of demarcation between nuanced patterns defining real news versus fake news.

3.4.3 Stochastic Gradient Descent (SGD).

Besides the ML algorithms discussed above, optimization techniques such as SGD have also been used in FND [43]. SGD is one of the most popular optimization techniques used to tackle complex ML issues and has been utilized for phony news detection tasks, too. In contrast to stochastic gradient descent, which runs the optimization process based on one or a small subset of training examples at a time, batch gradient descent updates model parameters based on the entire training dataset. Among the advantages that SGD brings to FND is computational efficiency, as SGD requires fewer computations per iteration compared to batch gradient descent, especially when dealing with large datasets. It can also process streaming data, so it is pretty fit for FND systems in real-time. It is usually more robust to noisy or imbalanced training data, which often is the case in the context of FND. Another merit is that SGD is relatively simple and very easy to implement thus, it can be used within a wide range of machine-learning frameworks and architectures [44]. The researchers applied SGD-based optimization techniques to train a classifier against logistic regression and neural network models aimed at drawing almost imperceptible lines of distinction between genuine and fake news articles. Using SGD, models gracefully update their parameters against the evolving nature of fake news; this is important in maintaining accurate and up-to-date FND abilities. Performance with SGD-based methods may have a high sensitivity to such hyperparameters as the learning rate and batch size this requires careful tuning and monitoring of hyperparameters to enable the stability and convergence of the optimization process.

3.4.4 LSTM.

Another interesting method for detecting fake news is the use of LSTM networks, a kind of RNN that excels at processing sequential input, like text [45]. Traditional RNN models could suffer from the vanishing gradient in the event of long-range dependencies. By contrast, LSTMs were designed to capture and memorize relevant information over more extended periods, thus able to model complex language patterns and contextual cues typical in news articles. In the domain of FND, LSTM models can be trained on textual content, metadata, and social engagement features of news articles to learn distinctive linguistic and behavioral features that distinguish real from fake news. The ability to model long-range dependencies allows LSTM-based models to learn complex interactions among parts of a news article and consider the general context in which it is shared and consumed. In the former case, the recurrent nature of LSTM models allows them for application in real-time or streaming scenarios, let through continuous monitoring and detection of rising fake news [46]. The methods that involved LSTMs showed an accuracy in opposition to traditional ML approaches, such as LR and SVM in classifying news articles into either natural or fake. The success of the LSTM network in the domain brings a message related to the use of advanced DL techniques to solve this new, complex, and evolving problem of FND.

4.1.1 Experiments using fastText.

The experimental results using all the learning models with fastText word-embedding are shown in Table 4. The CNN model outperforms the others with an accuracy of 90%, and it also shows strong precision (88%), recall (86%), and F1 score (87%). The LSTM model, while having a good accuracy of 87%, lags in precision (71%) and recall (75%), resulting in a lower F1 score of 73%. The RF model shows a slightly higher accuracy of 87%, with balanced precision (80%) and recall (87%), and an F1 score of 82%. Both the SGD and LR models have similar performances with accuracy close to 88% and identical precision (77%), recall (88%), and F1 scores (82%). The VC model, which combines LR and SGD, has an accuracy of 87.61% and maintains the same precision (77%), recall (88%), and F1 score (82%) as the individual SGD and LR models. Overall, CNN stands out as the best-performing model in terms of overall metrics.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Results of classifier classification using fastText.

https://doi.org/10.1371/journal.pone.0314174.t004

4.1.2 Experiments using fastText subword.

Table 5 provides a comparative analysis of the performance of different ML models using fastText with subword information. The CNN model shows an accuracy of 88%, 86% precision, 83% recall, and F1-score of 84%, indicating strong and balanced performance. Similarly, the LSTM model also achieves an accuracy of 88%, with a slightly lower precision of 85%, 81% recall, and 82% of F1-score. The RF model matches the accuracy of 88%, 83% precision, 88% high recall, and F1 score at 84%, showcasing its robustness in recall. The SGD and LR models both achieve an accuracy of 88%, 77% precision, 88% recall, and F1-scores of 83% and 82%, respectively. Lastly, the VC model, combining LR and SGD, has an accuracy of 87%, 77% precision, 87% recall, and 82% F1-score reflecting a slightly lower performance in accuracy while maintaining balanced precision and recall.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Classifier classification outcomes using fastText-subword.

https://doi.org/10.1371/journal.pone.0314174.t005

4.2.1 Experimental results using combined word-embedding features.

Finally, to train the models, With proportions of 33%, 34%, and 33%, respectively, the three popular word-embedding techniques (fastText, fastText-subword, and Glove) features are merged. Next, important features are extracted by reducing dimensions through the use of PCA. It is noticeable that the performance of models is greatly enhanced by combining various word embedding techniques, as shown in Table 7. The CNN model exhibits exceptional performance with the highest accuracy of 94.58%, precision of 95.35%, recall of 97.29%, and F1 score of 96.11%, making it the top performer across all metrics. The LR model achieves an accuracy of 92.35%, with a precision of 91.11%, recall of 89.11%, and an F1 score of 90.11%, showing strong but slightly lower performance compared to CNN. The LSTM model has an accuracy of 90.35%, with a high precision of 92.47%, recall of 94.32%, and an F1 score of 93.39%, indicating very good performance, particularly in recall and F1 score. The RF model records an accuracy of 88.98%, precision of 90.11%, recall of 93.11%, and an F1 score of 91.16%, showing good overall performance with a focus on recall. The SGD model achieves an accuracy of 89.24%, with a precision and recall both at 92.11% and an F1 score of 92.11%, indicating balanced performance across metrics. Lastly, the VC model, combining LR and SGD, performs strongly with an accuracy of 93.65%, precision of 93.11%, recall of 94.11%, and an F1 score of 93.16%, showing robust and balanced performance, slightly outperforming individual LR and SGD models. The comparison of all learning models using combined features is shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Comparison of all learning models using combined features.

https://doi.org/10.1371/journal.pone.0314174.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Experimental results of all learning models using the combined word-embedding features.

https://doi.org/10.1371/journal.pone.0314174.t007

4.2.2 Comparison of performance of State-of-the-Art studies.

An analysis of performance is conducted in comparison to current research to show the importance of the proposed method. Teams that participated in the competition have been chosen for this specific reason. The comparative outcomes presented in Table 8 demonstrate that the proposed framework, which employs the CNN model with blended characteristics, performs better at identifying fake news than the most advanced algorithms.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Comparison with previously published studies listed in research work [51].

https://doi.org/10.1371/journal.pone.0314174.t008