Dataset collection

For sentiment analysis, a dataset is extracted containing from Twitter. Twitter is used for data collection due to several factors. First, Facebook follows a friend-based network where only the people in the friend list can view the information on a profile page [39] which makes it private or closed. Hence, the data collection from Facebook to perform sentiment analysis is limited. Second, Twitter provides access to tweets with a ‘developer’ account and data collection is open and easy. Third, In addition, Twitter provides an application programming interface (API) that simplifies the task of data collection. Last and foremost is the length of tweets which is limited by 140 characters which makes it easy to perform analysis on short messages as compared to other social media platforms. A total of 8911 tweets from Twitter related to four hashtags including #facemask, #facemaskprecautions, #covid19, #covid19precautions by utilizing the Tweepy library. We collect this data during January 2022 during the peak period of the COVID-19 5th wave. The dataset contains three attributes including user ID, location, and tweet text. A few sample tweets from the dataset are shown in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Text from sample tweets.

https://doi.org/10.1371/journal.pone.0272350.t001

Data preprocessing

The collected text is further subjected to preprocessing as it includes unnecessary data such as hashtags, URLs, punctuation marks, et. We utilized a variety of processes using the NLTK (Natural Language Toolkit) to clean the data [40]. Tokenization is one of the foremost tasks as it breaks down a text into small chunks known as tokens. In this study, we implemented RegexpTokenizer in Python language on the text data to provide the computer with a particular number of tokens that it can process to provide the desired outcome.

The tweets are the text which also includes punctuation marks such as “!@#$%*()”. The punctuation characters are not significant for the sentiment of the tweet and thus are regarded as unnecessary. Therefore, we removed them to clean the data for the learning models to interpret the text data with more efficacy. The learning models require data that is relevant for the analysis such as a tweet containing the number “4” or “3” does not have any impact on the sentiment of the tweet. Accordingly, null values are the noise that leads to ambiguous results. Therefore, numeric values are removed to enhance the performance of learning models.

The extracted tweets contain a variety of hashtags along with the URL which does not contribute to the sentiment of the text. Therefore, the hashtags and URLs are removed from the tweets which result in less noise and more relevant data for the learning models Stop-words are the most recurring words in the text document and do not contribute to the learning of machine learning models. Therefore, we imported the stopwords library from the NLTK and removed the stopwords from the tweets. Similarly, the learning models are case sensitive, such as they consider the words ‘Good’ and ‘good’ as two different words. It increases the feature space and the computational complexity of the model. For text classification, case normalization helps to improve the performance of learning models.

Stemming and lemmatization are two techniques that involve the transformation of a word into its root form. Stemming works by removing the beginning or end of an inflected word by taking prefixes and suffixes into account. However, removing the suffixes and prefixes might sometimes not be successful. Therefore, to overcome this limitation of stemming, lemmatization has also been integrated into this study. Lemmatization helps to transform the extended words into their base form and thus reduces the complexity of feature space by reducing the number of unique words as extended forms of words are considered different words from machine learning classifiers. Lemmatization differs from stemming in terms of considering the morphological analysis of the inflicted words [41]. In this study, we utilized PorterStemmer and WordNetLemmatizer for stemming and lemmatization, respectively [41].

Table 2 shows the text from sample tweets before and after the above-mentioned preprocessing steps are carried out. It can be observed that numbers, punctuation, stopwords, URLs, and special symbols have been removed and the tweet text has been tokenized.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Sample tweets text before and after preprocessing steps.

https://doi.org/10.1371/journal.pone.0272350.t002

TextBlob

For a supervised sentiment analysis of tweets, labeled data is required. TextBlob is used to annotate the tweets related to COVID-19 safety precautions. TextBlob is a Python library that is widely used to process data. It includes API (Application Programming Interface) to perform several NLP (Natural Language Processing) tasks like phrase extraction, translation and sentiment analysis, etc. [42]. We categorize the data into three classes positive, negative, and neutral. Among a total of 8911 records, TextBlob categorized 3193 records as positive, 4599 records as neutral while 1119 are labeled as negative, as shown in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Number of samples for neutral, positive and negative sentiments by TextBlob.

https://doi.org/10.1371/journal.pone.0272350.g002

Fig 3 shows the ratios of positive, negative, and neutral sentiments, as labeled by TextBlob. It can be observed that neutral sentiments make up most of the dataset with 51.61% of the dataset. It is followed by the positive and negative tweets which make 35.83% and 12.56%, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Ratio of positive, negative and neutral sentiments.

https://doi.org/10.1371/journal.pone.0272350.g003

Feature extraction

Learning algorithms are highly dependent on the features to make accurate predictions [43]. This makes extraction of features a very crucial step as highly correlated features will produce a better outcome and highly accurate classification and vice versa. Often, it is better to utilize more than one feature extraction approach to analyze their efficacy. This study follows a similar norm and selects TF-IDF and BoW as feature extraction techniques.

The BoW is a flexible and simple feature extraction technique that is defined as the representation of text data within a document. It quantifies the frequency of the occurrence of the word in the document irrespective of its position and importance in the document [44]. BoW mainly involves two metrics including the vocabulary of words that are known by the model and the measure of the occurrence of those particular words which are present in the vocabulary.

TF-IDF is a statistical technique that assesses the relevance of a word in a document or a collection of documents. TF-IDF not only vectorizes the text data but also quantifies the features among the whole corpus [45]. TF-IDF is obtained by taking a product of two metrics including the number of times a word has appeared in a document and the IDF (inverse document frequency) of a word in a collection of documents [46]. It assigns weights with respect to the importance of a term in a given corpus.

Machine learning models

Sentiment classification models are the learning algorithms that categorize the given text into three sentiments. Several well-known machine learning models have been selected for the task at hand in this study including AC, ETC, LR, RF, GBM, KNN, SVC, and DT. These models are optimized regarding their prediction accuracy by using different sets of hyperparameters where a complete list of hyperparameters used in this study is provided in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Hyperparameter settings of machine learning sentiment classifiers.

https://doi.org/10.1371/journal.pone.0272350.t003

Proposed voting classifier

EB-DT is a probabilistic voting classifier that involves an ensemble of three best performing individual classifiers including ETC, GBM, and DT, as shown in Fig 4. The proposed model is based on the combination of three classifiers and carries out prediction tasks using the highest probability of the output variable from each of the three classifiers. The proposed EB-DT combines the performance of individual classifiers into a strong learner [51]. It works by aggregating the output of each of the three classifiers using voting. Afterward, prediction is performed based on the majority voting criterion. EB-DT integrates soft voting to predict the output class by averaging the probability assigned to the classes by each of the individual classifiers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Architecture of proposed EB-DT voting classifier.

https://doi.org/10.1371/journal.pone.0272350.g004

The choice of an estimated ensemble model stands upon several grounds. First, ensemble models has the tendency to show better performance as reported in the existing literature [32, 45, 55]. Utilizing more than one model potentially reduces the misclassification rate. Secondly, the choice of models varies concerning the nature of data and models can be influenced by the size of the feature set, the ratio of class samples (balanced vs imbalanced datasets), etc. which can increase the probability of model overfit or underfit. Using multiple models on the datasets and combining their output mitigates such risks. Thirdly, the voting criterion is selected because of the results reported for voting classifiers where the ensemble models tend to show better performance than individual models [33, 56]. Keeping in view the diversity of the text tokens used in the tweets, it seems a probable choice to utilize multiple classifiers. Different types of machine learning models like tree-based or linear models, show a different level of performance with sentiment analysis tasks, and using more than one classifier to make the final prediction tends to increase the probability of correct class prediction.

Each model in the proposed EB-DT ensemble takes the text input features to predict probabilities for positive, negative, and neutral sentiments. Mathematically it can be written as (1) where C_p is the prediction probability by the EB-DT using soft voting criteria, N is the number of models, and avgC is the per sentiment probability by each model.

The C_p can also be written as (2) (3)

For EB-DT, Eq 3 is used to find the probability of positive sentiment against a tweet. Here Pos_DT is the probability of positive class by the DT model, Pos_GBM is the probability by the GBM, and Pos_ETC is the probability of positive class by the ETC model. Similarly, EB-DT calculates the probability for negative and neutral sentiments as follows (4) (5)

The ensemble model EB-DT makes the final prediction using (6)

The functioning of the EB-DT can be further elaborated using an example run. For this purpose, the tweet “One of two shots so far, but have always, and will continue to wear the mask” is taken. DT probabilities against this tweet are (Pos = 0.3, Neg = 0.2, Neu = 0.5), ETC shows probabilities as (Pos = 0.2, Neg = 0.2, Neu = 0.6), while GBM shows (Pos = 0.2, Neg = 0.1, Neu = 0.7) for the given tweet. Using Eq 2, average probabilities can be calculated as (7) (8) (9)

The average probabilities are then used by the argmax function to predict the final class of the tweet which in this case is neutral as it has the highest average probability considering the models that form the ensemble. (10) (11)

Latent Dirichlet allocation

LDA is a well-known probabilistic topic modeling method that is utilized for the extraction of topics from the given input data [57]. The word Latent in LDA conveys the meaning of existing information that is yet to be discovered. The fundamental purpose of topic modeling is to extract the hidden topics using the Dirichlet model which exploits the patterns of similar words occurring frequently and repeating together. LDA estimates the occurrence of a word in a document which results in building data points and estimating the probabilities. Furthermore, probabilities for the topics in the document are generated which are further classified into the corpus or set of documents [58]. LDA considers two main metrics: 1) The corpus is composed of topics, and 2) The topics are composed of words.

Experimental results of machine learning sentiment classifiers

Sentiment classification of tweets is conducted on the preprocessed dataset using TfidfVectorizer(). Table 4 presents the results of machine learning classifiers when trained using the TF-IDF features. Results show that the boosting classifier including GBM and tree-based classifiers including ETC and DT outperform other conventional machine learning models with a 0.97 accuracy score. GBM leverages the regularization methods to reduce the over-fitting of training data resulting in more accurate results. SVC and RF perform slightly poorly in classifying the tweets using TF-IDF features with a 0.95 accuracy score, however, RF yields a higher F1 score of 0.95 as compared to SVC which achieves a 0.93 F1 score. The difference in the performance of RF and ETC is due to the ability of ETC to use randomized features for the training of de-correlated trees instead of the optimal ones which control the over-fitting and enhance the performance of the model. LR performs comparatively lower with a 0.93 accuracy score and 0.91 F1 score. In line with this, AC performs poorly as well with a 0.90 accuracy score and 0.88 F1 score. KNN does not work well with high dimensional data and shows performance with the lowest accuracy score as compared to other classifiers in predicting the sentiments of the tweets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Experimental results of machine learning sentiment classifiers using TF-IDF features.

https://doi.org/10.1371/journal.pone.0272350.t004

Fig 5 graphically presents the performance comparison of machine learning sentiment classifiers using TF-IDF features. It can be observed that similar to accuracy, DT, ETC, and GBM achieved optimum precision, recall, and F1 scores. The performance of classifiers in terms of sentiment labels, when combined with TF-IDF is displayed in Fig 6. The machine learning sentiment classifiers perform well for the prediction of the neutral class as the number of records corresponding to the neutral class is higher as compared to the other two classes. Whereas, an overall decrease in the performance of the classifiers can be observed in the prediction of negative class due to a smaller number of records corresponding to negative sentiments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Experimental results of Sentiment classifiers using TF-IDF features.

https://doi.org/10.1371/journal.pone.0272350.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Performance comparison of classifiers with TF-IDF features for individual class.

https://doi.org/10.1371/journal.pone.0272350.g006

Apart from TF-IDF, we implemented BoW for sentiment classification experiments. Table 5 demonstrates the results of machine learning classifiers when combined with BoW features. The boosting classifier along with tree-based classifiers including ETC and DT show better performance compared to other classifiers. However, an improvement in the performance of other classifiers can be observed when integrated with BoW features as compared to their performance with TF-IDF features. The BoW feature set is larger as compared to TF-IDF due to its extraction of features irrespective of their position and structure in the text which resulted in the improved performance of the classifiers. The accuracy of SVC and RF has been improved from 0.95 with TF-IDF to 0.96 with BoW. However, the performance of KNN does not improve as it does not work well with high-dimensional data.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Experimental results of machine learning classifiers using BoW features.

https://doi.org/10.1371/journal.pone.0272350.t005

Fig 7 presents the comparison of the overall performance of sentiment classifiers when combined with BoW features. It can be observed that the tree-based models including RF, DT, and ETC perform comparatively better as compared to the other models in terms of precision, recall, and F1 score. Similarly, GBM also yields the overall highest precision, recall, and F1 score. KNN however performs the worst while SVC, AC, and LR also perform comparatively low as compared to tree-based models and boosting models. However, an improvement in their performance can be observed with the increase in the feature set size by BoW as compared to TF-IDF.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Experimental results of machine learning classifiers using BoW features.

https://doi.org/10.1371/journal.pone.0272350.g007

Fig 8 shows that the performance related to the neutral class is comparatively better as compared to the negative and positive classes. This is due to the small number of records of the negative and positive classes and a large number of neutral records. A large number of records enables the classifiers to learn the patterns more effectively thus resulting in better performance for that particular class.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Performance analysis for individual class using BoW features.

https://doi.org/10.1371/journal.pone.0272350.g008

Experimental results of deep learning classifiers

This study also makes use of three deep learning models including LSTM, GRU, and ensemble model CNN-LSTM. The details of the layer-wise architecture of deep learning models are provided in Fig 9. All models use an embedding layer at the beginning with an input dimension of 100 and a vocabulary size of 5000. LSTM uses a dropout layer with a 0.5 drop rate, followed by an LSTM layer, and a dense layer. GRU also uses the same dropout layer, followed by a GRU layer, a simple RNN (Recurrent Neural Network) layer, and a dense layer in the end. The combined CNN-LSTM network uses a Conv1D layer after the embedding layer which is followed by a max-pooling layer. In the end, it uses two dense layers of 16 and 3 neurons, respectively. All models use the ‘categorical_crossentropy’ loss function while the ‘Adam’ optimizer is used.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Layered architecture of deep learning models.

https://doi.org/10.1371/journal.pone.0272350.g009

Experimental results for deep learning models given in Table 6 show that LSTM shows an accuracy of 0.97 as compared to GRU and CNN-LSTM each with an accuracy score of 0.96. With its ability to learn longer dependencies LSTM outperforms GRU and CNN. As this study is concerned with the multi-class classification problem, we will also consider the F1 score of the classifiers for a better understanding of their performance. It can be observed that the macro average F1 score of three of the deep learning models is 0.95. However, the classifiers show better performance in the prediction of the neutral class. The deep learning models perform well when subjected to a large dataset. The underlying dataset in this study consists of 8911 records which maybe not be large enough to obtain the highest performance as compared to the tree-based RF, DT, and ETC classifiers. CNN and GRU fail to outperform the traditional machine learning classifiers due to their low learning and slow convergence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Experimental results of deep learning classifiers in sentiment classification of tweets.

https://doi.org/10.1371/journal.pone.0272350.t006

Experimental results of proposed EB-DT classifier

The tree-based models including DT, and ETC showed robust performance along with the boosting classifier GBM as discussed previously. Therefore, this study combines the best performing classifiers to further improve the overall performance of the resulting ensemble. For this purpose, we proposed a voting classifier EB-DT which is an ensemble of three base classifiers including ETC, DT, and GBM. The resulting EB-DT classifier makes final predictions by aggregating the predictions of base classifiers under the soft voting criteria.

Table 7 shows the experimental results of the proposed EB-DT using TF-IDF and BoW features. It can be observed that the proposed EB-DT classifier shows robust performance with a 0.99 accuracy, along with similar macro average precision, recall, and F score. However, the performance of the proposed EB-DT model decreased when combined with the BoW feature set. TF-IDF extracts features that have high predictive information regarding the target variable. However, the feature extraction technique BoW extracts all of the known vocabulary words from the data irrespective of their relevance in the whole corpus which results in lower performance of the proposed EB-DT. In the case of sentiment labels, it can be observed that the F1 score for the neutral class is 0.99 with both TF-IDF and BoW feature sets while the negative class, performs comparatively lower with a 0.96 F1 score for both features. Results confirm that using the voting method to combine the outputs of three highly performing classifiers yields the best result regarding the accuracy, precision, recall, and F1 score and achieves the highest accuracy of 0.99 which is higher than both the traditional machine learning and deep learning classifiers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Experimental results of the proposed EB-DT with TF-IDF and BoW features.

https://doi.org/10.1371/journal.pone.0272350.t007

Performance analysis of sentiment classifiers

Fig 10 shows the accuracy score of all the sentiment classifiers utilized for experiments in this study. It can be seen that the proposed model EB-DT shows highly accurate results in comparison with other classifiers when combined with TF-IDF. It also performs well with BoW features as compared to other classifiers when used with BoW features. Furthermore, deep learning models perform comparatively lower than the proposed voting classifier. EB-DT leverages the benefits of DT, ETC, and GBM and achieves highly accurate results for classifying the sentiments of extracted tweets. Overall, the performance of KNN is observed to be the lowest as it does not work well with high dimensionality data. As the text features extracted by TF-IDF and BoW have a high dimension which decreases the performance of KNN. Whereas, the boosting classifier AC also does not perform for both TF-IDF and BoW features.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Performance comparison of classifiers in sentiment classification of tweets.

https://doi.org/10.1371/journal.pone.0272350.g010

Proposed approach comparison with previous studies

To show the significance of the proposed approach, the performance comparison is carried out with existing state-of-the-art studies. Several studies are selected for comparisons, such as [32] proposed an ensemble model SVE for the medical review sentiment predictions. The study [33] used voting criteria to combine the GBM and SVM for user reviewer sentiment classification. Similarly, the study [34] used soft voting criteria to combine LR and SGDC models. Performance comparison results are presented in Table 8 indicating the superior performance of the proposed approach with a 0.99 accuracy score which is far better than existing works. In addition, the robustness of the proposed approach is evaluated by comparing its execution time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Performance comparison with previous studies.

https://doi.org/10.1371/journal.pone.0272350.t008

The proposed model is better in terms of computational time as compared to other studies. Although [34] provides a much smaller execution time, its accuracy is substantially low as compared to the proposed model. Also, the current study uses three models while existing studies use two models for the ensemble.

Topic modelling

The positive and negative tweets are subjected to LDA to extract topics that correspond to the factors impacting the sentiments of the public opinions towards following COVID-19 safety precautions. We extracted a total of 5 topics from both positive and negative tweets individually, as shown in Table 9. A total of 40 most occurring words are also extracted from the tweet dataset using LDA. The resulting keywords for positive sentiments are “love, cute, good, mouth, nose, comfortable, clean, best, fit, cover” and the main keywords for negative sentiments are “fuck, work, dark, nose, cover, adjust, unbreathable”. From the extracted topic keywords, it can be judged that the negative sentiments appeared due to the masks being unbreathable causing discomfort in wearing them. Another cause for the negative sentiments can be the double-layered facemasks covering the mouth and nose of people which causes difficulty in speaking and breathing. However, from the keywords extracted from positive sentiments, it can be observed that people felt safe by wearing the masks and they find it fascinating as they breathe clean air. The re-usability of the masks is also one of the main factors causing the positive opinions from the public.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Topics extracted from positive and negative tweets using LDA.

https://doi.org/10.1371/journal.pone.0272350.t009

Fig 11 shows the percentage ratio of positive and negative key words found using LDA. It can be observed that ‘resuabl’ and ‘wear’ have the highest ratio for positive terms while ‘bad’ and ‘fuck’ are the most frequently used words among the found negative terms.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Positive and negative tweets topics term percentage in data, (a) Positive Topics Terms, (b) Negative Topics Terms.

https://doi.org/10.1371/journal.pone.0272350.g011

Analyzing change in sentiments over Time-I

This study further analyzes peoples’ sentiments regarding COVID-19 restrictions over time to study the impact of time on the change in tweets’ sentiment. For this purpose, a separate dataset is collected after one month for the analysis. The same data extraction procedure is adopted as before and tweets are extracted for 17 February when the new recorded COVID-19 cases were low. We analyze the data using the proposed sentiment analysis approach and a third-party application ‘sentiment viz’ [59]. Fig 12 shows the sentiment count on the new data. It can be seen that an increasing trend is observed for positive tweets. Apparently, the people realize the effectiveness and benefit of wearing a facemask against COVID-19 which shifts their sentiments in favor of these restrictions. Similarly increase in the negative sentiments has been observed, however, the ratio of positive sentiments is substantially higher than both neutral and negative tweets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Comparison of positive, negative and neutral tweets on 1st January and 17 February 2022.

https://doi.org/10.1371/journal.pone.0272350.g012

Analyzing change in sentiments over Time-II

The third set of experiments is performed for analyzing the change in COVID-19 restrictions-related sentiments on the data collected for 19 February 2022. Comparative results of positive, negative, and neutral sentiments are shown in Fig 13. Trends for peoples’ sentiments show that a gradual increase in positive sentiments is prevalent. Over time, peoples’ sentiments have shifted from neutral to negative and positive. However, tweets containing positive sentiments are higher than negative sentiments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Comparison of positive, negative and neutral tweets on 1st January, 17 February and 19 February 2022.

https://doi.org/10.1371/journal.pone.0272350.g013

Comparison of proposed approach results with third-party solutions

For corroborating the results of the current study, a separate analysis is performed using a third-party solution, called ‘sentiment viz’. Figs 14–16 show the sentiment viz app results on the new dataset. A high number of happy emotions are observed in the new dataset. Results from sentiment viz confirm the results obtained using the proposed approach. Figs 14 and 15 show that tweets show calm, relaxed and happy emotions for COVID-19 precautions as compared to before.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. People emotions in new tweets dataset extracted using sentiment viz app.

https://doi.org/10.1371/journal.pone.0272350.g014

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 15. People emotions and sentiments in new tweets dataset extracted using sentiment viz app.

https://doi.org/10.1371/journal.pone.0272350.g015

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 16. Word cloud in terms of pleasant and unpleasant categories.

https://doi.org/10.1371/journal.pone.0272350.g016

Fig 16 shows the word cloud for the new dataset. It shows that the pleasant category contains the safety, beneficiary, etc. words while the unpleasant category contains annoying, risk, death, etc. words. These words also confirm the words found using topic modeling with LDA.

Statistical T-test

We also deploy a T-test to show the statistical significance of the proposed EB-DT voting classifier [60]. Statistical T-test compares the proposed EB-DT results with other state-of-the-art models. In the output, if the critical value is greater than or equal to the t-stat then T-test accepts the null hypothesis (H_o) indicating that the T-test rejects the alternative hypothesis which shows that both compared results are not statistically significant.

When the results of EB-DT are compared with other models’ results using TF-IDF the T-test gives a t-stat value of 7.483 and a critical value of 0 which shows that the T-test rejects the H_o. It indicates that the results from the proposed EB-DT model are statistically significant. When we compared the EB-DT results with other models’ results using the BoW feature the T-test gives a t-stat value of 3.667 and a critical value of 0 which also shows that the T-test rejects the H_o to show the statistical significance of the proposed approach.

Policy implications

This study is designed to analyze the reviews of the public regarding COVID-19 restrictions. Although such restrictions serve as the first line of defense and are initiated for the safety of the people from the pandemic, people do not share the same sentiment. Consequently, people show their willingness and dissent on social media platforms. Analyzing such views can provide useful insights into people’s sentiments and provide an overall picture of what people think of these restrictions. Consequently, current policies about the COVID-19 restrictions can be drafted and revised accordingly. Additionally, the analysis of change in people’s sentiment over time can be beneficial to examine the impact of specific policies for a particular period. The study does not define any such policy or analyze it, it simply provides insight into people’s sentiments. Since people’s views and sentiments are deemed important, especially in developed countries, they do have an impact on the decision-making process and can be used to take corresponding actions.