https://orcid.org/0000-0002-6100-4895
Figures
Abstract
Autism spectrum disorder (ASD) is a developmental disability characterized by persistent impairments in social interaction, speech and nonverbal communication, and restricted or repetitive behaviors. Currently Electroencephalography (EEG) is the most popular tool to inspect the existence of neurological disorders like autism biomarkers due to its low setup cost, high temporal resolution and wide availability. Generally, EEG recordings produce vast amount of data with dynamic behavior, which are visually analyzed by professional clinician to detect autism. It is laborious, expensive, subjective, error prone and has reliability issue. Therefor this study intends to develop an efficient diagnostic framework based on time-frequency spectrogram images of EEG signals to automatically identify ASD. In the proposed system, primarily, the raw EEG signals are pre-processed using re-referencing, filtering and normalization. Then, Short-Time Fourier Transform is used to transform the pre-processed signals into two-dimensional spectrogram images. Afterward those images are evaluated by machine learning (ML) and deep learning (DL) models, separately. In the ML process, textural features are extracted, and significant features are selected using principal component analysis, and feed them to six different ML classifiers for classification. In the DL process, three different convolutional neural network models are tested. The proposed DL based model achieves higher accuracy (99.15%) compared to the ML based model (95.25%) on an ASD EEG dataset and also outperforms existing methods. The findings of this study suggest that the DL based structure could discover important biomarkers for efficient and automatic diagnosis of ASD from EEG and may assist to develop computer-aided diagnosis system.
Citation: Tawhid MNA, Siuly S, Wang H, Whittaker F, Wang K, Zhang Y (2021) A spectrogram image based intelligent technique for automatic detection of autism spectrum disorder from EEG. PLoS ONE 16(6): e0253094. https://doi.org/10.1371/journal.pone.0253094
Editor: Wajid Mumtaz, National University of Sciences and Technology, PAKISTAN
Received: February 3, 2021; Accepted: May 27, 2021; Published: June 25, 2021
Copyright: © 2021 Tawhid et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: In this study, we have used third party dataset from King Abdulaziz University Hospital, Saudi Arabia, Jeddah. It is a publicly available dataset that can be found in this link, https://malhaddad.kau.edu.sa/Pages-BCI-Datasets.aspx. Anyone can access the dataset for her/his study through formal request using the contact form in the above link or direct email to Dr. Mohammed Jaffer Alhaddad (email: malhaddad@kau.edu.sa).
Funding: This study is supported by the Australian Research Council Linkage Project (Project ID: LP170100934). Nexus eCare is the industry partner for this project. The funder or the commercial affiliation does not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: One of the authors (Frank Whittaker) is at the time of publication in the journal, employed by Nexus eCare. This does not alter our adherence to PLOS ONE policies on sharing data and materials. There are no patents, products in development or marketed products associated with this research to declare. All other authors have declared that no other competing interests exist.
Introduction
Autism spectrum disorder (ASD) is a group of complex neurological developmental disorders that includes autism, childhood disintegrative disorder, Asperger’s syndrome and an undetermined form of pervasive developmental disorder [1]. The range and severity of ASD symptoms can vary widely that commonly includes difficulty with social communication and interactions, obsessive interests, reduced eye contact and restricted or repetitive behaviors. ASDs begin in early childhood, mostly within five years of age and remains for the rest of the life [2]. World Health Organization (WHO) reported that globally in every 160 children, one child diagnosed as ASD [2]. According to the autism spectrum, autism frequency rate increases around 40% from one in 100 to an approximate one in 70 in Australia [3]. The Centers for Disease Control and Prevention (CDC) stated that one child in about 54 children is diagnosed with an ASD in the U.S. in 2020 [4]. ASDs can substantially restrict an individual’s ability to carry out everyday activities and participate in social movement. ASDs often negatively impact the person’s educational and social achievements, employment opportunities, limit the capacity to conduct daily activities and participation in society. Globally, ASD individuals are frequently subjected to abuse, differentiation and infringement of human rights. [2]. So far, no cure has been found for ASD but early intervention can improve brain development and also can enhance learning, communication and social skills. To do so, an effective, efficient and high accuracy system is needed to diagnose ASD.
The human brain with about 86 billion neurons, is considered to be the most complicated biological system in the known world that regulates all our thoughts, perceptions, memories, feelings and actions. Functional condition of the brain is a big data collection and also the main source of information for neurological disorder diagnosis which makes it a vital and wide area of study in the field of biomedical science. Different techniques are available for capturing the functional activity of the brain such as, positron emission tomography (PET), magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), electrocorticography (ECoG) and electroencephalography (EEG) [5–7]. Among these techniques, EEG is widely used because of its outstanding temporal resolution, usability, non-invasiveness, low set-up costs and mass availability for clinicians [8]. Neurons uses electrical impulse of different frequency bands for their communication which is recorded in EEG through electrodes attached to the scalp. This produces a large volume of EEG multi-channel signals that neurologists visually interpret to identify and recognize neurological disorders [9]. However, due to the lack of standard assessment criteria, this visual inspection is not a rational evaluation technique and also time-consuming, error-prone, exhaustive, subjective decision with human error and has reliability problem [10].
As the technological advancement grows day by day, so the computer-aided diagnosis (CAD) has become an integral part of the medical industry. Various studies have carried out to diagnose ASD using EEG signal. Those studies can be broadly divided into two groups depending on the feature extraction and classification technique. First approach is known as machine learning (ML) technique in which different time-frequency based features are extracted from EEG signal and then those extracted features are used for ASD classification using different ML based classification techniques. The key success of the ML based classification depends on mining significant features from EEG signals and till now various researchers have attempted this approach for ASD classification. Sheikhani et al. [11] used short-term Fourier transform (STFT) for feature extraction and k-nearest neighbor (kNN) for classification to achieve an accuracy of 82.4% over a dataset consists of 17 subjects (10 ASD and 7 control group). In a later study [12], they used STFT and statistical analysis with kNN on a dataset of 28 subject (17 ASD, 11 control group) to achieve 96.4% accuracy. Bosl et al. [13] proposed a diagnostic approach to use EEG data as a biomarker for children at a high risk for ASD. They extracted features using minimum mean square error (mMSE) and classified using kNN, naïve bayes (NB), and support vector machine (SVM) to produce classification accuracy over 90% on a dataset of 79 infants (46 high risk for autism (HRA), 33 controls) of 6 to 24 months of age. In a later study [14], they used a data-driven approach for ASD classification in which EEG data from 188 infant (89 low-risk controls (LRC), 99 HRA; ages of 3 to 36 months) participants were used. EEG signal was decomposed into six sub bands using wavelet transform (WT) and nine different non liner features were extracted from each sub bands. Using leave-one-out cross-validation, they evaluated obtained features as input to SVM for classification and achieved a sensitivity and specificity value exceeding 95% at some ages in distinguishing ASD subjects from the LRC subjects. In another study, Eldridge et al. [15] used variance in time by computing the sum of signed differences (SSD) and mMSE features from pre-processed signals and fed to different classifiers. On a dataset of 49 children (19 ASD and 30 Non ASD), highest accuracy of 79% was achieved with NB classifier. In [16], Grossi et al. introduced a complex algorithm named MSROM/I-FAST for EEG processing with seven machine learning algorithms namely: sine net neural networks (Sn), logistic regression (LR), sequential minimal optimization (SMO), kNN, K-contractive map (K-CM), NB and random forest (RF) to classify autism. Using 25 subjects’ (15 ASD and 10 typically developing (TD)) resting state EEG data with leave-one-out cross-validation a highest accuracy of 92.8% was achieved with RF classifier. Heunis et al. [17] used recurrence quantification analysis (RQA) for feature extraction from resting state EEG of 14 children (7 ASD and 7 TD) and feed into SVM classifier. They yielded a high accuracy of 92.9% with leave-one-subject-out validation process. A decision support system (DSS) named ASDGenus was proposed by Haputhanthri et al. [18] for ASD diagnosis using 15 participants (10 ASD, 5 control) EEG data. They combined statistical features (mean, standard deviation) before and after applying DWT for each channel and applied correlation-based feature selection (CFS) for significant feature selection. Finally, four classifiers (LR, SVM, NB and RF) were applied and the highest achieved accuracy was 93% using RF classifier. In a later study [19], they extended ASDGenus incorporating shannon entropy for feature extraction of the EEG data and also used the average temperature of the thermogram face images of the participants. Using a dataset of 17 participants (8 ASD, 9 control), they achieved an accuracy of 88% using only EEG data with RF classifier and 94% accuracy using both EEG and thermogram image data with both LR and MLP classifiers. In a recent study [20], abdolzadegan et al. extracted linear (power spectrum, wavelet transform, fast fourier transform) and nonlinear (fractal dimension, correlation dimension, lyapunov exponent, entropy, detrended fluctuation analysis and synchronization likelihood) from 45 subjects’ (34 ASD, 11 non ASD) EEG data and applied different feature selection techniques (mutual information, information gain, minimum-redundancy maximum-relevancy and genetic algorithm). For classification SVM and kNN used and achieved an accuracy of 90.57% and 72.77% respectively. Functional connectivity and temporal relationship of brain activity was used to classify and predict the severity score of ASD by jayarathna et al. [21]. They decomposed the signal into five feature bands (delta, theta, alpha, beta, and gamma) and five feature sets were created using amplitude and power for each electrode. Using 43 different classifiers on a dataset of 17 subjects (8 ASD and 9 control), they achieved an accuracy of 98.06% using JRip. In the ML based classification approaches mentioned above build features from raw EEG data using feature engineering approach, which needs experts with comprehensive knowledge of the target feature domain [22].
Recently, deep learning (DL) based classification approaches are getting popular among the researchers due to its ability to learn features from raw data automatically and also perform classification using those features in automatic process [22]. Ahmadlou et al. [23] proposed a fractality and wavelet-chaos-neural network based ASD diagnosis system. They introduced the idea of using fractal dimensions (FDs), a non-integer dimension that shows the degree of complexity and self-similarity of a signal as a feature. Using Eye-closed EEG data from 17 subjects (9 ASD, 8 TD) with a two layer radial basis function neural network (RBFNN), they achieved 90% accuracy. Using the same database in their later study [24] where they used improved visibility graph (VG) for fractality investigation based feature named power of scale-freeness of VG (PSVG). Enhanced probabilistic neural network (EPNN) was used for classification and got an accuracy of 95.5%. In another study of same authors used the analysis of functional connectivity of brain using fuzzy Synchronization Likelihood and diagnoses ASD based on that [25]. Using EEG data from 18 subjects (9 ASD, 9 TD) with EPNN classifier they obtained 95.5% accuracy. Djemal et al. [26] introduced an ASD diagnosis system using discrete wavelet transform (DWT), shannon entropy (SE) and artificial neural network (ANN). Two-layer ANN with ten-fold cross-validation was used for classification on an EEG dataset of 19 subjects (9 ASD and 10 non ASD). They segmented the data into 50 second long segments with two different overlapping techniques. With half overlapping segmentation provided very good classification accuracy of 99.7% and without any overlapping in segmentation got an accuracy of 98.6%. Alturki et al. [27] also used 50 second segmentation and decomposed the signal into sub bands using DWT. They used logarithmic band power (LgBP), standard deviation (SD), variance, kurtosis, and SE to extract features from the segmented decomposed sub bands and feed those features to different classifiers namely: linear discriminant analysis (LDA), SVM, kNN, and ANN. Using a dataset of 19 children (9 ASD and 10 non ASD), they achieved the highest accuracy of 98.2% using SE and ANN classifier. The main benefit of the DL based classification approaches is that it does not require any specific feature domain experts for feature extraction from the raw data. The DL methods perform both feature extraction and classification automatically and generates better results than ML based classification processes but it works as a black box to the user [28]. In the same study [21], the authors have also used CNN model on power spectrum of electrodes to evaluate the long-term dependencies between ASD and EEG data. Using a three layer CNN model they achieved an accuracy of 90%. Sometimes researchers use crafted features as input for DL models instead of using large scale raw EEG data to reduce the computation time and also for training the model with important features [23–27].
Although the state-of-the-art techniques have achieved various levels of effectiveness in some cases but their ability of robustness and generalization are limited. This is because it is often very hard to extract representative features from EEG signals using current techniques due to the non-stationary nature and presence of noise in EEG signals. In order to describe EEG, none of the previous approaches considered a two-dimensional (2D) time-frequency image where extraordinary features would be unveiled from different perspectives.
This study proposes a new 2D EEG spectrogram image-based scheme involving ML and DL for automatic identification of ASD. In this study, 2D time-frequency (T-F) spectrogram images are used for describing the nonstationary characteristics of EEG signals and finally the images are assessed through ML and DL based process, separately. For EEG signals, spectrogram image is a type of visual feature representation in the T-F domain. Frequency band varies with time in the spectrogram image, and various colors in the image reflect different energy values in the EEG signal. [29]. Previously, few studies have used time-frequency (T-F) based images for the classification of neurological disorders such as epilepsy [10], epileptic seizures [30], clinical brain death diagnosis [29], schizophrenia [28] and classification of sleep stage [31], but never for the classification of ASD. In our recent work [32], we have used this T-F based spectrogram image of EEG signal for ASD classification using ternary CENTRIST (tCENTRIST) [33] with SVM classifier. In this study, we have extended our previous work of T-F image-based ASD classification using both ML and DL based approach. In the ML based classification, we have used tCENTRIST for feature extraction from spectrogram images due to its computational simplicity and better performance. Extracted features are classified using six different classifiers named: NB, LDA, RF, kNN, LR and SVM. Ten-fold cross validation is used to evaluate the performance of ML based classification approaches. For the DL based scheme, three different convolutional neural network (CNN) models are used to classify the spectrogram images with the dataset divided into 70%, 15%, 15% for training, validation and testing the model respectively. The findings obtained from the proposed methods are contrasted with other current literature that used the same dataset.
This study’s major contributions are summed up as follows:
- Introduction of an efficient and effective automatic classification model for identifying ASD from non ASD subjects using T-F spectrogram image of EEG signals.
- Investigating the performance of six ML based classifiers with tCENTRIST based textual feature extraction method.
- Investigating three different CNN models for DL based classification.
- Improving the efficiency of classifications relative to existing methods.
The rest of this article is structured as follows. Section presents details of the proposed method. Section describes the dataset and performance evaluation parameters used in this study. Experimental results and their corresponding discussions are given in Section and finally, Section contains concluding statement and reveals possible future work.
Methodology
In this study, an ASD classification technique is proposed based on spectrogram image of EEG signals. Fig 1 provides a summary of the proposed framework. The system can be divided into three parts: pre-processing and spectrogram image generation, machine learning based classification and deep learning based classification.
Pre-processing and spectrogram image generation
In this section, at first, artifacts are removed from raw EEG data using pre-processing techniques like re-referencing, filtering and normalization. Common average referencing (CAR) is used for re-referencing, in which the average value of all electrode collection (common average) is used as reference. In the second step of pre-processing, infinite impulse response (IIR) filter is used to low pass filter the signal at 40Hz cut off frequency and finally the filtered signals from each electrode is normalized to the interval [-1, 1]. After that, pre-processed signals are segmented into 3.5 second window frames for each subject of the dataset. Using Short-Time Fourier Transform (STFT) for each of the above segments, the spectrogram plot is generated in the last step and saved as image. These spectrogram images are used as input for both the ML and DL based methods.
Machine learning based process
The ML based classification process consists of three sub processes: feature extraction, dimension reduction and classification. For extracting representative features from those spectrogram images, Ternary CENTRIST (tCENTRIST) is used. It is a Local Ternary Pattern (LTP) and CENsus TRanformed hISTogram (CENTRIST) based image feature extraction method proposed by Dey et al. [33]. Ternary CENTRIST (tCENTRIST) is a fusion of LTP and CENTRIST that uses LTP in place of Linear Binary Pattern (LBP) of CENTRIST [34]. This tCENTRIST technique has given better performance for garments texture classification [33] and gender classification from face image [35] and also computationally simple. It uses the Spatial Pyramid Matching (SPM) scheme to capture the global image structure that divides an image into sub regions and LTP based histogram of those regions are generated and concatenated to construct a single histogram as a feature for that image. Principal Component Analysis (PCA) is applied to reduce the extracted feature dimensions and finally the reduced features set is used as input to different ML based classifiers. For classification, six different classifiers are used: NB, LDA with pseudolinear discriminant analysis type, RF, kNN with nine as neighbor number (k = 9), LR and SVM with linear kernel (used LibSVM [36]).
Deep learning based process
For the DL based classification, CNN model is used as it is the best DL model to deal with image related problems [37, 38]. In this study, three different CNN model is used for performance evaluation. Model 1 consists of three convolution blocks with each of them having a max pool layer. On top of it there is a fully connected layer with 512 units which is activated by a relu activation function. Overall layout of the model 1 is given in Fig 2. Second model is same as the first model except that 20% dropout is applied on the last max pool layer. Applying dropout will randomly set 20% of the neurons to zero during each training epoch. Fig 3 shows the block diagram of model 2. Last CNN model contains 4 alternating convolutional and max pooling layers, followed by a 25% dropout after every two convolution/max pooling pair. After the last pooling layer, a fully-connected layer with 256 neurons, then another 50% dropout layer, and finally a softmax classification layer for two classes is attached. Full layout of the model 3 is given in Fig 4.
Performance evaluation
This section initially describes the detailed overview and pre-processing method of the dataset used in this experiment. After that, performance parameters with equations are discussed that are used for evaluating the performance of the proposed system.
Dataset
This proposed method is evaluated using the dataset from King Abdulaziz University (KAU) Hospital, Saudi Arabia, Jeddah [39]. It is a publicly available dataset that can be found in this link, https://malhaddad.kau.edu.sa/Pages-BCI-Datasets.aspx. Anyone can access the dataset for their study through formal email request to the owner of the dataset Dr. Mohammed Jaffer Alhaddad (malhaddad@kau.edu.sa). We also did the same process and obtained the dataset for this study. Participants anonymity is ensured by not publishing any personal identification information of the subjects. The dataset contains sixteen subjects with twelve from ASD group (3 girls and 9 boys, age 6–20 years of age) and four subjects from control group (all boys, 9–13 years of age) with no past track of neurological disorders. Signals from subjects were recorded using Ag/AgCl electrodes with g.tec EEG cap, g.tec USB amplifiers, and BCI2000 software in relaxing state to get the artifact-free EEG data. The data recording was carried out from 16 channels using the international 10–20 system, as shown in Fig 5 with right ear lobe as REF and AFz as GND. A band-pass filter with a pass frequency band (0.1–60Hz) and a notch filter with a frequency band (60Hz) was used during the recording time to filter the dataset. Finally, all EEG signals were digitized at 256Hz sampling rate.
According to the proposed method, data is pre-processed using CAR, IIR and normalized. After pre-processing, those signals are segmented into 3.5 second time frames to get the precise information and finally spectrogram plot images are produced from those fragments using STFT. A total of 4657 images are produced with 3276 images from ASD subjects and 1381 images from non ASD subjects. Sample spectrogram images from the proposed method are given in Fig 6 where Fig 6a shows images from ASD group and Fig 6b shows images from non ASD subjects. These images are used as input for classification process of both ML and DL based techniques.
(a) Images from ASD group, (b) Images from non ASD subjects.
Image generation and ML based experiments are carried out in MATLAB (R2020a) environment in a computer with intel core i5 64bit processor with a frequency of 1.7GHz and 8 GB memory. DL based experiments are carried out in Google Colab environment [40].
Performance evaluation parameters
The categorization performance of the proposed framework is measured using Receiver Operating Characteristic (ROC) parameters such as True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN) to calculate the sensitivity, specificity, F1 score and overall accuracy using (1)–(4). These assessment criteria allow to estimate the performance behavior of the classifiers [41–46].
(1)
(2)
(3)
(4)
Here,
- TP implies that spectrogram image from ASD subject is correctly diagnosed as ASD class.
- TN implies that spectrogram image from healthy subject is correctly diagnosed as healthy class.
- FP implies that spectrogram image from healthy subject is falsely diagnosed as ASD class.
- FN implies that spectrogram image from ASD subject is falsely diagnosed as healthy class.
ROC graph is a very useful tool to visualize the reliability of the classifier which is generated by plotting sensitivity (true positive rate) in Y-axis and 1-specificity (false positive rates) in X-axis. Area under the ROC curve (AUC) is a common metric for evaluating the performance of binary classifier. AUC value always maintains the below inequalities:
(5)
It is evident from 5 that the AUC value 1 indicates the classifier has a perfect capacity to discriminate and the value below or equal to 0.5 implies that the classifier has no discriminative ability at all [47].
Result and discussion
In this section, detailed results of the proposed classification methods are discussed. This discussion is done in two subsections for the proposed two different classification process: ML based classification performance and DL based classification performance.
Results for machine learning based process
For ML based classification, performance of the proposed system is evaluated using k-fold cross-validation technique. In this process, dataset is arbitrarily divided into k subsets of equal size. Then the performance is evaluated by training the system using k-1 subsets and testing is done by using the remaining subset. This process is iterated for k times (k-fold), where each subset is used once for testing. Here, ten-fold cross-validation is used, where spectrogram images are divided into two subsets of 90% and 10% for training and testing respectively. This process is repeated for 10 times so that every image from the dataset should belong to the test subset exactly once. Finally, the results from the ten-fold are averaged to produce a single overall classification metrics. In ML based classification, six different classifiers are used: NB, LDA with pseudolinear discriminant analysis type, RF, kNN with k = 9, LR and SVM with linear kernel.
Table 1 reports the overall performance for (1)–(5) of above six classifiers. Among the six classifiers, the SVM based scheme shows the overall best performance with 95.25% accuracy while the NB based method shows the overall lowest performance with 72.09% accuracy. In case of sensitivity, RF is in the highest place (99.27%), SVM in the second highest with 97.07% and NB is the lowest position (sensitivity of 66.83%). Fig 7 shows fold-wise sensitivity for the different classifiers. On the other hand, kNN shows the highest average specificity value of 96.13%, followed by SVM with 90.95% and RF in the last position with 70.02% specificity. Fold wise specificity for different classifiers are depicted in Fig 8. Although SVM and LR are not the best performer for individual sensitivity or specificity value but overall consideration of F1 score and accuracy values, they become the top two classifier among the others. This is because although RF has highest sensitivity value (99.27%) but very poor specificity value (70.02%). Whereas kNN has highest specificity value (96.13%) but low sensitivity value (90.67%) compared to others. On the other hand both SVM and LR have kept good sensitivity and specificity value to lead their F1 score and accuracy values to 0.97, 95.25% and 0.96, 94.95% respectively. Fold wise accuracy for different classifiers are given in Fig 9.
In order to further evaluate the performance, ROC curve for different classifiers are shown in Fig 10 where curves of kNN, LR and SVM are almost overlapped. The AUC value is an index for classifier performance where lager area under ROC curve indicates better performance of the classifier. AUC values for different classifiers are given in Table 1, where SVM, LR and kNN have highest AUC value of 0.98 whereas NB has lowest AUC of 0.77. Fold wise AUC for different classifiers are shown in Fig 11.
Results of deep learning based process
As mentioned in the methodology section, three different CNN models are used to perform the classification process in DL based classification. In this process, full image dataset is divided into three parts of 70%, 15%, 15%, where 70% images are used for training the model, 15% for validation and last 15% for testing the trained model. All the models are trained with 50 epochs as because the models get overfitting after that. All the models are trained using 64 batch size with additional batch size 32, 128 and 256 is used for model 3 training to check the impact of the batch size on the model’s performance. Overall performance of the different models are listed in Table 2.
From the Table 2, it is clear that all the DL based models give better result than the ML based classifiers. Model 3 with batch size 64 produces the best result with 99.15% accuracy and F1 score 1.00. Model 1 and 2 have good sensitivity value but due to less specificity value, overall F1 score and accuracy values are lower compared to model 3. Among the batch size based different versions of model 3, batch size 128 has produced the best sensitivity of 99.60% whereas batch size 32 gives the highest specificity value of 99.10%, but overall best performance is produced by batch size 64. Batch size 256 gives the second best overall performance with accuracy and F1 score of 99.00% and 0.99 respectively. Batch 128 and 32 are in third and fourth position with 98.72% and 98.15% accuracy respectively with both having same F1 score (0.99). ROC curve for all the DL based models are given in Fig 12. Area under ROC curve for different models are also given in Table 2, where model 3 with batch size 64 and 256 have AUC almost 1 (0.99), model 3 with batch size 32 and 128 have AUC 0.98 and model 1 and 2 have AUC 0.94.
As the model 3 with batch size 64 has proven to be the best model among the proposed three DL models for identification of ASD from TD children, a detailed discussion of the loss vs accuracy graph for that model is given here. Fig 13 plots epoch wise training and validation loss vs accuracy for model 3 with batch size 64, where training and validation accuracy increases close to 100% as the losses goes down close to zero with the increase of epochs, indicating that the CNN model have found a good fit to the training data, with a lower loss value on the validation set.
The comparison of the proposed approach with existing work done using the same ASD dataset used in this analysis is shown in Table 3. Among the existing works, [39, 48, 49] used ML based classification whereas works [26, 27, 50] used DL for ASD classification.
In study [39, 48, 49], all researchers extracted features from EEG signal using Fast Fourier Transform (FFT) method. Using Regulated Fisher Linear Discriminant (RFLD) for classification, Kamel et al. [49] obtained 92.06% accuracy whereas Fisher Linear Discriminant Analysis (FLDA) was used for classification by Alsaggaf et al. [48] and Alhaddad et al. [39] to achieve an accuracy of 80.27% and 90.00% respectively. A recent study of Nur et al. used Multilayer Perceptron Network (MLPN) classification method and obtained an accuracy of 80% [50]. Djemal et al. [26] used DWT with SE on 50 seconds segment length with an ANN classifier. They achieved an accuracy of 98.60% for non-overlapping segments. Alturki et al. [27] also extracted features using DWT with SE and classified using ANN to obtain an accuracy of 98.20%. The proposed technique using spectrogram image with CNN classifier gives 99.15% accuracy which outperforms them all.
Table 4 provides an overall summery of existing ASD classification study using different datasets. Detailed information about those existing methods are discussed in Section. Since all those works used different dataset for validation, which makes it difficult to do a fair comparison with the proposed method to those works.
Conclusion
In this study, a T-F spectrogram image of EEG signal is introduced for classification of ASD from normal children. Short-Time Fourier transform based spectrogram images are generated from EEG signal and both the ML and DL based techniques are used for classification. For the ML based classification, six different classifiers are used to classify the features extracted by tCENTRIST. In the DL based process, three different CNN models are used on the spectrogram images. The results showed that SVM gives the highest classification accuracy in the ML based classification with an accuracy value of 95.25% and in the DL based classification process, the proposed CNN model achieved an accuracy of 99.15%. Finally, a comparison is made with other state-of-the-art approaches in the literature, that used the same dataset used in this analysis. The findings showed that with this outcome, the proposed approach outruns most of the techniques in existing literature and can be used as a basis for CAD system that aim to detect other neurological disorders where EEG recordings are used for diagnosis. In future studies, this technique can be used to develop CAD systems for other neurological disorders like Mild Cognitive Impairment and Alzheimer’s disease.
References
- 1. Elsabbagh M, Divan G, Koh YJ, Kim YS, Kauchali S, Marcín C, et al. Global prevalence of autism and other pervasive developmental disorders. Autism research. 2012;5(3):160–179. pmid:22495912
- 2.
WHO. Autism spectrum disorders;. Available from: https://www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders.
- 3.
(Aspect) ASA. Autism prevalence rate up by an estimated 40 Available from: https://www.autismspectrum.org.au/news/autism-prevalence-rate-up-by-an-estimated-40-to-1-in-70-people-11-07-2018.
- 4.
Speaks A. Autism Statistics and Facts;. Available from: https://www.autismspeaks.org/autism-statistics.
- 5. Siuly S, Alcin OF, Bajaj V, Sengur A, Zhang Y. Exploring Hermite transformation in brain signal analysis for the detection of epileptic seizure. IET Science, Measurement & Technology. 2018;13(1):35–41.
- 6. Siuly S, Li Y, Zhang Y. EEG signal analysis and classification. IEEE Trans Neural Syst Rehabilit Eng. 2016;11:141–144.
- 7. Yin J, Cao J, Siuly S, Wang H. An Integrated MCI Detection Framework Based on Spectral-temporal Analysis. International Journal of Automation and Computing. 2019;16(6):786–799.
- 8. Blankertz B, Tomioka R, Lemm S, Kawanabe M, Muller KR. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal processing magazine. 2007;25(1):41–56.
- 9. Siuly S, Li Y, Wen PP. Clustering technique-based least square support vector machine for EEG signal classification. Computer Methods and Programs in Biomedicine. 2011;104(3):358–372. pmid:21168234
- 10. Alçin ÖF, Siuly S, Bajaj V, Guo Y, Şengu A, Zhang Y. Multi-category EEG signal classification developing time-frequency texture features based Fisher Vector encoding method. Neurocomputing. 2016;218:251–258.
- 11.
Sheikhani A, Behnam H, Mohammadi MR, Noroozian M, Golabi P. Connectivity analysis of quantitative Electroencephalogram background activity in Autism disorders with short time Fourier transform and Coherence values. In: 2008 Congress on Image and Signal Processing. vol. 1. IEEE; 2008. p. 207–212.
- 12. Sheikhani A, Behnam H, Mohammadi MR, Noroozian M, Mohammadi M. Detection of abnormalities for diagnosing of children with autism disorders using of quantitative electroencephalography analysis. Journal of medical systems. 2012;36(2):957–963. pmid:20711644
- 13. Bosl W, Tierney A, Tager-Flusberg H, Nelson C. EEG complexity as a biomarker for autism spectrum disorder risk. BMC medicine. 2011;9(1):18. pmid:21342500
- 14. Bosl WJ, Tager-Flusberg H, Nelson CA. EEG analytics for early detection of autism spectrum disorder: a data-driven approach. Scientific reports. 2018;8(1):1–20. pmid:29717196
- 15. Eldridge J, Lane AE, Belkin M, Dennis S. Robust features for the automatic identification of autism spectrum disorder in children. Journal of neurodevelopmental disorders. 2014;6(1):12. pmid:24936212
- 16. Grossi E, Olivieri C, Buscema M. Diagnosis of autism through EEG processed by advanced computational algorithms: A pilot study. Computer methods and programs in biomedicine. 2017;142:73–79. pmid:28325448
- 17. Heunis T, Aldrich C, Peters J, Jeste S, Sahin M, Scheffer C, et al. Recurrence quantification analysis of resting state EEG signals in autism spectrum disorder–a systematic methodological exploration of technical and demographic confounders in the search for biomarkers. BMC medicine. 2018;16(1):1–17. pmid:29961422
- 18.
Haputhanthri D, Brihadiswaran G, Gunathilaka S, Meedeniya D, Jayawardena Y, Jayarathna S, et al. An EEG based channel optimized classification approach for autism spectrum disorder. In: 2019 Moratuwa Engineering Research Conference (MERCon). IEEE; 2019. p. 123–128.
- 19. Haputhanthri D, Brihadiswaran G, Gunathilaka S, Meedeniya D, Jayarathna S, Jaime M, et al. Integration of Facial Thermography in EEG-based Classification of ASD. International Journal of Automation and Computing. 2020;17(6):837–854.
- 20. Abdolzadegan D, Moattar MH, Ghoshuni M. A robust method for early diagnosis of autism spectrum disorder from EEG signals based on feature selection and DBSCAN method. Biocybernetics and Biomedical Engineering. 2020;40(1):482–493.
- 21.
Jayarathna S, Jayawardana Y, Jaime M, Thapaliya S. Electroencephalogram (EEG) for delineating objective measure of autism spectrum disorder. In: Computational Models for Biomedical Reasoning and Problem Solving. IGI Global; 2019. p. 34–65. https://doi.org/10.4018/978-1-5225-7467-5.ch002
- 22. Rim B, Sung NJ, Min S, Hong M. Deep Learning in Physiological Signal Data: A Survey. Sensors. 2020;20(4):969. pmid:32054042
- 23. Ahmadlou M, Adeli H, Adeli A. Fractality and a wavelet-chaos-neural network methodology for EEG-based diagnosis of autistic spectrum disorder. Journal of Clinical Neurophysiology. 2010;27(5):328–333. pmid:20844443
- 24. Ahmadlou M, Adeli H, Adeli A. Improved visibility graph fractality with application for the diagnosis of autism spectrum disorder. Physica A: Statistical Mechanics and its Applications. 2012;391(20):4720–4726.
- 25. Ahmadlou M, Adeli H, Adeli A. Fuzzy synchronization likelihood-wavelet methodology for diagnosis of autism spectrum disorder. Journal of neuroscience methods. 2012;211(2):203–209. pmid:22968137
- 26. Djemal R, AlSharabi K, Ibrahim S, Alsuwailem A. EEG-based computer aided diagnosis of autism spectrum disorder using wavelet, entropy, and ANN. BioMed Research International. 2017;2017:1–9. pmid:28484720
- 27. Alturki FA, AlSharabi K, Abdurraqeeb AM, Aljalal M. EEG Signal Analysis for Diagnosing Neurological Disorders Using Discrete Wavelet Transform and Intelligent Techniques. Sensors. 2020;20(9):1–17. pmid:32354161
- 28. Aslan Z, Akin M. Automatic Detection of Schizophrenia by Applying Deep Learning over Spectrogram Images of EEG Signals. Traitement du Signal. 2020;37(2):235–244.
- 29.
Yuan L, Cao J. Patients’ EEG data analysis via spectrogram image with a convolution neural network. In: International Conference on Intelligent Decision Technologies. Springer; 2017. p. 13–21.
- 30. Li M, Sun X, Chen W, Jiang Y, Zhang T. Classification Epileptic Seizures in EEG Using Time-Frequency Image and Block Texture Features. IEEE Access. 2019;8:9770–9781.
- 31. Jadhav P, Rajguru G, Datta D, Mukhopadhyay S. Automatic sleep stage classification using time–frequency images of CWT and transfer learning using convolution neural network. Biocybernetics and Biomedical Engineering. 2020;40(1):494–504.
- 32. Tawhid M, Siuly S, Wang H. Diagnosis of autism spectrum disorder from EEG using a time–frequency spectrogram image-based approach. Electronics Letters. 2020;56(25):1372–1375.
- 33. Dey EK, Tawhid M, Ahad N, Shoyaib M. An automated system for garment texture design class identification. Computers. 2015;4(3):265–282.
- 34. Wu J, Rehg JM. Centrist: A visual descriptor for scene categorization. IEEE transactions on pattern analysis and machine intelligence. 2010;33(8):1489–1501. pmid:21173449
- 35. Tawhid MNA, Dey EK. A gender recognition system from facial image. International Journal of Computer Applications. 2018;180(23):5–14.
- 36. Chang CC, Lin CJ. LIBSVM: A library for support vector machines. ACM transactions on intelligent systems and technology (TIST). 2011;2(3):1–27.
- 37. Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE transactions on medical imaging. 2016;35(5):1285–1298. pmid:26886976
- 38. Deng L, Yu D. Deep learning: methods and applications. Foundations and trends in signal processing. 2014;7(3–4):197–387.
- 39. Alhaddad MJ, Kamel MI, Malibary HM, Alsaggaf EA, Thabit K, Dahlwi F, et al. Diagnosis autism by fisher linear discriminant analysis FLDA via EEG. International Journal of Bio-Science and Bio-Technology. 2012;4(2):45–54.
- 40.
Google. Welcome to Colaboratory;. Available from: https://colab.research.google.com/notebooks/intro.ipynb.
- 41. Siuly S, Kabir E, Wang H, Zhang Y. Exploring sampling in the detection of multicategory EEG signals. Computational and mathematical methods in medicine. 2015;2015:1–12. pmid:25977705
- 42. Hassan AR, Siuly S, Zhang Y. Epileptic seizure detection in EEG signals using tunable-Q factor wavelet transform and bootstrap aggregating. Computer methods and programs in biomedicine. 2016;137:247–259. pmid:28110729
- 43. Siuly S, Yin X, Hadjiloucas S, Zhang Y. Classification of THz pulse signals using two-dimensional cross-correlation feature extraction and non-linear classifiers. Computer Methods and Programs in Biomedicine. 2016;127:64–82. pmid:27000290
- 44. He J, Rong J, Sun L, Wang H, Zhang Y, Ma J. A framework for cardiac arrhythmia detection from IoT-based ECGs. World Wide Web. 2020;23(5):2835–2850.
- 45. Alvi AM, Siuly S, Wang H, Sun L, Cao J; World Scientific. An adaptive image smoothing technique based on localization. 2020; p. 866–873.
- 46. Supriya S, Siuly S, Wang H, Zhang Y. Automated epilepsy detection techniques from electroencephalogram signals: a review study. Health Information Science and Systems. 2020;8(1):1–15. pmid:33088489
- 47. Fawcett T. An introduction to ROC analysis. Pattern recognition letters. 2006;27(8):861–874.
- 48. Alsaggaf EA, Kamel MI. Using EEGs to diagnose autism disorder by classification algorithm. Life Science Journal. 2014;11(6):305–308.
- 49. Kamel MI, Alhaddad MJ, Malibary HM, Thabit K, Dahlwi F, Alsaggaf EA, et al. EEG based autism diagnosis using regularized Fisher Linear Discriminant Analysis. International Journal of Image, Graphics and Signal Processing. 2012;4(3):35–41.
- 50. Nur AA. Autism spectrum disorder classification on electroencephalogram signal using deep learning algorithm. IAES International Journal of Artificial Intelligence. 2020;9(1):91–99.