https://orcid.org/0000-0002-8683-8810
Figures
Abstract
Deep learning architectures comprise hierarchies in modern machine learning studies, and these are not only full of semantic depth; they are also structured systematically by a sequence of nonlinear transformations. In particular, in convolutional neural networks, the activation functions are the canonical nonlinear construction blocks that control the expressiveness of the model, gradient flow in the process of back-propagation, and hierarchical feature abstraction. The canonical activation functions, which are the rectified linear unit (ReLU), the sigmoid, and the hyperbolic tangent (tanh) have non-adaptive and non-dynamic operation properties. As a result, they limit the capacity of a network to learn the complex nonlinear dynamics of a variety of data distributions and maximize the multi-level dependencies that occur among hierarchical layers. To overcome the above constraints, the current proposal proposes the implementation of Symmetric Modifiable Slope Activation (SMSA), which is a method that enables the dynamically calibrated neural layers through simultaneous modification of the slope, rang and the curvature of the activation space. SMSA is implemented in three different forms, first is slope parameter (n) is defined manually, a globally adjustable one with a learnable coefficient , and (iii) a layer-wise adaptive regime allowing each convolutional layer to learn its coefficient
). The strategies listed above were evaluated on the benchmark datasets MNIST, Fashion-MNIST, HAM10000 and COVID-19, using both lightweight and conventional convolutional neural networks architectures, including Custom-CNN, SBNet-CNN, ResNet18, VGG16 and DenseNet121. The empirical findings also support the idea that SMSA(x, n) is more suitable to increase convergence rates and to improve the learning of discriminative features in data-specific regimes, whereas SMSA(x,
) its adaptive counterpart is more suitable to provide better optimization stability, faster convergence, and better predictive performance. Indeed, using layer-wise SMSA in the SBNet- CNN model on the Fashion-MNIST data, the test accuracy of 97.52% was reached, which is, in fact, better than what ResNet18(94.18%) and VGG16(94.13%) had done. In addition, the layer-wise adaptation provided more robustness with imbalanced classes by achieving a classification test accuracy of 98.59% in HAM10000 and having a 4% positive change in the accuracy of COVID-19 classification with DenseNet121. These findings support the importance of the adaptive activation mechanisms which have a critical influence on the regulation of gradient dynamics, the development of nonlinear representations, and the process of generalization. Future studies will focus on generalizing SMSA to graph-based neural networks, transformer models, and self-supervised paradigms, which is complemented by a comprehensive theoretical analysis of gradient flow and representation geometry, which will eventually bring the state of the next-generation neural network design.
Citation: Raza A, Ali A, Ullah S, Rehman B, Ali Z (2026) Development of adaptive activation functions with curvature and range modulation for abstract image feature learning in complex CNNs for cross-domain applications. PLoS One 21(8): e0355613. https://doi.org/10.1371/journal.pone.0355613
Editor: Zahid Mehmood, University of Engineering and Technology Taxila Pakistan, PAKISTAN
Received: October 11, 2025; Accepted: July 23, 2026; Published: August 14, 2026
Copyright: © 2026 Raza et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The implementation code used for the proposed activation function and experiments will be made publicly available in an online repository to ensure reproducibility. All data required to reproduce the results, including values used in graphs and performance tables, are included within the manuscript https://github.com/Aliraza927/Aliraza/blob/984cd2301c2fc3b0e7d7f49df4c2357dd220467c/SMSA1_WOR.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist regarding this publication.
Introduction
Deep learning was a revolutionary factor in enhancing artificial intelligence in the application of artificial intelligence in healthcare, medical imaging, and automated diagnostic systems. It allowed end to end learning of complex visual data, which greatly contributed to accuracy, flexibility, and clinical decision support. However, challenges such as vanishing or exploding gradients, unstable convergence, and weak generalization continued to limit the performance of deep neural networks, especially on sensitive and heterogeneous datasets [1]. With continuous, bounded, and differentiable activation functions, deep learning neural networks depict exceptional capability to approximate complex and nonlinear function between image data and true image categories. Based on the Universal Approximation Theorem, deep learning architectures can achieve arbitrary levels of accuracy when given a depth and properly organized activation dynamics. Such a theoretical background lies behind their exceptional results in a broad spectrum of visual recognition tasks, both generic performance evaluation metrics and more critical medically important visual imaging tasks [2]. Activation functions are significant in determining the complex non-linearity aproximation function and learning dynamics of neural networks, which promotes the constant development and optimization of activation learning strategies, particularly those that extend adaptability and expressivity [3]. Deep learning models are nowadays being used in high-rank image recognition and sensitive classification issues in which the relationships between inputs and outputs are very nonlinear and hard to approximate [4]. Traditional activation is prone to saturation which causes vanishing gradient making deep training of the network difficult. Functions that are not sharp like Softsign and Arctangent functions find it difficult to reproduce complex or high-frequency features. Activations are very flat such as ArcSinh and Gudermann, which offer poor sensitivity about the origin, leading to slow convergence. Unstable updates can be caused by too sharp transitions in highly steep functions such as a Logistic (with large beta). These constraints make learning less effective and limits the scope of the model to generalizing to a variety of data [5]. In order to overcome this difficulty, it is necessary to come up with adaptive activation functions having trainable parameters. These functions are supposed to provide dynamic control of important properties such as nonlinearity, limitedness, saturation and gradient action. By instantiating these adaptive mechanisms in the learning process using parameter optimization, neural networks can produce more precise, data-sensitive approximations and be more effective at problems with different and complicated data distributions.
The paper has engaged in a detailed research on the development, testing and empirical evaluation of a family of activation function named as Symmetric Modifiable Slope Activation (SMSA). The three distinct configurations of these functions have been presented including manually tunable, globally adaptive, and layer wise adaptive and the purpose is to make convolutional neural networks expressive and increase their learning capacity. First fixed slope form of SMSA has allowed an explicit control over the nonlinear response by means of a static parameter and has also shown dataset specific optimal behavior. This shape has made converging it by tuning activation sensitivity based on the statistical properties of the input distribution stable. In further enhancement of the learning dynamics, an adaptive S shaped version of SMSA has been formed by incorporation of a learnable parameter, which is the , that has been self-regulating the slope of activation and curvature in the course of training. This parameterization has maintained gradient flow across layers, alleviated vanishing gradient problems, and added smooth convexity that is consistent with the hierarchical nature of feature abstraction of deep architectures. Besides the theoretical design, a structurally efficient CNN has been created named SBNet that has been built with a moderate budget of parameters and has been tested in the Fashion MNIST data set to confirm the effectiveness of SMSA when limitations are held on model complexity. Additionally, the adaptive SMSA functional has been implemented on various high similarity and texture rich datasets such as MNIST, Fashion MNIST, HAM10000 and the COVID19 Radiography Dataset. These datasets present significant challenges of class separability because of slight interclass fluctuation. There have been experimental results that SMSA has eased the establishment of stronger decision boundaries that have allowed deep networks to achieve greater discriminative power and enhance generalization. There has been extensive cross validation and the results of classification have been compared systematically with that achieved by the use of the standard activation functions including ReLU, ELU, Swish, Mish and Leaky ReLU in various architectures. The adaptive variants of SMSA have always demonstrated better convergence behaviour, better stability in the optimisation process and accuracy on unseen data. These findings support the possible power of SMSA as an effective, data receptive activation scheme that can be used with deep learning models that process complex visual data.
Literature review
Convolutional neural networks (CNNs) built on deep learning technologies have profoundly influenced the evolution of intelligent systems, especially in the spheres of healthcare, autonomous technologies, e-commerce, and medical diagnostics as well as in a vast number of industrial and commercial applications. In these areas, CNNs have helped in automated feature extraction, image-based diagnostics, real-time decision-making, and large-scale pattern recognition with minimal human intervention [6,7]. Although such successes have been achieved, it is frequently the case that large-scale annotated datasets are inadequate in practical applications. Most practical cases only have access to relatively small datasets, which is restricted to issues of privacy, annotation can be expensive, and domain-specific factors are limiting to size [8,9]. As a result, CNN designs in these applications are now being geared towards the ability to be small, data efficient, and can generalize well and show high accuracy even when limited data is available. The dimensions and magnitude of the dataset have played a significant role in determining the structural design of CNNs especially, its depth and number of parameters [10,11]. Traditionally, deeper CNN models have been used when there are large datasets, and high resolution and hierarchies of features may require more than just deep layers to support effective representation learning [12]. On the other hand, small or moderate-sized datasets may also have the best performance with lightweight CNN models that have fewer layers and fewer parameters that minimize overfitting and decrease computation requirements [13]. This correlation between model size and dataset size has been repeatedly confirmed by recent empirical research. The convolutional operations, implemented in deep learning systems, have resulted in the development of deep CNNs, now becoming key in image recognition tasks in a variety of application areas. These architectures are specifically crafted to grow with data complexity and resolution, so that they are especially useful in high-stakes applications like healthcare and autonomous systems. CNN models have been improved over time as they concentrate on depth optimization, parameter efficiency, and inference speed to satisfy the requirements of modern applications in terms of performance and deployment. Empirical evidence has also pointed out that parameter size has a strong effect on the predictive power of deep CNNs. Indicatively, Kainat and Mujeeb [14] tested ResNet50 on CIFAR-10 and Fashion-MNIST, where they reported higher accuracy of 86% on Fashion-MNIST compared to 73% on CIFAR-10 despite having an imbalanced data. Their findings highlighted the significance of balancing of datasets, regularization of datasets and hyperparameter optimization of datasets with the intention of improving generalization and minimizing computational expenses. Aradea et al. [15] proposed AdaptiveSpinalNet, a hybrid CNN, which combined adaptive kernels with SpinalNet, whereby their VGG-5 + AdaptiveSpinalNet model had 95.21 percent validation accuracy with just 1.1M parameters on Fashion-MNIST, a smaller dataset. Equally, Raza et al. [16] introduced a low-complexity O-CNN with NGNDG-AF activation function, which achieved 99% training and 98% validation accuracy on HAM10000, as compared to refined models like ResNet152V2. They also incorporated explainable AI techniques such as Grad-CAM and Grad-CAM++ in their model to increase its interpretability, thereby increasing its likelihood of clinical adoption. Al Mahmud et al. [17] optimized CNN with augmentation methods on HAM10000 and obtained 97.78% diagnostic accuracy and 97.9% precision, recall, and F2-score. Simultaneously, Hussain and Shouno [18] successfully designed the MAGRes-UNet, a framework that adds Multi-Attention Gates (MAGs) and residual links to extract fined features in medical image segmentation, the MAGRes-UNet architecture, and provided its results in medical image segmentation and regression. Their model, which was trained on CE-MRI datasets and HAM10000 datasets using Mish and ReLU activations using Adam and AdamW optimizers, significantly outperformed U-Net and ResUNet. In particular, it has reached 99.94% accuracy, 98.29% iOU, and 97.75% Dice on CE-MRI and 99.71% accuracy, 97.83% iOU, and 97.36% Dice on HAM10000. All of these findings point towards one thing, which is that CNN performance and efficiency are highly dictated by compatibility among the architectural depth, parameter size, and dataset complexity. The essence of these developments is the connection between nonlinear activities and linear changes. Although the structure of neural networks is built on affine operations, the nonlinear activation functions allow the neural network to learn and model complex hidden patterns in the data. The Universal Approximation Theorem is a proposition that the neural network with the correct number of neurons and activation functions can be utilized to approximate any continuous function within any desired accuracy. This makes the role of activation functions not only the helper functions but also central enablements of expressiveness, flexibility, and representational strength of modern CNN architectures [19]. Activation functions have a far reaching effect on various parameters of neural network dynamics, such as gradient flow, the smoothness of optimization, convergence, and the stability. They prove to be especially crucial in the context of sensitive fields like medical image analysis, where the pattern is usually faint, and datasets are extremely diverse. In this regard, the choice of activation functions is also significant in determining model performance. Functions that offer gradients that are smooth, functions with lower bounds or functions with adaptive characteristics are particularly effective in enhancing the stability of the training and the consistency of the outcomes when using generalization [20]. The paper by Ullah et al. [21] methodically investigated various adaptive activation functions on the Fashion-MNIST data set with VGG16. Their experiments found Tanhsoft-3 and AHerfReLU to be the most accurate with a validation accuracy of 93.91% and 93.92%, respectively. The Smish (93.53%), TanhLU (93.41%), Pserf (93.29%), and Serf (93.16%) architectures were also competitive, which demonstrated the helpfulness of tunable activations to enhance the generalization capacity of CNNs. More developments to this problem were introduced by Pandey and Srivastava [22] who introduced the Rectified Tangent Activation (RTA). RTA has performed better than traditional activations (like ReLU, Swish, ELU and Tanh) across datasets like CIFAR-100, Fashion-MNIST and Chest X-ray. It provided better accuracy, recall, F1-score and validation rates in ResNet18-based evaluation, getting 91.76% on CIFAR-100 and 78.84% on Chest X-ray. Such outcomes also underscored the flexibility, effective convergence and categorizational power of RTA. Similar results were reported by Nanni et al. [23], who showed that it is possible to significantly improve the biomedical image classification by changing the range of activation functions in CNN backbones. Substituting ReLU with other frameworks like SReLU, APLU, MeLU, GaLU, Mish, Swish and PDELU in ResNet50 resulted in significant performance improvements on 13 medical datasets. The wMeLU(255) model was the best stand-alone with the highest average accuracy of 85.27% compared to the standard ReLU baseline (84.55%). In addition, further improvements were achieved with ensemble approaches which relied on stochastic activation selection where StoFullAS15(255) attained the highest average accuracy of 90.31%. These results substantiate the hypothesis that diverse activations (layered) ensembles always better perform compared to fixed-activation counterparts in biomedical imaging tasks. Further contributions came out by Biswas et al. [24], who suggested the activation functions of SMU and SMU-1. The designs are admimations of ReLU, Leaky ReLU, Maxout and GELU with increased smoothness and flexibility. Replacement of ReLU with SMU showed significant improvements on CIFAR-100 Top-1 accuracy, improving by 6.22%, 3.39%, 3.51%, and 3.08% when used with ShuffleNet V2, PreActResNet-50, ResNet-50, and SeNet-50, respectively. This direction was developed further by the works of Harmon and Klabjan [25]. They described an approach based on an activation ensemble. Their approach allows several activation functions to co-exist in every layer with weight parameters of interest being dynamically optimized to select the most effective activation per neuron. This multi-activation approach improved the performance of a variety of architectures and datasets, such as 99.34% to 99.40% improvement of MNIST CNNs and 95.16% to 96.28% improvement of ISOLET FFNs. These advances confirmed the strength of activation ensembles as an effective tool of enhancing plasticity and learning ability in neural networks. Jagtap et al. [26] proposed both neuron-wise and layer-wise adaptive activations to enhance the convergence and performance of deep and physics-informed neural networks, in which a scaling-factor is a learnable parameter of each activation, which the network can optimize by training. Their approach with an extension by a slope-recovery term in the loss function does not contain sub-optimal critical points, and their gradient flow conditions implicitly, without any explicit matrix work, thus providing more stable and faster optimization. The superiority of the Mish and AdamW configuration was statistically established by a T-test with the p-values being significantly less than 0.05 in all significant metrics. These findings indicate that the combination of attention mechanisms and adaptive optimization can greatly improve CNN on complex medical image classification tasks [27,28]. A significant weakness, however, still exists: most activation functions are either convergence efficient, or classification accurate, but do not typically both at the same time across a variety of architectures and datasets. This limits scalability and compromises reliability in high stakes areas like clinical diagnostics. In general, it shows that combination of transformation mathematically based, expressive activation functions, and hierarchical learning features enable CNNs to categorize highly complex image datasets with good accuracy. Concurrently, the combination of dense and rich layers can enhance further performance, but the process of determining the best configuration may be complex, and one may need to perform and optimize a large number of experiments. Such a process not only makes computations more expensive but also makes the development of models more time-consuming in the real world. Going forward, it is presumed that future developments will depend on the synergetic interaction of mathematical models, architectural research, and adaptive nonlinearities. These advancements are required to attain resource efficient, high-accuracy, and clinically accurate deep learning solutions.
Materials and methods
The Symmetric Modifiable Slope Activation (SMSA) is a curvature-based, nonlinear activation-function specially made to meet the intractable problems of gradient attenuation and limited adjustability which plague traditional activation functions. Two complementary instantiations of SMSA have been described; each is parameterised by an integer n, which controls the slope, and an adaptive variant, which adds to the process a hyper-parameter b, which controls curvature, boundedness and gradient sensitivity. The activation has been carefully mathematically tested in order to rule out the possibility of vanishing or explosive gradients. Empirical studies indicate that it is differentiable and smooth, and thus allows the stable propagation of inputs with a wide range of magnitudes. It is the first derivative of which is continuous, and the criteria of smoothness of higher order are met, so that gradient-based optimisation algorithms can exploit local error surfaces in a way which does not have to face singularities or discontinuities. A layer-wise adaptive approach has been used to improve the flexibility of the representation. The convolutional layers choose their SMSA profile autonomously in an optimal manner, which generates a personalised dynamical behaviour through the feature hierarchies. The combination of these mechanisms, which is synergistic, results in a much expressive model. Experimental pipeline was run on GPUs-accelerated hardware, hence making it easy to train large models, converge fast and be computationally tractable. First, the mechanism was tested on a Smart Block Network (SBNet) applied to the Fashion-MNIST dataset and allowed analyzing the convergence properties and generalisation.The SMSA has been used in future works to apply to classical deep learning backbones, including VGG16, ResNet18 and DenseNet121, both in fixed and adaptive configurations. All models have been trained in the same framework of the experiment, to enable an equal comparison, where the metrics of performance have been measured based on classification accuracy, the dynamics of loss convergence, as well as the generalisation gap, thus, giving information on the strength and stability of the obtained architectures.
Mathematical formulation of S shape activation function
The Symmetric Modifiable Slope Activation (SMSA) operation has been developed in two different versions a manually controlled one, called as , and an adaptive version called as
. They are both characterized by symmetric, piecewise-smooth expressions that are meant to ensure that differentiability and gradient propagation are stable. The learnable parameter, which is illustrated as a slope and curvature modulation, can be dynamically modulated during the training process. Such fundamental properties as boundedness, adaptability, and gradient preservation have been well-developed and demonstrated in practice. According to the definition given in Eq (1), Symmetric Modifiable Slope Activation (SMSA(x, n)) is the solution to the shortcomings of the traditional activation functions, including Sigmoid, Tanh, Softsign, Arctangent, ArcSinh, Gudermann, Algebraic Sigmoid, and Logistic, as it maintains a symmetric shape around the point of x = 0, with an explicit slope control via the fact that the slope shape is regulated by a single parameter, which is often denoted n. This provides detailed control of gradients hence improving optimization dynamics in training.
The variable x represents the input value, while n > 0 is a scaling parameter that controls the slope and symmetry of the function in Eq (1). Secondly the proposed Adaptive-SMSA function Eq (2) is defined as
where is a tunable parameter and sign(x) is the signum function defined by
Fig 1 below demonstrates clearly that the parameter n in the proposed activation function entails a very specific variational control of the slope to the proposed activation function which is SMSA(x,n). The zero-gradient domain is concurrently formed and the area of the exponential exploration of the nonlinear response is carved by modulating n. Similarly, Figs 1 and 2 illustrate the control of concavity, range, and total non linearity by the learnable parameter of the adaptive form of SMSA, namely, . The process of fine-tuning of the value of beta therefore makes the model more flexible to diverse
network complexities and network depths.
The SMSA function exhibits fixed boundedness in Eq (4), saturating at
regardless of the parameter n, as verified in Fig 1. In contrast, the adaptive form
demonstrates dynamic boundedness in Eq (5), saturating at
, where
is a learnable parameter. This dependency on
enables flexible control over the activation range, allowing the network to adjust its nonlinearity during training, as verified in Figs 1 and 2. The analytical formulation of the derivative of the
function is given by:
The first derivative has been computed for
. The identity (x2) sign(x) = |x|x holds for all nonzero x and has been utilized to simplify the expression.
then, the first derivative is for x > 0, |x| = x, so:
for x < 0, , so the result is
The derivative of versus the independent variable x, as shown in the Eq (6) reveals that the slope intensities are controlled by a single parameter namely the parameter n. This kind of dependency offers explicit control of the gradient flow of the network. Adaptive formulation
has its derivatives described in Eqs (8) and (9). These derivatives are a dynamically varying entity, which can be dynamically varied by changing the parameter
during training, and can therefore enable the network to be per-layer modulators of the gradient. Convergence and optimisation of this type of modulation are possible. Curvature of SMSA is convex at the position where x = 0 and this facilitates a very favourable and effective gradient propagation at the early stages of training. As the magnitude of the input increases, the SMSA asymptotically switches to a concave regime thereby dispensing the risk of gradient explosions or saturation effects. It implies that the adaptive curvature process renders deep convolutional neural networks more robust because they have good learning dynamics and do not encounter the common phenomena of vanishing or exploding gradients in deep architecture.
On the inputs that meet the condition , the second-order derivative
is positive, which proves that the function is located in a convex regime. On the other hand, when the value of x exceeds the
of three, the second derivative becomes negative, meaning that the shape is concave. Such a concavity/ convexity transfer gives SMSA the capability to strongly adapt the sensitivity of the gradient, and hence to promote stable optimisation in deep network designs.
The second derivative of the adaptive SMSA has the second term in the Eq (11), which has been used to obtain its curvature as shown in the equation below: In case the input is such that it satisfies: the numerator:
is strictly positive and hence the second derivative is
, and thus the deriving is conclusive as to a convex behaviour. Conversely, at a value of
, the value of
is negative, and
, thus exhibiting concave behaviour. The inflection point, at
, where convexity changes to concavity, as
. This learnable parameter controlled adaptive curvature modulation allows the SMSA to adapt flexibly to nonlinear patterns by dynamically changing the convexity and concavity in the course of the optimisation process.
The proposed combineds function, as illustrated in Table 1 and Fig 1, generalizes classical S-shaped activation functions by skillfully implementing at least Sigmoid, Tanh, Softsign, and similar functions by a single tunable parameter, n. This flexibility allows the nonlinearity to be under control dynamically thus making the gradient flow rapid and stable convergence in training. This in turn increases the learning capacity and optimisation of deep neural networks of the SMSA.
Layer-wise adaptive learning strategy
In modern deep learning studies, it has been established that neural networks have architectures to discover hierarchical encodings of features, where early layers learn simple patterns, and later layers learn more abstract semantic features [29,30]. Both the nonlinear transformation of each constituent depth and, as a result, the inherent properties of the activation function can have a controlling effect on the abstraction properties of the network, and the geometry of representations at that depth. It is on this basis that in order to realize the representational potential of each stratified layer to a full extent, nonlinearities specifically designed to display curvature, smoothness, and flexibility characteristics relevant to the needs of the functional requirements of the layer in question are essential. The adaptive parameter of each convolutional layer is a convolutional layer-specific parameter, namely, the parameter, denoted by , thus permitting the layer-specific non-linear modulation process that refines the abstraction of the features gradually, starting with fine-grained local patterns, and progressing to more complex and high-level semantic features. This type of dynamic adjustment has been shown to enhance classification accuracy in problems which are sensitive to deep, hierarchical representation learning, such as object recognition, structured image analysis, etc.
The network has a specific, learnable parameter assigned to each layer, namely, the l layer of the network, denoted as, in a given mini-batch training epoch, the ith pass through the network, the learnable parameter is denoted as . It is a parameter that governs the non-linear dynamics of the activation function, ‘
in the layer. The value of each of the activation parameters is modified to a new one, i.e.,
, as the process progresses to the next epoch,
. Therefore, the nonlinearity injected by the functional, namely denoted as
changes over training epochs hence facilitating the network to re-adjust its nonlinear transformation properties in a direction that reflects the current direction of the learning activity. The Stochastic Mapping Search Algorithm in Eq (2) with Eq (12) as a whole is a probabilistic meta-heuristic optimisation framework. It searches a complex search space systematically by transforming stochastically and adapting mapping models, hence finding convergence to either optimal or near-optimal solutions by a prudent tradeoff of exploratory and exploitative search.
Architectures: SBNet-CNN, Optimized CNN, ResNet18, VGG16 and DenseNet121
This article has conducted a methodical alteration of the conventional convolutional neural network (CNN) systems to determine the effectiveness of the proposed Symmetric Modifiable Slope Activation (SMSA) on various data. Specifically, a multi-block CNN [16] was used on HAM10000, ResNet18 and VGG16 on Fashion-MNIST, and DenseNet121 [31] on X-ray images of COVID-19. In both scenarios, the regular activation functions were replaced with SMSA variants, and adaptive strategies, such as the layer-wise and global schemes, were added to increase the flexibility of the training. In the case of MNIST, a lightweight CNN was trained using SMSA to achieve computational efficiency, and the SBNet -CNN architecture was used in Fashion-MNIST to incorporate layer-wise adaptive SMSA to learn scalable nonlinearity. The SBNet- CNN in Tables 2–4, to be described with a 593,665 parameter 256-filter convolutional block, which was then followed by adaptive SMSA, pooling, and dropout layers. Later intermediate layers added further 1.77M parameters to the initial set of feature layers by repeated 256 channel convolution, to increase feature separation. The fully connected stage was then flattened into a 2304-dimensional vector then comprised of a 64-neuron layer with 147, 520 parameters and a 128-neuron layer with 8, 320 parameters, both with SMSA, batch normalization (512 parameters) and dropout. The network ended with a 10-class Softmax output whose output layer has 1,290 parameters. Generally, SBNet -CNN repeatedly used SMSA at every layer, which combined adaptivity, nonlinearity, and regularization to produce strong performance on Fashion -MNIST.
In Table 5, we have presented an architecture that uses a DenseNet121 backbone with adaptive activation of SMSA to categorize the COVID-19 radiographic images. A foundational model, which is instantiated without the use of pretrained weights, has been parametrized, and the rest, having the final twenty layers, have been parametrized with globally shared learnable coefficients of the nonlinearity (in the form of ), thus allowing the nonlinearity to be modulated across the network. This core has been followed by series of fully connected layers with each being followed by an SMSA transformation and dropout operation in order to support task-specific feature extraction. Such a composite layout allows a controlled gradient spread and fast convergence in the presence of complicated medical pictures, due to the integration of global flexibility in the deeper layers of the network.
The CNN customized to the present study is a simplified version of Fig 3 of this work, keeping the number of layers and filter size constant. It is made of convolutional layers, each followed by SMSA(x, n), which are separated by max-pooling and finally by fully connected layers, which are task-oriented to the classification problem.
Mathematical formulation and design of custom CNN architecture for MNIST dataset.
The MNIST dataset consists of grayscale images with a spatial resolution of . Each image can be represented as a two-dimensional matrix:
where denotes the grayscale pixel intensity normalized within the range [0,1].
where denotes the input feature vector, and
,
, W(k) represent the trainable weight matrices at successive layers, with
,
, and b(k) as their respective biases. The adaptive SMSA activation function fSMSA is applied element-wise in each hidden layer, enabling non-linear transformations. The terminal output is derived through the use of the Softmax function that normalizes the raw logit values into a probabilistic distribution of classes, hence, making it possible to make multi-class decisions using the function with a high discriminative power. In order to describe the forward-propagation dynamics of the proposed architecture, it is convenient to write down the final output concisely as:
Experimental setup
This section presents the datasets, network architectures, training configuration, and computational resources used to evaluate the proposed Symmetric Modifiable Slope Activation (SMSA) and its adaptive variant. The overall aim is to perform a stringent test of behavioural properties of the model on a range of convolutional neural network architectures and test datasets, both general image classification and medical imaging specialised tasks.
Datasets included MNIST, Fashion-MNIST, HAM10000, Covid19.
The data represented by images that are used in computer-vision activities are by definition multi-dimensional arrays, with each dimension addressing either spatial, chromatic or intensity concepts that are necessary to visual representation. By feeding these high-dimensional image tensors to deep CNNs, the models are approximated to the respective labels by learning hierarchical feature patterns successively in each layer. However, due to the visual similarity that is common in pictures of different classes in a given dataset, classification problems may end up being more and more difficult, requiring strong feature-extracting and discrimination approaches. Four separate datasets were used in order to assess the generalizability of SMSA comprehensively:
- MNIST: An extremely popular benchmark containing 70, 000 grayscale images of handwritten digits (60, 000 to be used during training, 10,000 to be used during testing), each of size 28x28 pixels, and grouped into ten digits categories.
- Fashion-MNIST: Fashion-MNIST has a similar format, but images of fashion products (e.g., shirts, shoes, bags) of ten categories, a more challenging task, as the similarity between classes is greater.
- HAM10000: The Human-Against-Machine dataset with 10000 training images, which is composed of 10015 pigmented lesion dermoscopic images of 7 classes. Its medical significance and biased classes make it an attractive standard of real-world application in healthcare. Table 6 contains the information on the preprocessing pipeline that consisted of oversampling methods, normalisation measures, systematic dataset reorganisation, and sequential train-test partitioning procedures.
- COVID-19 Chest X-Ray Dataset: Collection of X-ray images, which are thoroughly checked and include confirmed COVID-19 pneumonia, other pneumonia cases, and normal pulmonary scans. Images were resized to
and augmented with rotation, flipping, and brightness scaling to reduce overfitting. The preprocessing details of COVID-19 Chest X-Ray Dataset in Table 7
The MNIST, Fashion-MNIST, HAM10000, and the COVID19 HR dataset collections offer a unique set of classification tasks that are challenging due to the different class compositions, the hidden similarities among classes and the complexity they possess. MNIST and Fashion-MNIST are balanced by nature, whereas HAM10000 and COVID-19 HR are highly imbalanced, which poses more challenges to both training and evaluation. These nonhomogeneous data distributions provide an effective testing environment in evaluating the flexibility and representational capability of the suggested SMSA functions. SMSA has demonstrated a remarkable capability to adjust the non-linearity control parameters, n and , on-the-fly, which allows convolutional neural networks to be accurate approximators in a repertoire of tasks. The ability allows establishing class-specific but overlapping decision boundaries, which is critical to the modeling of the subtle differences to occur in complex datasets. It follows that SMSA has a wider range of applicability by increasing CNN generalization, even when there is high ambiguity in classification.
Theoretical analysis on categorical cross entropy loss, gradient flow and optimization smoothness.
The sparse categorical cross-entropy loss function is frequently used in multi-class classification tasks where target labels are represented as class indices. It measures the cross-entropy between the true label distribution y and the predicted probability distribution p, where p is obtained through the Softmax activation function. The vector y typically encodes the correct class as a one-hot vector, while p provides the model’s estimated probabilities for each class. This loss function has been derived from the general cross-entropy formulation and serves as a standard choice for classification problems involving mutually exclusive classes. Sparse Categorical Crossentropy Loss is defined as
from Eq (14) and defined as
the chain rule is again applied to compute the gradient of the loss with respect to the parameters of the last hidden layer, as shown in Eqs. (14) and (15). the weighted sum is given by . Its gradient with respect to the weight
is
, and with respect to the bias
, it is 1. Use results from Eqs. (1)–(16).
Training details, hyperparameters, optimizers, schedulers.
Table 8 outlines the complete training protocol for mentioned CNNs model equipped with Adaptive SMSA activation on the MNist, Fashion MNist, Ham100000 and COVID-19 datasets. The training process has utilized the Adam optimizer with an initial learning rate of , which is annealed using a cosine scheduler to ensure smooth convergence. Categorical Cross-Entropy has been employed as the loss function to minimize classification error across multiple classes. To stabilize gradients and improve optimization, gradient clipping and weight decay have been integrated. Further, the SMSA activation mechanism involves an adaptively modulated
-parameter, and the time dependence of the
-parameter is recorded during training thus the time dependence gives the non-linear dynamics of adaptation in an efficient form.
Evaluation metrics: Definitions and mathematical formulation.
The effectiveness of the suggested activation in CNN frameworks was measured using six classic measures of performance listed in Table 9. These measures are accuracy, precision, recall, F1-score, convergence rate and computation cost. Accuracy refers to the percentage of correct findings in general whereas on the other hand, precision and recall measure the ability of the model to reduce false positive and false negative findings, respectively. The F1-score provides a harmonised approach because it combines precision and recall. The convergence speed is a measure of the speed with which the model converges in training, and computational cost is a measure of the efficiency of training as indicated by the training time, memory requirements, and floating-point operations. Taken together these measures provide the overall assessment of the faithfulness of classification, dynamics of learning and the efficiency of resources.
Results and discussion
This part provided a rigorous examination of the empirical information that corresponds to the mechanism of Symmetric Modifiable Slope Activation (SMSA) and, through it, project our findings onto the current literature on the design of activation functions to deep learning. The empirical experiment, which entails an all-encompassing gathering of benchmark exercises, will likewise demonstrate a constructive variation in the performance as compared to the canonical nonlinearities such as ReLU, ELU, and Swish. This improvement is apparent in shallow convolutional designs as well as in deep residual networks therefore attesting to the flexibility of SMSA. Besides the conventional performance metrics, there is an underlying sensitivity analysis of gradient slope hyperparameter, which is directly proportional to the gradient flow, which in turn influences the speed of convergence and the generalization capability. The relationships between hyperparameter optimisation and learning dynamics have a complex relationship which is explained by this observation. Finally, the paper describes the efficiency of SMSA through a sequence of statistical perturbation experiments, which meticulously shows how it works in a wide spectrum of architectural paradigms. The overall findings confirm the value of the curvature-managed activations in the formation of representation-learning paths and in stabilisation of the optimisation in the convolutional neural-network settings.
Ablation study on gradient flow dynamics, range modulation, and nonlinearity characteristics
The graphics presented in Figs 4–17 have been summarised in Table 10 and reflect the experiment results of the (Eq. 1)’s slope control parameter n across the MNIST, Fashion-MNIST, and HAM10000 databases. The tuning of n has continuously determined convergence behavior, performance (computational efficiency), and generalization (training-validation accuracy curves that overlap) performance at all sets of data.
Changes in loss and accuracy trends are observed during training and validation.
Variations in training and validation loss and accuracy are observed as n increases.
Changes in training and validation loss and accuracy are observed as n varies.
Variations in convergence behavior, loss, and accuracy are observed as n increases.
For the MNIST dataset, Figs 4 and 5 and Tables 11 and 12 have shown that the parameter n possesses a broad optimal range from 0.6 to 10, within which convergence has remained rapid and gradient stability has been preserved. However, excessive gradient exploitation at lower values of n has marginally affected generalization. When the convolutional neural network is used with low values of n, it is highly likely to overfit, Memorizing the details of the training images instead of capturing generalised, clustering properties that define classes. As such, the model achieved high training accuracy but not be able to maintain performance when seen on unseen data, which leads to a significantly lower validation accuracy than the training accuracy, although overall classification performance has remained robust within this interval.
In the case of the Fashion MNIST dataset, Figs 6 and 7 and Tables 13 and 14 have shown that the optimal range has significantly narrowed to . Within this interval, the model has achieved accelerated learning and enhanced generalization. At very small values of n, gradient exploitation does not accelerate convergence resulting in a decline in predictive accuracy. In contrast, values of n that are too large trigger a slowing of the convergence process that does not offer further generalization performance improvements, and hence, a dataset-specific trade-off is indicated. The observations highlight the fact that the SMSA (Eq (1)) mechanism is highly sensitive to the scaling parameter. As a result, it has made careful calibration of this parameter crucial to the maintenance of stability in the performance conditions of a variety of learning situations. HAM10000 collection is an interesting case study of high-dimensional classification literature, with seven clinically important dermatoscopic classes, which have a significant level of overlap of features.
A carefully fine-tuned convolutional neural network, used in combination with SMSA (Eq (1)), has converging training and validation accuracy curves, especially when n is in the range of 0.8 to 2.0. In this interval, CNN indicated high generalization and opposed to memorization, thus demonstrating that SMSA supports a nonlinear approximation of the complex class structure. In the case of the HAM10000 dataset, Table 15, Figs 8–10 illustrate that the optimal results of parameter tuning lie between the range of 0.8 and 0.2. The model is always characterized by the gradual degradation of its gradient, optimal convergence, and high classification rates within this range. Besides, the differences in n within this spectrum increase robustness to the inherent visual similarities of the dataset, which also maintains generalisation without undermining convergence dynamics. The empirical evidence supports the formulated hypothesis that the slope-control parameter acts as a key modulator of gradient dynamics, and thus, enables adaptive control of the convergence rate, the computational costs, and the generalization ability overall of the model to a wide range of complexities of data-sets. Also, the CNN is stable in terms of training over large epochs, without the tendency of gradient vanishing or over-fitting. The behaviour of this nature highlights the effectiveness of the SMSA, represented as in Eq (1), in trade-off representational power with computational efficiency. The influence of the SMSA function’s slope parameter in Eq (1) on the optimization performance of optimized CNN has been systematically evaluated using the HAM10000 skin lesion dataset. Extensive experimentation has revealed that low values (0–0.3) have caused sparse and unstable gradient propagation, resulting in ineffective feature learning and poor classification accuracy, as illustrated in Figs 6, 8 and 9. The key findings are that a big disjuration between training and validation accuracy suggests that the convolutional neural network is mainly memorizing data peculiarities rather than generalising. Conversely, overlapping training and validation curves indicate that the network has acquired abstract transferable features. In addition, the performance of classification is related to the parameter n, the slope of activation. Typically, neural network models with moderate number of parameters give good performance with particular data sets of different complexities.
Moreover, the AUC scores in this range have remained consistently low, with significant class overlap and indistinct decision boundaries, as evidenced by Fig 15. The confusion matrix in Figs 11–13 has further confirmed elevated misclassification rates due to inadequate feature separability. Figs 16 and 17 have demonstrated that the slope parameters have significantly improved the class approximation confidence levels in the optimized CNN. It has also been noted that the network has been more prone to memorize features instead of learning abstract representations in the absence of proper slope control because of over exploration of gradient. This has been the main cause of its continually low levels of confidence on the prediction of classes. The confusion matrix provides a class based perspective of the model performance by indicating the classes the model is classifying well and those that are difficult due to overlapping features. It points out where the model has difficulties with making clear distinctions. The SMSA(x,n) activation gets better separation of the classes by controlling the gradient using slope parameter n to give finer decision boundary between the problematic classes.
Conversely, medium slope values (about n = 1–4) have facilitated steady gradient flow which has facilitated better learning of representations. This has been translated to sharper class separation, better AUC in Fig 15 and better classification reliability. The curves of ROC of this regime have proved to be more sensitive and specific and this also swayed toward better suitability of these n values. The findings show that slope modulation can be significant to specify network dynamics and to optimize decision boundaries to complex visual recognition tasks such as skin lesion classification. On the other hand, a moderate range of slope (0.6–2.0) has supported strong and steady gradient flow that can ensure the CNN converges effectively and has higher test accuracy and significantly better AUC based on all classes. The confusion matrix in Figs 11–13 in this optimum range (0.6–2.0) has recorded clear and well separated boundaries of classes with minimum errors in classification. Nevertheless, in cases where the slope values have been overdone (3–10 and above), gradient saturation has already been experienced and the gradients have vanished around zero indicating the eventual stoppage of learning. This has caused stagnated network optimization, leveled accuracy and deterioration in the AUC performance. The results indicate that adaptive slope modulation inside SMSA Eq (1) is essential to the control of the learning dynamics of deep CNNs, particularly on complex data such as HAM10000, where accuracy maximization, increasing AUC values, and reducing errors on the confusion matrix are essential and discriminative classification results are found.
The adaptive evolution of the parameter across multiple layers, as guided by SMSA with Eq (2), has been effectively illustrated in Fig 14 for SBNet-CNN and ResNet18 on the FashionMNIST dataset, Fig 19 for VGG16-CNN on FashionMNIST, and Fig 26 for Optimiz-CNN on HAM10000. This behavior has contributed significantly to achieving optimal convergence and enhancing classification performance across diverse architectures and datasets.
A detailed study of the parameter curves of the layers of SBNet-CNN has shown that in about 90% of the layers, the
value has been constantly within the 0.6 to 1 range as shown in Fig 14. This trend has indicated the adaptive behaviour of the adaptive Eq (2) with Eq (12) mechanism that has provided the necessary non-linearity to the SBNet-CNN architecture. In this
regime, the denominator in the adaptive SMSA
has been large enough to dominate the numerator term in the adaptive SMSA formulation, leading to a suppression of gradient exploration in most of the layers, which are seen in Fig 2. This suppression has not caused the network to merely memorize the training image features; rather, it has facilitated the extraction of generalized feature representations, leading to the formation of well-separated and distinct decision boundaries. Consequently, SBNet-CNN has achieved higher test accuracy and robust generalization performance, as substantiated by the results presented in Fig 21 for SBNet-CNN. In contrast, the
trajectories observed in ResNet-18 have shown a continuous decline, where most
values have decreased from 0.4 down to 0.01 as in Fig 14. In this lower
regime, the numerator term of adaptive SMSA has become dominant, which has intensified gradient exploration in approximately 93% of ResNet-18’s layers in Fig 14 and mostly layer having non linearity as in Fig 2.
This excessive exploration has led the network towards memorization of training-specific features, resulting in overlapping and ambiguous decision boundaries between various image classes. Consequently, misclassification rates have increased, and a substantial performance gap has emerged between validation and training accuracies, which has been consistently observed in Fig 21 for ResNet18-CNN. For VGG-16, instead of analyzing each layer’s trajectory individually, an average
trajectory across layers has been computed. These averaged values have predominantly remained within the range of 0.3 to 0.36 obserserved in Fig 19, suggesting that a moderate degree of gradient exploration has been maintained less aggressive than ResNet-18 but higher than SBNet-CNN. This trend has been captured in the apparent differences in training, validation, and test accuracies of VGG-16 performance measures. HAM1000 classification task has brought more challenging issues because it is characterized by image classes sharing high visual similarities. Here, it can be seen that the non-linear activation pattern of the network has shown the network use of a unique behavior of the network as denoted by its adaptivity behavior known as its
.
This observation has revealed the ability of the adaptive SMSA with the mechanism of Eq (2) in conjunction with the one of Eq (12) to adapt its responses with changing image difficulty in order to enable effective learning and enhanced classification of images under difficult conditions. The change in the values of in all the layers of the optimized CNN has been depicted in Fig 18. Most layers have ensured that their
is kept between 0.4 and 1, and this has contributed to reduction of the over-exploration of the gradient. A few layers have been observed to be close to 0.1, with a localized area of high gradient activity. In general, non-linearity in behavior between layers has been noticed to be consistent as shown in Fig 18. The performance results have been verified by the result of Fig 26, wherein obtained rapid convergence, convergence stability and great generalization. The complementary analysis of Figs 18 and 26 provides a result with significant scientific value. The orientation of the learnable parameter
is highly divergent within the network layers, most of the values taking values over 0.4.
The implication of such trend is the existence of an active, layer-specific adaptivity, as opposed to a relaxation to some fixed (or even saturated) regime of activation. Thereby, the observed variability supports the idea that the Adaptive SMSA activation is not breaking down into a single and tight functional archetype, rather, each of the layers modulates its nonlinearity independently according to its representational demands.
This dynamics are evidence of an effective, layer-by-layer exploration of the activation space wherein different network depths learn to be differentially sensitized to curvature which in turn allows learning of hierarchical representations of features. Fig 26 also showed that no observable discrepancy was found between training and validation accuracy, and test accuracy is highly correlated with training and validation accuracy. The implication of such concordance is that there is little overfitting and strong extrapolating extracted features. The consistency in training, validation, and testing demonstrates that the model has acquired consistent and stable characteristics. This signifies that the results are not a result of overfitting but consistant learning across all the stages. Although the adaptive SMSA mechanism helps to control the non linearity modulation, a large share of the observed generalization performance can be attributed to the O -CNN architecture, which effectively exploits adaptive activation dynamics. It has been empirically demonstrated that convolutional neural networks perform better with adaptive nonlinearity added. It increases the learning capability of the networks and leads to a more stable optimization process, it improves generalization. As one looks at the learnable parameter which gets successively learnt by different layers there is a definable stratification of the functions in the network. Initial layers of the convolutional model store relatively small maxima of curvature, and thus facilitate the effective extraction of low-level spatial information (edges and textures). More so, successively deeper layers assume increasingly larger values of beta, which simplifies more powerful nonlinear transformations, which are needed to formulate complex semantic representations. This slope of the gradient of
is harmonious with the canonical representation learning hierarchies of canonical hierarchical representation, in which the complexity of features increases in parallel with the depth of networks. Besides, no parameter saturation ensures a constant gradient flow during training, effectively preventing the issue of vanishing or exploding gradients and hence a smooth convergence. Formulation SMSA(x, n) means that the slope parameter is to be determined manually, which is more engineering work, and potentially not as scalable to a wide variety of tasks. This formulation was used to analyze the behavior of curvature and gradient characteristics that depend on the interval of such an activation function. To overcome this limitation, different versions of adaptive variants such as (x,
) and layer-wise learnable parameters were introduced that allows the network to autoregulate the parameters of activation during training. It is an autonomous process which improves optimization stability, scaling to large datasets, and transfer to diverse datasets and CNN architectures. SMSA(x,
) in which the parameter
is trained end-to-end by back-propagation. As a result, the adaptive one allows the data-driven gradient changes, which enhances the scalability and cross-task generalisation without having to adjust any parameter manually.
Performance comparison with baseline activation functions
The FashionMNIST dataset with its similarity among the inter-classes and minimal differences in features have posed a serious challenges in the design of the activation functions. Tables 16–19 and Fig 20 have given a comparative analysis of the standard and advanced activations, the suggested SMSA (both fixed and adaptive). The fixed SMSA specified by Eq (1) has demonstrated a higher validation accuracy than ReLU, ELU, and Leaky ReLU with a high of 93.5% accuracy in Table 16. This has been enhanced by the fact that it has a symmetric structure and an adjustable slope in the form of stabilized gradient flow and quick convergence despite overlapping classes. Standard static activations have pushed most of the layers into zero gradient or unstable areas and cannot thus be optimized. Conversely, SBNetCNN with the adaptive SMSA in the layer-wise in Eq (2) with Eq (12) has the highest test accuracy of 97.52% according to Table 17.
It has been able to defend close to 95% of layers against gradient problems by training a discrete nonlinearity on each layer. This flexibility has introduced better boundaries in classes, generalization and performance even with complicated overlapping conditions. SMSA has provided better gradient control and faster convergence at a lower cost than functions such as Mish, Swish, AHerfReLU [21] and swish [32]. These findings have provided adaptive SMSA in Eq (2) with Eq (12) as an effective and valid strategy of classification of image rich in features. The effectiveness of the layer-wise adaptive SMSA activation function has been proven by the results provided in Table 17, together with the evaluation measures of ResNet18. In contrast to older nonlinearity: ReLU, Swish, ELU and more recent ones like RTA, the adaptive SMSA in Eq (2) with Eq (12) has allowed every convolutional layer to learn independently its best nonlinearity in training. This flexibility has enhanced learning processes, increased expressive capacity and led to improved classification performance. When taking the example of ResNet18, which combines more than 30 pretrained layers, the adaptive SMSA has been implemented across around 70% of the network layers. This partial integration has been found to be enough not to degrade gradient in most parts of the network leading to the reported gains in performance in all the major performance measures such as precision, recall, F1-score as well as test accuracy on the FashionMNIST dataset. Nevertheless, the gradient flow in ResNet18 and Vgg16 has sometimes become more pronounced due to the fixedness of the remaining pretrained layers (around 30%), and it prevents full removal of overlapping decision boundaries between the models. Therefore, despite ResNet18 with adaptive SMSA being superior in all performance measures compared to all other static and recent activation functions, it has been shown to have slightly low classification accuracy compared to SBNet-CNN. This distinction is accredited to the absolute adaptability of SBNet-CNN, that has allowed more considerable movement of gradients and enhanced capacity to discriminate features across all levels. However, the adaptive SMSA as expressed in Eq (2) along with the expression in Eq (12) has proven itself to be a well-developed and scalable method in achieving better CNN performance, especially in deep networks such as ResNet18. To ensure a fair and rigorous evaluation, all activation functions have been implemented using the same optimized CNN architecture under identical environmental and hyperparameter settings on the HAM10000 (skin cancer) dataset. The training regime, data preprocessing pipeline, learning rate schedule, and optimizer configurations have been uniformly maintained across experiments. A critical analysis of Fig 21 shows that although both SBNet and ResNet-18 show a perceptible gap in training and validation accuracy, the gap is significantly bigger in the case of ResNet-18. The trajectories for ResNet18 in Fig 14 point to the fact that, in ResNet-18, the maintains the value of beta much less than 0.3 (Exploration occure), and not having variation in the
trajectories for layers after epoch 15(adaptivity vanished). On the other hand, SBNet maintains the value of
much higher than 0.6, thus allowing continuous layer-wise curvature modulation and stabilization, which together provide a contracted feature generalization and proven better out-of-sample performance.
Fig 20 and Table 18 have presented a clear performance differentiation among the tested activation functions, namely ReLU, ELU, NGNDG, and the proposed SMSA. Notably, SMSA in its fixed form (n = 1) has already achieved the highest validation accuracy (97.87%) and the lowest training time (281.81s) among non-adaptive methods, surpassing even recent alternatives like NGNDG and ELU in terms of both convergence speed and generalization. Moreover, training and validation loss has been steady and low in case of SMSA which shows a stable and efficient learning curve. Performance has further been increased with the adaptive version of SMSA, which is controlled by the Eq (2). Table 19 shows that this setup has obtained close to perfect classification scores: 98.9% validation accuracy, 0.989 F1-score and a very high AUC of 0.9994 which indicates that the model is very sensitive discriminatively. Also, the model has taken just 37 epochs before converging, which further shows its efficiency in computing. The gradient flow, feature separability and stability of the loss landscapes have been improved by the capability of the adaptive SMSA to dynamically change its slope across layers. This has been especially useful with the HAM10000 dataset that consists of highly imbalanced and morphologically overlapping skin lesion classes. This series of findings have all affirmed that the suggested SMSA activation particularly in its adaptive form has significant benefits compared to traditional and recent activation functions in terms of the efficiency and effectiveness of training, as well as the classification.
Comparative performance analysis of CNN models
This subsection has provided a comparative study of CNN model performances. This assessment has been performed on the Fashion MNIST dataset, and then on the HAM10000 dataset. An in-depth comparison of the results obtained on the two datasets is presented.
Comparative evaluation of SBNet-CNN with ResNet18 and VGG16 on Fashion-MNIST.
To confirm the proficiency of the suggested SBNet-CNN architecture in Tables 2–4 that had been combined with layer-wise adaptive SMSA activations given in Eq (2), a thorough comparison with well-known CNN architectures, namely ResNet18 and VGG16, was performed. This analysis has highlighted architectural performance, number of parameters, computational performance, and classification performance on the Fashion-MNIST data.
In experiments with layer wise adaptive SMSA for Fashion Mnist, ResNet18 and VGG16 have been augmented with adaptive SMSA activations to ensure a fair performance comparison. While these architectures have benefited from the learnable non-linearity introduced by SMSA, their inherently deep and parameter-heavy designs have still resulted in higher computational costs. In contrast, the proposed SBNet-CNN has adopted a lightweight architectural design that, combined with layer-wise adaptive SMSA activations, has allowed the network to flexibly modulate feature transformations while maintaining a minimal parameter footprint. Specifically, SBNet-CNN (with SMSA) has comprised only 1.7 million parameters, significantly fewer than the 11.2 million and 14.7 million parameters of ResNet18 and VGG16, respectively. Consequently, SBNet-CNN has achieved over an 80% reduction in parameter count compared to VGG16, while simultaneously enhancing model expressiveness through SMSA-induced slope modulation at each layer. Fashion-MNIST dataset has presented a much more difficult classification problem than traditional benchmarks as it is more intra-class variant, less inter-class separability, complex textures and shapes, grayscale fine-grained variability, and low image resolution (28 x 28 pixels) per image. Delicate differences among visual similar classes like shirts, tops, and coats have required fine feature extraction abilities, which traditional architectures have found difficult to satisfy. Also, the lack of the color information has forced models to use only the patterns of structure, and strong non-linear feature transformations have become essential in the non-linear classification. It has been rigorously demonstrated that the proposed SBNet-CNN is indeed effective in dealing with these complexities through its lightweight yet flexible architecture using its evaluation on this dataset. Combining layer-wise adaptive SMSA activations has allowed the network to non-linearly, slope, and activation range control per layer, allowing it better resolution of finer spatial detail with minimal computational cost. The results of experiments in Tables 20–Table 21 were also confirmed by performance curves demonstrated in Figs 21–25. The SBNet-CNN has reached a validation accuracy of 98.2% and a training accuracy of 99.1%, and it only takes 12 epochs to converge.
Comparatively, ResNet18 with SMSA took only 16 epochs to converge and the validation accuracy was only 94.5% compared to99.2% with ResNet18, even though the training accuracy was a bit higher. Equally, VGG16 combined with SMSA converged after 18 epochs and reached a validation accuracy of 95.0%, and training accuracy of 99.5% and 99.1% indicating a more significant generalization gap. These findings indicate that the adaptive SMSA-based SBNet-CNN is computationally cost-effective, and has a substantially better generalization capacity and learning throughput than the parameter-intensive traditional architectures.
Confusion matrices of SBNet-CNN, ResNet18-CNN and Vgg16-CNN on the Fashion-MNIST test data set have been generated to assess classification quality as depicted in Fig 19 and Figs 22 and 23. SBNet-CNN has its overall accuracy of 97.52% (9752 correct and 248 incorrect) which compares to the accuracy of 94.18% attained by ResNet18-CNN, (9418 correct and 587 incorrect predictions) and 94.13% by VGG16-CNN, (9413 correct and 587 incorrect predictions). The per-class distribution shows that SBNet-CNN has brought much fewer misclassifications to visually similar categories like the ones of Coat, Pullover and Shirt, in which traditional architectures have failed because of the overlapping feature boundaries. This has been enabled by the specific adaptive non-linearity added to the activation functions of the SMSA, which have added parameters of learnable slope and saturation ranges to each layer.
These dynamic characteristics have enabled SBNet-CNN to dynamically reconfigure activation responses to match the complexity of features in each layer thus improving intra-class compactness and inter-class separability. The power to refine non-linearity at a grain granular level has reinforced discriminative boundaries of ambiguous classes, resulting in a significant decrease in the misclassification errors. Consequently, SBNet-CNN has become even stronger in terms of both robustness and generalization as well as efficient feature encoding than the more complex ResNet18-CNN.
The ROC curves and the scores of the AUCs achieved on the test data with the SBNet-CNN architecture and ResNet18-CNN architecture fitted with the layer-wise adaptive SMSA activation function have been plotted on Fig 25 on the FashionMNIST dataset. The AUC value has measured the capability of the model to rank the classes and has captured the quality of features that are learned by the CNN as well as the performance of the activation function in modelling non-linear decision boundaries. SBNet-CNN using SMSA has reached an AUC value of 1.00 on all ten classes, which implies flawless class separability. AUC values of 0.98 to 1.00 suggest extremely reliable discrimination by ResNet18-CNN with layer-wise SMSA across all classes. These findings on the test data have not only demonstrated the ability of the CNN architectures to produce discriminative features, but have also demonstrated the use of layer-wise adaptive activation in allowing each layer to tune its non-linearity to the complexity of representations at various depths. The high AUC scores achieved between the architectures have highlighted the efficiency of the SMSA mechanism to improve the decision boundaries even with minor inter-class similarities. Furthermore, the consistency of the findings in all classes has been used to show that the activation is highly generalizable despite training data. This layer adaptive has enabled the network to maintain high-ranking confidence with different complexities of features. These findings have shown that the proposed activation function can be scaled to architectures of different depth and representational hierarchies.
It has been suggested that the greater performance (test accuracy 97.52%) of SBNet-CNN in Table 22 than all other models was enabled by the fact that its architecture enabled the induction of compact yet expressive feature hierarchies through the strategic placement of convolutional blocks each with adaptive SMSA activations. The modulated gradient flow is using the learnable slope parameters of SMSA that accelerates convergence and removes vanishing gradients and adaptive range control that dynamically varies the sensitivity of activations to preserve discriminative features with varying data distributions. Further, the non-linearity ability of SMSA has facilitated each of the layers to offer autonomous balancing of sharp and smooth activations to offer a response that is explicitly adopted to the richness of the input features. This architecture conformity with lightweight CNN backbone and layer-wise adjustable non-linearity has been a great contributor to the ability of SBNet-CNN to produce high-level accuracy with reduced computational expense. SBNet-CNN has also been more adaptable to heterogeneous patterns of features compared to conventional CNNs which typically rely on fixed activation functions, i.e., ReLU or Swish. This has been particularly useful where there is a fine and context-specific inter-class variation. The network has not just done a better generalization of the undiscovered data but has also illustrated a better robustness in the challenging classification circumstances. One of the salient observations that are the result of the Fashion-MNIST experimental studies is succinctly summarized in Figs 7 and 11, Fig 21, where a Custom CNN with the addition of the mechanism of the SMSA(x, n) is shown and in Figs 7 and 11. SBNet with the addition of the mechanism of the adaptive activation SMSA(x,) is demonstrated in Fig 21. Custom CNN structure, by definition, is relatively primitive and contains a limited number of trainable parameters; therefore, even though it can achieve a reasonable amount of feature generalization, it does not achieve complete convergence. On the other hand, the SBNet framework, which is combined with the adaptive activation, theSMSA(x,
) wisely uses non-linear transformation between consecutive layers. This adaptive modulation enables the network to develop more hierarchical representations which in turn spawn faster convergence, stabilised optimisation dynamics and greater levels of generalisation. Thus, SBNet is shown to be better at converging and more generalising compared to the less complex example of the Custom CNN. The deep networks such as ResNet18 and VGG16 can be constrained to their partially adaptive forms. In such models, multiple layers are fixed in place as a result of using pretrained weights that may limit gradient flowing and diminish complete end-to-end adaptiveness. However, low-level features are mostly learned at an early stage of the layers, whereas higher-to-lower levels generate more abstract and generalized features. Such mid-level representations can be improved by introducing layer-wise adaptive SMSA in about 70% of the trainable layers. This results in a marked improvement in the learning dynamics and classification performance as opposed to the case of the activated static settings. The analysis has included this observation and its implications.
Comparative evaluation of Optimized CNN and DenseNet121 against baseline CNNs for skin cancer and COVID-19 Classification.
Convolutional neural networks have demonstrated very good performance Figs 26 and 27 in sensitive medical images like skin cancer classification. Adaptive activation functions have been incorporated in this performance to enhance it. The suggested SMSA model has brought in the learnable parameters of each layer, which allows slope and curvature to be dynamically controlled. These parameters have enabled the network to be able to modify its nonlinearity in accordance to a layer-specific learning requirement. The development of the values of
in the various epochs in Fig 18 has proved convergence and interpretability. The layers have had a clear
curve, which represents how the network can be balanced to flexibly between expressiveness and regularization. This adaptivity has been able to increase shallow and deep extraction of features and this has improved robust learning. SMSA has been able to train faster compared with fixed activations without compromising or degrading classification accuracy. The model has recorded good performance on sensitive and complicated images of skin lesions. As can be seen, in Tables 23 and 24, the proposed Adaptive-SMSA is more predictive than Fixed SMSA(1) and training always. The increased training and validation accuracy (0.991 and 0.990) and very low loss values indicate that the adaptive model is more optimally stabilized and it has a better generalization. In addition to that, it also converges significantly faster (40 vs. 100 epochs) and reduces overall training time (224s vs. 500s), a fact that affirms the utility of learnable nonlinearity parameters in speeding up convergence and enhancing model achievement. These findings have established that adaptive activation functions are an important component to CNN generalization and robustness.
and validation metrics.The class-wise performance presented in Table 25 shows that the Optimized CNN using SMSA(1) produces a very balanced and steady classification with all seven skin-lesion categories. The final test accuracy of 98.00% with a macro- and weighted F1-score of over 98 proves the strong multiclass discriminative power of the model. Some of the classes achieve a near-perfect accuracy and recall (such as Classes 0.0, 3.0, and 5.0), meaning acceptable identification with no significant false positives or false negatives. Even though Class 4.0 has a slightly lower recall at 89% it does not affect the overall macro and weighted measurements, thus confirming the presence of strong generalization despite the presence of class imbalance.
A direct comparison of the proposed Optimized CNN with the state-of-the-art methodologies is provided in Table 26, which makes it clear that the former is superior. Compared to antecedent approaches such as those proposed by Gupta, Shahin and Carcagni (accuracy is between 83.10% to 90.00%), and more complex architectures such as DCAN-Net (accuracy is 97.57%), the SMSA(1)-based CNN presented below achieves maximum accuracy (99.00% to 99.00%), highest precision (99.00%), recall (99.00%), F1-score (98.89%), and test accuracy (98.14%). This uniform improvement on all the measures of evaluation supports the augmented discriminative capacity bestowed by the SMSA activation mechanism.
Table 27 gives additional empirical data on the effectiveness of the Adaptive variant of the SMSA used with the test dataset. Here, the Optimized CNN is better at all the other tested convolutional neural network architectures: precision of 98.45%, recall of 98.35%, F1-score of 98.49%, accuracy of 98.87%, and ROC-AUC of 99.40%. This study recorded a significantly high ROC-AUC score compared to the rest of the tested CNN architectures, such as MobileNet, ResNet-152v2, VGG variants, and U-Net based hybrids. This finding shows better separation of classes and threshold independent behavior, which further supports the usefulness of adaptive nonlinearity control. On the same note, Table 28 supports the long-term high performance of the proposed approach on the validation set. The model achieves a validation accuracy of 99.10% and ROC-AUC of 99.50% which is higher than all baseline architectures with a slight difference between validation and test values. The exceptionally small differences in Precision and F1-score also support the idea of the stability of learning dynamics and strong generalization. When combined, the results of all the tables support the main claims that the integration of SMSA, especially the Adaptive Beta SMSA, strengthens the stability of optimization, the increase in feature representation, and, finally, achieves better classification results compared to traditional CNN architectures.
The SMSA architecture captures the concept of implicit meta-learning and nonlinearity engineering through a parameter that is trainable, denoted . That this innovation allows the simultaneous optimization of both synaptic weights and activation dynamics, it gives the model a layer-wise ability to modify its slope, curvature, and range of activation.The resultant adaptive modulation effectively counters gradient instability and at the same time increases the representational flexibility. Moreover, the learning-augmented activation paradigm is an exploration of nonlinear activation functions, where the exploration is nonlinear and continuously differentiable, thus avoiding reliance on heuristic nonlinearities. The hypothesis that more effective convergence is created by adaptive nonlinearity and increases the robustness of generalization in deep convolutional neural network models is supported by empirical evidence: higher classification accuracy, insignificant difference between validation and test performance, and significantly higher ROC-AUC values.
COVID-19 image classification has posed significant challenges for CNN-based models due to factors such as data scarcity, class imbalance, high inter-class similarity, heterogeneous imaging sources, and the presence of subtle pathological features. To mitigate these issues, a DenseNet-121 model integrated with the adaptive SMSA activation function has been employed. The proposed model has achieved a classification accuracy of 99.03%, an AUC of 98.74%, and a precision of 98.2%, as presented in Table 29. Compared to standard DenseNet-121 in Table 30, which achieved 97.03% accuracy, the inclusion of the SMSA activation has improved performance by approximately 2%. Furthermore, training time has been reduced from 989 seconds to 380 seconds, demonstrating both computational efficiency and improved learning capability. The Table 31 has shown that the proposed model also outperformed several state-of-the-art methods from the literature in terms of sensitivity, AUC, and overall classification accuracy.
The Figs presented above, Figs 28 and 29, show the training and validation accuracy/loss curves. The training accuracy has also been at an average of over 98.5% across all the epochs although the validation accuracy has been more than 99% after the 10th epoch. The loss on validation has drastically reduced after epoch 5 and stayed constant at small values, reflecting a high degree of generalisation to unseen data and no overfitting, despite a small and imbalanced dataset. SMSA adaptiveness of the activation functionality has been a main contributing factor to this enhancement. As shown in Fig 30. The trainable coefficient has been steadily growing between about 0.295 and 0.304 as it has been trained.
This adaptive response has seen the network dynamically adapt its non-linear response to model more subtle, diffused manifestations of COVID-19 including ground-glass opacities and consolidations. The DenseNet-121 model with SMSA activation has shown to be superior in COVID-19 classification due to its improved capability to perform feature extraction, overfitting control, and dynamic non-linearity. Computational efficiency and generalizability of the model viewpoint indicate that the model can be applied in real-time clinical settings. Combined, the two versions of SMSA have done image classification problems more and more efficiently as the nonlinear representational requirements of the deep convolutional neural network designs. Activation Learning slopes and curvatures may be adaptively rescaled using a learnable hyperparameter whose name is the , thereby making an implicit kind of meta-learning, the transformation dynamics as well as network weights are co-optimized. These nonlinearity which are designed assist to enhance discrimination feature learning, stabilize gradient propagation and decrease the gap between the validation and the test performance. Empirical evidence in the current research demonstrates the concept that the SMSA framework can not only offer optimum classification and classification but also guarantee sound reliable, generalization of image information sets of various and complex data.
Statistical validation of classification performance: paired t-test and confidence interval analysis for ResNet18 and SBNet CNN architectures
As summarized in Table 32 represents the statistical analysis that has provided useful insights into the performance of the ResNet18 CNN and SBNet CNN models. The 95% confidence intervals, namely [0.915, 0.945] in the case of ResNet18 and [0.968, 0.972] in the case of SBNet, suggest that the accuracy of the classification estimates are very precise. The mean accuracy of the SBNet-CNN model is greater thus indicating a high classification ability as compared to ResNet18. The very high t-values of ResNet18 and SBNet 108.47 and 852.94 respectively with a p-value of less than 0.0001, provide strong statistical evidence against the null hypothesis that the baseline accuracy equals 0.10. These results support the fact that both models have significantly improved their performance. What is more, the low standard error of the values indicates the low level of variation of the values among the folds, which further increases the confidence in the generalization and reliability of the model
Fig 31 visually shows the relative accuracy of classification of the ResNet18 and SBNet CNNs, and Fig 32 uses results to show the difference between the observed mean accuracy and the expected value of the accuracy, and the error margins. Combined, these analyses prove the suitability of the two networks, with a significantly better and more consistent performance of SBNet-CNN on the Fashion MNIST dataset.
Conclusion
This paper addresses the question of how to design adaptable nonlinear activation functions in a convolutional neural network by proposing Symmetric Modifiable Slope Activation (SMSA) function and analyzing them in three different variants: manually tuned SMSA with a fixed slope parameter n, globally adaptive SMSA with a learnable slope parameter , and the proposed layer-wise adaptive SMSA where SMSA has a learnable slope parameter,
, independently learned through each convolutional layer, in its three variants which are described by Eq 2. Systematic evaluation was conducted on datasets of different structural complexity, which included MNIST, Fashion MNIST, HAM10000, and COVID −19, using small and large CNNs, namely Custom CNN, SBNet -CNN, ResNet18, VGG16, Optimized CNN, and DenseNet121. Experimental findings indicate that the manually tuned SMSA has good feature discrimination with the slope parameter n chosen according to the characteristics of the dataset. However, the above method presents the reliance on manually-acquired hyperparameters, which limits its ability to scale-up to heterogeneous tasks. This problem is solved by globally adaptive-SMSA, which does not change the nonlinearity of the network, but independently optimizes the slope of activation by the gradient descent mechanism. The greatest improvement is achieved in the form of the proposed layer-wise adaptive SMSA, which allows every convolutional layer to employ its own nonlinear transformation. This is an adaptive process that inhibits divergence, regularizes the gradient propagation and also enables hierarchical learning of salient features. The effectiveness of the suggested method is supported by empirical assessments. The SBNet-CNN with layer-wise adaptive SMSA achieved a test, validation and training accuracy of 97.52%, 98.89% and 99.1% respectively on the Fashion-MNIST dataset and only took 17 training epochs to converge, in comparison to more complex networks like ResNet-18 (94.18%) and VGG-16 (94.13%). In a similar manner, experiments run on the HAM10000 data set exhibited an improvement in the feature discriminability and faster convergence. Besides, when integrated with DenseNet-121 on the COVID-19 dataset, integration demonstrated a higher robustness considering extreme imbalance in classes and was about 4% more precise compared to the baseline model. On the whole, these results emphasize the importance of the adaptability of activations to deep learning system optimization. The proposed layer-wise adaptive-SMSA allows nonlinear transformations to be heterogeneous across network depth, allowing each layer to work in a favorable activation regime. This method is therefore significantly more stable in optimization, higher in gradient propagation, and able to cross-dataset (and cross-architecture) generalize. Further developments in the use of SMSA in transformer-based models, graph neural networks, and large-scale vision systems and theoretical investigations into the dynamics of adaptive activation curvature should be explored in future research to better understand the role of nonlinear adaptability in deep representation learning.
References
- 1. Kumar R, Kumbharkar P, Vanam S, Sharma S. Medical images classification using deep learning: a survey. Multimed Tools Appl. 2024;83(7):19683–728.
- 2. Du K-L, Zhang R, Jiang B, Zeng J, Lu J. Understanding Machine Learning Principles: Learning, Inference, Generalization, and Computational Learning Theory. Mathematics. 2025;13(3):451.
- 3. Sayın KA, Gürsoy NK, Yolcu T, Gürsoy A. On the Synergy of Optimizers and Activation Functions: A CNN Benchmarking Study. Mathematics. 2025;13(13):2088.
- 4. Fan J, Ma C, Zhong Y. A selective overview of deep learning. Stat Sci. 2021;36(2):264–90. pmid:34305305
- 5. Wang K, Jiang P, Meng J, Jiang X. Attention-based DenseNet for pneumonia classification. IRBME. 2022;43(5):479–85.
- 6. Kiczek B, Batsch M. Exploration of Unsupervised Deep Learning-Based Gear Fault Detection for Wind Turbine Gearboxes. Energies. 2025;18(14):3630.
- 7. Trimarco V, Izzo R, Pacella D, Manzi MV, Varzideh F, Lembo M, et al. The COVID-19 pandemic increased the incidence of newly diagnosed cancers: evidence from a large cohort study in Southern Italy. BMC Med. 2025;23(1):399. pmid:40598436
- 8. Yong Luo, Tongliang Liu, Dacheng Tao, Chao Xu. Multiview matrix completion for multilabel image classification. IEEE Trans Image Process. 2015;24(8):2355–68. pmid:25872208
- 9. Mishra J, Sharma RK. Optimized FPGA Architecture for CNN-Driven Voice Disorder Detection. Circuits Syst Signal Process. 2025;44(6):4455–67.
- 10. Sun T, He L, Fang X, Xie L. Enhanced Multilinear PCA for Efficient Image Analysis and Dimensionality Reduction: Unlocking the Potential of Complex Image Data. Mathematics. 2025;13(3):531.
- 11. Kim J-M. Integrating Copula-Based Random Forest and Deep Learning Approaches for Analyzing Heterogeneous Treatment Effects in Survival Analysis. Mathematics. 2025;13(10):1659.
- 12. Chrysos GG, Wu Y, Pascanu R, Torr PHS, Cevher V. Hadamard Product in Deep Learning: Introduction, Advances and Challenges. IEEE Trans Pattern Anal Mach Intell. 2025;47(8):6531–49. pmid:40232897
- 13. Hassan DA. Deep neural network-based approach for computing singular values of matrices. SJUOZ. 2025;13(1):1–6.
- 14. Irfan K, Rehman MU. Performance analysis of ResNet50 on balanced and imbalanced image classification datasets. Sciencetech. 2025;6(2):16–29.
- 15. Aradea A, Rianto R, Herlina N, Hoeronis I. Self-Learning Model for Pattern Recognition in Vision System Based on Adaptive Kernel. IJCAI. 2025;49(14).
- 16. Raza A, Ali A, Ullah S, Anjum YN, Rehman B. Optimizing skin cancer screening with convolutional neural networks in smart healthcare systems. PLoS One. 2025;20(3):e0317181. pmid:40132163
- 17. Al Mahmud A, Azam S, Khan IU, Montaha S, Karim A, Haque A, et al. SkinNet-14: a deep learning framework for accurate skin cancer classification using low-resolution dermoscopy images with optimized training time. Neural Comput Appl. 2024;36(30):18935–59.
- 18. Hussain T, Shouno H. MAGRes-UNet: Improved Medical Image Segmentation Through a Deep Learning Paradigm of Multi-Attention Gated Residual U-Net. IEEE Access. 2024;12:40290–310.
- 19. Balestriero R, Humayun A, Baraniuk R. On the Geometry of Deep Learning. Notices Amer Math Soc. 2025;72(04).
- 20. Xiao L, Li X, Jia L, Liu S. Improved finite-time solutions to time-varying Sylvester tensor equation via zeroing neural networks. Appl Math Computat. 2022;416:126760.
- 21. Ullah A, Imran M, Basit MA, Tahir M, Younis J. AHerfReLU: A Novel Adaptive Activation Function Enhancing Deep Neural Network Performance. Complexity. 2025;2025(1).
- 22. Pandey GK, Srivastava S. Rectified Tangent Activation (RTA): A Novel Activation Function for Enhanced Deep Learning Performance. IEEE Access. 2025;13:120028–39.
- 23.
Nanni L, Lumini A, Ghidoni S, Maguolo G. Comparisons among different stochastic selections of activation layers for convolutional neural networks for health care. In: Cognitive and Soft Computing Techniques for the Analysis of Healthcare Data. Elsevier; 2022. p. 151–64.
- 24.
Biswas K, Kumar S, Banerjee S, Pandey AK. Smooth Maximum Unit: Smooth Activation Function for Deep Networks using Smoothing Maximum Technique. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2022. p. 784–93.
- 25.
Klabjan D, Harmon M. Activation Ensembles for Deep Neural Networks. In: 2019 IEEE International Conference on Big Data (Big Data). IEEE; 2019. p. 206–14.
- 26. Jagtap AD, Kawaguchi K, Em Karniadakis G. Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks. Proc Math Phys Eng Sci. 2020;476(2239):20200334. pmid:32831616
- 27.
Lyakhova UA. Neural network skin cancer recognition with a modified cross-entropy loss function. In: International Conference on Actual Problems of Applied Mathematics and Computer Science. Cham: Springer Nature Switzerland; 2022. p. 353–63.
- 28. Han SS, Moon IJ, Lim W, Suh IS, Lee SY, Na J-I, et al. Keratinocytic Skin Cancer Detection on the Face Using Region-Based Convolutional Neural Network. JAMA Dermatol. 2020;156(1):29–37. pmid:31799995
- 29. Dubey SR, Singh SK, Chaudhuri BB. Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing. 2022;503:92–108.
- 30. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44. pmid:26017442
- 31. Khan AR, Mujahid M, Alamri FS, Saba T, Ayesha N. Early-Stage Melanoma Cancer Diagnosis Framework for Imbalanced Data From Dermoscopic Images. Microsc Res Tech. 2025;88(3):797–809. pmid:39573895
- 32. Yuan H, Ren Q. Divide-and-conquer routing for learning heterogeneous individualized capsules. PLoS One. 2025;20(7):e0329202. pmid:40737290
- 33. Nocentini O, Kim J, Bashir MZ, Cavallo F. Image Classification Using Multiple Convolutional Neural Networks on the Fashion-MNIST Dataset. Sensors (Basel). 2022;22(23):9544. pmid:36502243
- 34.
Chaturvedi SS, Gupta K, Prasad PS. Skin lesion analyser: an efficient seven-way multi-class skin cancer classification using MobileNet. In: International conference on advanced machine learning technologies and applications. Singapore: Springer Singapore; 2020. p. 165–76.
- 35. Huang H-W, Hsu BW-Y, Lee C-H, Tseng VS. Development of a light-weight deep learning model for cloud applications and remote diagnosis of skin cancers. J Dermatol. 2021;48(3):310–6. pmid:33211346
- 36.
Shahin AH, Kamal A, Elattar MA. Deep Ensemble Learning for Skin Lesion Classification from Dermoscopic Images. In: 2018 9th Cairo International Biomedical Engineering Conference (CIBEC). IEEE; 2018. p. 150–3.
- 37.
Carcagnì P, Leo M, Cuna A, Mazzeo PL, Spagnolo P, Celeste G, et al. Classification of skin lesions by combining multilevel learnings in a DenseNet architecture. In: International conference on image analysis and processing. Cham: Springer International Publishing; 2019. p. 335–44.
- 38. Chaturvedi SS, Tembhurne JV, Diwan T. A multi-class skin Cancer classification using deep convolutional neural networks. Multimed Tools Appl. 2020;79(39–40):28477–98.
- 39. Alsunaidi SJ, Almuhaideb AM, Ibrahim NM, Shaikh FS, Alqudaihi KS, Alhaidari FA, et al. Applications of Big Data Analytics to Control COVID-19 Pandemic. Sensors (Basel). 2021;21(7):2282. pmid:33805218
- 40. Aladhadh S, Alsanea M, Aloraini M, Khan T, Habib S, Islam M. An Effective Skin Cancer Classification Mechanism via Medical Vision Transformer. Sensors (Basel). 2022;22(11):4008. pmid:35684627
- 41. Thwin SM, Park H-S, Seo SH. A Trustworthy Framework for Skin Cancer Detection Using a CNN with a Modified Attention Mechanism. Appl Sci. 2025;15(3):1067.
- 42. Ismail WN, Alsalamah HA. Efficient Harris Hawk Optimization (HHO)-Based Framework for Accurate Skin Cancer Prediction. Mathematics. 2023;11(16):3601.
- 43. Mohakud R, Dash R. A Hybrid Model for Classification of Skin Cancer Images After Segmentation. Int J Image Grap. 2025;25(03).
- 44.
Sharma G, Chadha R. An Optimized Predictive Model Based on Deep Neural Network for Detection of Skin Cancer and Oral Cancer. In: 2023 2nd International Conference for Innovation in Technology (INOCON). IEEE; 2023. p. 1–6.
- 45. Ogundokun RO, Li A, Babatunde RS, Umezuruike C, Sadiku PO, Abdulahi AT, et al. Enhancing Skin Cancer Detection and Classification in Dermoscopic Images through Concatenated MobileNetV2 and Xception Models. Bioengineering (Basel). 2023;10(8):979. pmid:37627864
- 46. Tabrizchi H, Parvizpour S, Razmara J. An Improved VGG Model for Skin Cancer Detection. Neural Process Lett. 2022;55(4):3715–32.
- 47. Kumar Lilhore U, Simaiya S, Sharma YK, Kaswan KS, Rao KBVB, Rao VVRM, et al. A precise model for skin cancer diagnosis using hybrid U-Net and improved MobileNet-V3 with hyperparameters optimization. Sci Rep. 2024;14(1):4299. pmid:38383520
- 48. Arulananth TS, Prakash SW, Ayyasamy RK, Kavitha VP, Kuppusamy PG, Chinnasamy P. Classification of Paediatric Pneumonia Using Modified DenseNet-121 Deep-Learning Model. IEEE Access. 2024;12:35716–27.
- 49. Rajaraman S, Candemir S, Kim I, Thoma G, Antani S. Visualization and Interpretation of Convolutional Neural Network Predictions in Detecting Pneumonia in Pediatric Chest Radiographs. Appl Sci (Basel). 2018;8(10):1715. pmid:32457819
- 50. Sadik F, Dastider AG, Subah MR, Mahmud T, Fattah SA. A dual-stage deep convolutional neural network for automatic diagnosis of COVID-19 and pneumonia from chest CT images. Comput Biol Med. 2022;149:105806. pmid:35994932
- 51. Manickam A, Jiang J, Zhou Y, Sagar A, Soundrapandiyan R, Dinesh Jackson Samuel R. Automated pneumonia detection on chest X-ray images: A deep learning approach with different optimizers and transfer learning architectures. Measurement. 2021;184:109953.
- 52. Chouhan V, Singh SK, Khamparia A, Gupta D, Tiwari P, Moreira C, et al. A Novel Transfer Learning Based Approach for Pneumonia Detection in Chest X-ray Images. Appl Sci. 2020;10(2):559.
- 53.
Ayan E, Unver HM. Diagnosis of Pneumonia from Chest X-Ray Images Using Deep Learning. In: 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT). IEEE; 2019. p. 1–5.