2.1 Multi-view learning

Multi-view learning aims to exploit complementary information from different modalities or feature spaces to improve generalization and robustness. Canonical Correlation Analysis (CCA) is a classical linear multi-view learning method [9], whose core idea is to find projection directions that maximize the correlation between two views. Given feature matrices from two views and , CCA solves the following optimization problem to obtain the projection directions and :

(1)

Where , and denote the covariance and cross-covariance matrices. By maximizing (1), CCA identifies the most correlated subspaces for feature compression and fusion.

For nonlinear relationships, Deep CCA extends CCA by introducing neural networks as feature extractors, learning nonlinear mappings and before performing correlation maximization. The objective remains similar to CCA:

(2)

Deep CCA and its variants (e.g., DMCCA) have become popular for multi-view representation learning [10]. While CCA-based methods are theoretically appealing for learning shared representations, they lack explicit modeling of rule importance and semantic discrepancies between views, making them less interpretable in the context of fuzzy systems. Fine-grained multi-view fusion under fuzzy frameworks thus remains a challenge.

2.2 TSK fuzzy systems and multi-view extensions

TSK fuzzy systems are widely used in classification and regression due to their interpretable rule-based structure [8]. In traditional TSK systems, antecedents are defined by fixed-form membership functions, while consequents are typically linear functions-balancing nonlinear modeling capacity with interpretability.

A standard TSK rule can be expressed as:

(3)

The Gaussian membership function for antecedent is:

(4)

The firing strength of rule is:

(5)

The system output is computed as:

(6)

For multi-view problems, some works design view-specific rules or fuse rule activations from different views to enhance integration. However, fixed-shape membership functions limit representational capacity when handling high-dimensional, heterogeneous multi-view data, making it difficult to capture complex intra-view and inter-view feature distributions.

2.3 Deformable membership functions

To improve adaptability to complex data distributions, deformable membership functions have recently attracted attention [11]. By introducing learnable deformation parameters—such as center offsets and width adjustments—membership functions can dynamically reshape, leading to more accurate partitioning of the input space and enhanced rule discriminability. Compared to fixed-parameter Gaussians, deformable membership functions offer greater flexibility and expressiveness, and have shown advantages in certain fuzzy and neuro-fuzzy models.

The idea draws inspiration from Deformable Convolutional Networks (DCN) in computer vision [12], which introduce learnable spatial offsets to convolution kernels for modeling object deformations. Drawing a direct analogy, just as DCNs break the fixed grid structure to handle irregular object shapes in the spatial domain, our proposed Deformable GMFs break the fixed distribution constraints of traditional fuzzy sets. By introducing learnable offsets to the membership function centers, the model can dynamically align with heterogeneous and irregular data distributions in the high-dimensional feature space. This capability is analogous to ‘deforming’ the receptive field in vision, allowing the fuzzy system to capture complex intra-class variations that static Gaussian functions fail to model.“ Similarly, in fuzzy systems, deformable Gaussians allow the rule boundaries to adapt dynamically to data distributions, rather than remaining static.

Recent research includes deformable Gaussian functions, stretched/twisted membership functions, and other learnable shape-parameterized functions [11]. These methods have demonstrated improved adaptability to heterogeneous, multimodal, and asymmetric data, especially in high-dimensional classification and regression. The deformable Gaussian antecedent proposed in this paper follows this line of research, extending the traditional GMF by adding offset parameters to boost expressiveness and flexibility.

2.4 Attention mechanisms in fuzzy systems

Attention mechanisms, a cornerstone of modern deep learning, dynamically adjust the focus of models on specific regions or features based on the input [13]. Incorporating attention into fuzzy systems allows adaptive reweighting of rules or views during activation and fusion, improving both discriminative ability and robustness.

Initially popularized in natural language processing, attention mechanisms have since been widely adopted in computer vision and time-series modeling. In fuzzy systems, attention has been applied to regulate rule importance, enhancing adaptability and data awareness [14]. For example, MIC_FuzzyNet leverages additive attention to selectively enhance certain rule activations, although it does not integrate with antecedent structural improvements [15].

In multi-view fuzzy frameworks, the joint design of attention with deformable membership functions remains largely unexplored. Coordinating these two mechanisms has the potential to improve dynamic rule activation, mitigate redundancy, and fully exploit semantic complementarity across views.

3.1 Model structure overview

This study proposes a novel multi-view TSK fuzzy system that integrates deformable Gaussian membership functions with a rule-level attention mechanism for high-dimensional heterogeneous data. As shown in Fig 1, the architecture consists of three modules: a deformable antecedent layer, an attention regulation module, and a TSK consequent layer, deployed as parallel fuzzy subsystems in the multi-view setting.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Architecture of the MDA-TSK-FS model.

https://doi.org/10.1371/journal.pone.0348610.g001

Each view is processed by an independent deformable Gaussian antecedent network, where learnable offset parameters allow membership function centers to adapt dynamically to data distributions. The resulting rule activations are then refined by a self-attention mechanism that adaptively assigns rule importance based on the current input, improving discriminative focus.

Finally, activations from all views are fused at the rule level to form a global rule response, which is passed to the TSK consequent layer for weighted inference. The end-to-end trainable design preserves interpretability while improving adaptability and classification accuracy.

3.2 Deformable Gaussian membership function

In conventional TSK fuzzy systems, the antecedent membership functions are typically fixed-parameter Gaussians(as defined in Eq. (4)). Where and denote the center and width parameters, respectively. While simple, this static formulation has limited adaptability in high-dimensional or complex data distributions, often resulting in imprecise fuzzy boundaries and reduced rule activation effectiveness.

To enhance flexibility and data adaptivity, we introduce the Deformable Gaussian Membership Function, which augments the center with a learnable offset:

(7)

Where is the initial center, is a trainable deformation term, and is the standard deviation.

This formulation enables a deformable fuzzy partition, allowing membership functions to dynamically adjust their centers during training in response to data characteristics. Functionally, this can be viewed as a weak attention mechanism at the antecedent level, improving boundary precision and capturing local variations. The approach retains the Gaussian form, thus preserving interpretability, while substantially improving nonlinear modeling capacity—particularly in scenarios with complex class boundaries or localized perturbations.

3.3 Attention mechanism for rule activation

In multi-rule fuzzy systems, the degree to which an input sample activates each rule can vary substantially.

Traditional TSK models compute activations directly from membership functions and apply a simple normalization, ignoring deeper semantic relationships between the input and the rules. This can result in ineffective or redundant rules influencing the final decision.

To address this, we incorporate a Self-Attention-based rule-level attention mechanism that dynamically adjusts rule weights for each input sample, enabling the model to focus on the most informative rules.

Given an input feature vector and rules, calculating the firing strength by directly multiplying membership values in high-dimensional spaces often leads to numerical underflow.To address this, we compute the log-firing strength of the r-th rule as the initial activation. Derived from the Deformable GMF definition in Eq. (7), the log-activation is formulated as:

(8)

Where , and are trainable parameters, correspond to the center, offset, and width parameters defined in Eq. (7), respectively.

To capture the complex dependencies within the feature space and dynamically assign importance to fuzzy rules, we employ a Multi-Head Self-Attention (MHSA) mechanism. Unlike simple additive attention, MHSA projects the input features into multiple subspaces, allowing the model to attend to different semantic aspects of the data simultaneously.

Given the input feature vector , we treat it as a sequence of length 1. The query (), key (), and value () matrices are obtained via linear projections. The Multi-Head Attention is computed as follows:

(9)

(10)

(11)

Where represents the output of the i-th attention head, is the number of heads, and is the output projection matrix. The vector denotes the feature representation after self-attention. Subsequently, a linear transformation with weights and bias maps the attended features to the rule scores . Finally, the rule attention weights are obtained by applying the Softmax function.

3.4 Multi-view structure and fusion strategy

To effectively integrate information from multiple heterogeneous views, we propose a rule-activation-level fusion framework. Instead of concatenating multi-view features at the input layer, each view is processed by an independent fuzzy subsystem, which generates rule activation responses using Deformable GMFs and Self-Attention. The outputs from all views are then fused at the rule activation level, enabling cross-view information interaction without increasing input dimensionality.

Formally, let there be views, with the input of view denoted as , the fuzzy subsystem for view produces:

(12)

Where is the vector of raw rule activation strengths, contains the rule attention weights from Self-Attention, denotes element-wise multiplication, and is the number of rules. The global fused activation vector is then computed as:

(13)

Where represents the adaptive importance weight for view . In our proposed framework, is defined as a learnable parameter vector. To strictly satisfy the constraint and ensure numerical stability, we apply a Softmax operation to the unnormalized weights during the forward pass. These weights are initialized uniformly and updated end-to-end alongside the antecedent and consequent parameters via the AdamW optimizer. This allows the model to automatically learn the optimal contribution of each view based on the training loss.

This fusion strategy supports fixed weighting, mean fusion, or a view-level attention mechanism to dynamically adjust inter-view contributions per sample. By performing fusion at the rule level, the model preserves modularity, improves adaptability to heterogeneous features, and offers stronger interpretability and structural control—particularly beneficial for handling multi-source data with inconsistent dimensionality and semantic diversity.

4.1 Dataset overview

To objectively evaluate the proposed method, we conducted experiments on five real-world public datasets, each representing different application domains and feature characteristics. Specifically, four of these datasets (Dermatology, Forest Type, Caltech7, and Handwritten) were selected from the UCI Machine Learning Repository and standard public benchmarks, while the Epileptic EEG dataset was adopted from [16]. The details of these datasets are as follows:

1. Dermatology – A medically oriented dermatology dataset containing both histopathological and clinical perspectives, used for extracting features relevant to specialized diagnosis.
2. Forest Type – A remote sensing dataset derived from satellite imagery of forests, with features extracted from spectral bands and wavelength information.
3. Epileptic EEG – A dataset of electroencephalogram (EEG) signals from epilepsy patients. Features are extracted using Discrete Wavelet Transform (DWT) and Wavelet Packet Decomposition (WPD) [16]. (Ethical Statement: This study utilizes a publicly available, de-identified dataset originally published by Andrzejak et al. As the data contains no personally identifiable information and is in the public domain for research use, independent ethical approval or participant consent was not required for this secondary analysis.)
4. Caltech7 – A multi-view image dataset consisting of 1,474 images. Features are extracted from three perspectives: Gabor filters, CENTRIST descriptors, and wavelet texture analysis.
5. Handwritten – A widely used multi-view learning dataset containing 2,000 samples with six different views, often employed as a benchmark for multi-view classification tasks.

We evaluated our model on five public multi-view datasets (Caltech7, Handwritten Numerals, Dermatology, Forest Type, and EEG DWT-WAV). To ensure reproducibility and fair comparison, we utilized pre-extracted feature representations stored in.mat format (e.g., HOG/GIST/LBP for Caltech7, and DWT/WAV for EEG), eliminating the need to process raw signals. For data preprocessing, zero values within the feature matrices were first adjusted by adding a negligible constant () to prevent numerical instability. Subsequently, all features were normalized using either Min-Max or Standard scaling to align heterogeneous scales across different modalities. The exact feature dimensionalities and complete pipeline scripts are thoroughly documented in our public repository.

4.2 Experimental settings

The proposed MDA-TSK-FS was implemented in PyTorch and conducted on a workstation equipped with an NVIDIA RTX 4090D GPU. To ensure reproducibility, we fixed random seeds for all twenty independent runs. The datasets were split into 80% training and 20% testing using stratified sampling to maintain class balance consistent with the original distributions. Crucially, all preprocessing steps (e.g., normalization) were fitted solely on the training set to prevent data leakage; the derived statistics were then applied to the test set. We employed the AdamW optimizer with an initial learning rate of 0.0005 and a weight decay of 1e-8. The batch size was set to 8, and the maximum number of epochs was 256. To prevent overfitting, we applied EarlyStopping with a patience of 50 epochs. The number of fuzzy rules was set to 51.

In addition to classification accuracy, we calculated the entropy and normalized entropy of rule-level self-attention weights to assess the uniformity of rule utilization. Previous studies had shown that changes in attention entropy were closely related to training stability, and low attention entropy could lead to unstable or even divergent training; therefore, attention entropy was commonly used as an important indicator for monitoring the training status in models involving attention mechanisms [17]. Moreover, the number of effective rules was also counted to characterize the actual participation of rules in model decisions. For the deformable Gaussian membership functions, the mean and standard deviation were used to evaluate the stability of the Center, Sigma, and Offset parameters. These metrics provided quantitative support for the observations from the box plots, thereby assessing the stability and alignment of the deformable Gaussian antecedents across different datasets or views.

4.3 Results

To validate the effectiveness of the proposed MDA-TSK-FS model in multi-view data fusion tasks, we conducted comparative experiments against a set of existing representative TSK models, including the standard TSK-FS, MBGD-RDA [18], TSK-MBGD-UR, TSK-MBGD-BN [19], HTSK, HTSK-LN, HTSK-LN-RELU [20], as well as TSK variants under various multi-view fusion frameworks, namely DMCCA-TSK, DGCCA-TSK, DTCCA-TSK, DCCAEG-TSK [10], and MVD-TSK-FS [21]. All models were evaluated on five benchmark datasets, with classification accuracy used as the performance metric.

Table 1 presents the classification accuracies of all models across the five datasets. For the proposed MDA-TSK-FS, we conducted 20 independent runs and reported the results as “Mean±Standard Deviation” to reflect the stability and robustness of the model.For the baseline comparisons, we adopted the results reported in the recent study by Nie et al. [21], which provided a comprehensive evaluation of these TSK variants on the same datasets. Since the run-wise variance data (standard deviations) for these baselines were not available in the cited literature [21], we treated their reported mean accuracies as fixed benchmarks. To rigorously validate the improvements, we performed a one-sample t-test comparing the distribution of our 20 independent runs against the best result reported by the baselines. The asterisks (*) in Table 1 indicate that MDA-TSK-FS achieves a statistically significant improvement (p < 0.05) on all five datasets.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Classification Accuracy of Different Models.

https://doi.org/10.1371/journal.pone.0348610.t001

The proposed MDA-TSK-FS consistently achieved the best or near-best performance in all cases, reaching accuracies of 0.9438, 0.9862, 0.9858, 0.8857, and 0.6975 on the Caltech7, Handwritten, Dermatology, Forest, and EEG datasets, respectively. Notably, MDA-TSK-FS demonstrated outstanding performance on the Handwritten and Dermatology datasets, significantly surpassing traditional TSK models (e.g., TSK-FS’s 0.9100 and 0.9075). Furthermore, on the Forest and EEG datasets, MDA-TSK-FS achieved 0.8857 and 0.6975, outperforming all competing models. In contrast, strong baselines such as TSK-MBGD-BN and HTSK-LN-RELU achieved only 0.6000 and 0.6550 on EEG, respectively, highlighting the effectiveness of integrating deformable Gaussian membership functions with an attention-based fusion mechanism in enhancing the model’s capacity to model complex, heterogeneous data distributions. As shown in Fig 2, we plotted the convergence curves of each dataset over the epochs. The loss decreased rapidly across all datasets, and convergence was generally achieved between 50 and 100 epochs.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. MDA-TSK-FS Convergence curves of all datasets.

https://doi.org/10.1371/journal.pone.0348610.g002

We plotted box plots of the rule parameter distributions for each dataset, as shown in Fig 3. The means of the Centers were generally close to zero and exhibited little variation across datasets, indicating that the deformable Gaussian functions could stably align the centers of multi-view features and prevent systematic shifts in rule distributions. Furthermore, the Sigmas remained within the range of 0.95–0.99 across different datasets, demonstrating consistent deformation in rule widths: even when facing datasets with significantly different distributions, the expansion and contraction of the rules stayed within a reasonable range, ensuring balanced feature coverage. The means of the Offsets were close to zero, suggesting that only minimal translations were required during parameter adjustment to adapt to the multi-view feature space.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Distribution of rule parameters across datasets.

https://doi.org/10.1371/journal.pone.0348610.g003

To further investigate the utilization of rules in the MDA-TSK-FS model under multi-view feature fusion, we computed the average rule attention weights for all 1,474 samples in the Caltech7 dataset and plotted the corresponding weight distributions for each view (Fig 4).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Distribution of rule attention weights across views in the Caltech7 dataset.

https://doi.org/10.1371/journal.pone.0348610.g004

The attention distribution of the first view (48-dimensional features) was relatively uniform, with the Top-5 rules exhibiting significantly higher weights: Rule 13, Rule 14, Rule 23, Rule 24, and Rule 46. This indicates that the model primarily relied on these rules for discrimination in this modality, while other rules still contributed to a lesser extent. For the second view (40-dimensional features), the Top-5 rules were Rule 5, Rule 10, Rule 22, Rule 25, and Rule 44, showing clear differences from the rules emphasized in the first view, which reflects that discriminative information is distributed differently across modalities and that the model can flexibly select important rules according to different feature patterns. In the third view (254-dimensional features), the Top-5 rules were Rule 5, Rule 18, Rule 22, Rule 36, and Rule 41. Notably, Rule 5 and Rule 22 also exhibited high importance in the second view, suggesting that these two rules may represent cross-modal discriminative patterns. Overall, these qualitative results demonstrate that the MDA-TSK-FS model dynamically allocates importance weights to fuzzy rules via rule-level self-attention, reinforcing core rules while maintaining flexible adjustment of non-core rules, thereby achieving efficient and stable discrimination in multi-view fuzzy systems.

Next, to further understand the rule mechanisms underlying the model’s performance, we computed, for each view, the contribution rates of the Top-5 rules, entropy, normalized entropy, the number of effective rules, and the proportion of active rules (Table 2).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Statistics of rule-level self-attention weights for each view.

https://doi.org/10.1371/journal.pone.0348610.t002

The results showed that the normalized entropy of most views was close to 1, indicating that the majority of rules contributed meaningfully to the model’s decisions. A few views, such as Handwritten View 3 and EEG View 2, exhibited slightly lower normalized entropy, suggesting a more concentrated rule distribution, where a small number of rules carried greater decision weight. The contribution rates of the Top-5 rules remained around 12%–16% for most views, whereas Handwritten View 3 reached 22%, indicating that decisions in certain views relied more heavily on a few core rules. Conversely, views with lower Top-5 contribution rates, such as Handwritten View 5 (9.88%), showed a more balanced distribution of rule importance. Regarding the number of effective rules, most views approached the total number of rules, with the proportion of active rules exceeding 95%, while a few views (e.g., Handwritten View 3 and EEG View 2) had slightly fewer effective rules, consistent with their higher Top-5 contribution rates. These statistics clearly illustrate the role of rule-level self-attention in the MDA-TSK-FS model: the Top-5 contribution rate reflects the static importance distribution of rules across a view, whereas rule-level self-attention weights are dynamically assigned for each sample, enabling the model to flexibly select and emphasize rules based on input. In views with high Top-5 contribution rates, a few core rules are more likely to receive higher attention weights, dominating the model’s decisions; in views with lower Top-5 contribution rates, attention weights are more evenly distributed, reducing dependency on specific rules and enhancing robustness to complex or noisy samples.

To demonstrate human-interpretable semantics, the learned rules can be explicitly extracted. For instance, the highly attended Rule 13 in View 1 of Caltech7 (Fig 4) is expressed as:

(14)

The attention weights () directly reflect view relevance; as shown in Fig 4, View 1 prioritizes texture-specific rules (Rules 13, 23), while View 3 selects higher-level semantic rules (Rules 36, 41). Furthermore, compared to standard TSK baselines (e.g., our MDA-Basic) which suffer from uniform rule firing and overlap, MDA-TSK-FS enforces rule sparsity. As indicated in Table 2, the attention mechanism concentrates the decision weight on a compact subset of “effective rules” (the Top-5 rules), significantly reducing redundancy and mitigating rule explosion.

4.4 Ablation study

To further investigate the contribution of each key component in the MDA-TSK-FS model, we conducted a systematic ablation study by selectively removing or retaining the deformable Gaussian antecedent and the rule-level attention mechanism. Four model variants were constructed for comparison: MDA-Basic, the baseline version using standard Gaussian membership functions without any enhanced structures, served as the lower-bound reference. MDA(w/o Attn) augmented the baseline with deformable Gaussian antecedents, allowing each rule center to adaptively shift, but excluded the attention mechanism to evaluate the independent effect of the D module. MDA(w/o D) maintained static Gaussian antecedents while introducing the rule-level attention mechanism, where a multi-head attention network dynamically adjusted rule activation weights, thus assessing the attention mechanism’s effectiveness in the absence of deformable antecedents. Finally, the complete MDA-TSK-FS model integrated both deformable Gaussian antecedents and rule-level attention, combining antecedent flexibility with adaptive rule selection.

The experimental results, as shown in Table 3, indicate that MDA-TSK-FS achieved the highest accuracy across all five datasets, confirming the synergistic advantage of combining the two modules. Incorporating only the deformable antecedent structure, MDA(w/o Attn) demonstrated substantial gains over MDA-Basic on datasets such as Dermatology and EEG, showing that learnable center shifts significantly enhance the regional representational capacity of rules. Similarly, MDA(w/o D) consistently outperformed MDA-Basic, indicating that the rule-level attention mechanism effectively adjusts rule importance according to input, thereby improving discrimination capability. On the EEG dataset in particular, the deformable antecedent and attention mechanism individually improved accuracy by approximately 7% and 6%, respectively, and their combination further boosted performance to 69.75%. These results clearly demonstrate that both components are individually effective, and when combined, they offer stronger complementarity and improved generalization ability.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Ablation experiment results of MDA-TSK-FS.

https://doi.org/10.1371/journal.pone.0348610.t003

4.5 Computational complexity analysis

Addressing the potential concern of “rule explosion” in high-dimensional multi-view fuzzy systems, we evaluated the practical efficiency of the proposed MDA-TSK-FS. Although the introduction of the multi-head attention mechanism and deformable parameters increases the model’s complexity compared to standard zero-order TSK systems, the computational cost remains highly manageable for practical deployment.

We conducted inference speed tests on the Caltech7 dataset (which contains high-dimensional views up to 254 dimensions) using an NVIDIA RTX 4090D GPU. The model contains approximately 0.47 M trainable parameters. The average inference time is approximately 1.23 ms per sample, enabling real-time processing capabilities. Furthermore, as shown in Fig 2, the training process typically achieves convergence within 50–100 epochs, demonstrating high training efficiency. These results indicate that MDA-TSK-FS effectively balances high classification performance with computational feasibility.