Open-set recognition (OSR)

OSR focuses on distinguishing between known and unknown categories, particularly in scenarios where unknown samples coexist with predefined classes. Traditional OSR approaches can be categorized more accurately as follows:

1. Distance-Based Models [16–18]:

These models leverage feature space distances, such as Euclidean distance or cosine similarity, to separate known and unknown samples. Examples include nearest neighbor approaches and models that define thresholds based on class centroids or prototypes.

2. Reconstruction-Based Models [19,20]:

These methods utilize reconstruction errors from generative models, such as autoencoders or variational autoencoders, to identify unknown samples. Known classes are expected to have lower reconstruction errors compared to unknown ones.

3. Probability-Based Models [21,22]:

These approaches estimate class probabilities using techniques like Softmax. However, Softmax-based models often struggle to differentiate between in-distribution (ID) and out-of-distribution (OOD) samples due to their overconfidence in predictions.

In this work, we adapt energy-based models [4,23], traditionally designed for OOD detection, to JP-OSR tasks. By leveraging their capability to establish a clear boundary between known and unknown classes, we introduce a novel approach for jamming pattern recognition, improving the robustness and scalability of OSR in real-world scenarios.

JPs recognition and feature extraction

Recognizing and extracting features from jamming signals are critical tasks in complex electromagnetic environments, especially under adversarial conditions. Existing methods can be broadly categorized as follows:

1. Traditional Spectral Analysis [24]: Techniques like Short-Time Fourier Transform (STFT) and Discrete Wavelet Transform (DWT) analyze signals in the time-frequency domain and effectively capture local signal characteristics. However, they struggle to handle overlapping or dynamic JPs, particularly when dealing with previously unseen jamming types.
2. Deep Learning-Based Approaches [25,26]: Neural networks, such as Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs), automate feature learning and excel in closed-set classification tasks. However, these methods often fail to generalize to open-set scenarios, where novel jamming types emerge, and they lack effective temporal modeling for highly dynamic signals.

The proposed SWT addresses the limitations of both traditional spectral analysis and deep learning-based approaches by integrating wavelet-based frequency decomposition with the temporal dynamics of spiking neurons. By combining time-frequency analysis and biologically inspired dynamics, SWT enables robust feature extraction for complex JPs. Specifically, it captures both fine-grained local frequency patterns and global temporal structures, while introducing sparse, event-driven activations through spiking neurons to enhance energy efficiency and improve robustness, particularly for non-stationary or high-dimensional signals.

Moreover, SWT seamlessly integrates with open-set recognition modules, allowing for efficient detection and classification of both known and unknown jamming signals. This unified framework ensures adaptability to dynamic electromagnetic environments, making it a highly effective solution for real-world applications.

Proposed data preprocessing techniques

The raw time-domain waveforms of the sampled jamming signals contain excessive data and redundancy, making them unsuitable for direct training of neural networks. Meanwhile, spectrograms lack temporal information, limiting their effectiveness for jamming pattern recognition when used alone. Therefore, this study leverages time-frequency images generated by Short-Time Fourier Transform (STFT) as input features for JP-OSR, combining both spectral and temporal characteristics to enhance recognition performance. By segmenting the raw signals into fixed-length windows and applying a Hanning window function to each segment, the STFT generates a time-frequency matrix:

(1)

where X(t,f) represents the time-frequency distribution, w[n] is the window function, and N is the segment length. This matrix captures both temporal and spectral characteristics of the signal, making it suitable for subsequent processing.

To further enhance interpretability, the time-frequency matrix is normalized to amplify feature contrast:

(2)

The normalized matrix is then mapped to a pseudo-color heatmap using a defined color scale, transforming it into a visual representation that emphasizes differences across frequency bands. This processed time-frequency image provides a robust foundation for feature extraction and signal classification.

Spiking wavelet transformer (SWT) for feature extraction

In this framework, the SWT serves as the primary feature extractor, and its specific structural diagram is shown in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The overview of SWT.

https://doi.org/10.1371/journal.pone.0325381.g002

The SWT architecture is composed of three main components: the Spiking Patch Splitting (SPS) module, Spiking Encoder Blocks, and a linear classification head. The SPS module integrates convolutional layers (ConvBN), spiking layers, and max pooling operations to enhance frequency perception while preserving spatiotemporal relationships via relative position embedding. The Spiking Encoder Blocks further refine global features through frequency-aware token mixing (FATM) and MLP operations, ensuring compatibility with neuromorphic hardware. FATM serves as the cornerstone of the SWT, integrating three parallel branches: a wavelet-based frequency learner for capturing time-frequency domain features, a convolutional spatial learner for extracting spatial characteristics, and a pointwise convolutional channel mixer for aggregating cross-channel information. This architecture enables the effective decoupling and fusion of spatial and frequency information, enhancing the model’s ability to perceive high-frequency visual components. To further optimize efficiency, SWT employs trinary spike encoding and block-diagonal modular weight matrices, significantly reducing parameter count and computational complexity. Additionally, the incorporation of a membrane potential residual connection strategy bolsters cross-layer signal transmission, maintaining both the biological plausibility and computational efficiency inherent to spike-driven computation. Finally, the linear classification head efficiently maps the extracted features to classification outputs. Due to the requirements of this study, we will not utilize its classification head and will only employ the first two modules for feature extraction.

SWT demonstrates several advantages that make it a powerful tool for signal processing tasks:

1. Improved Frequency Representation: The frequency-aware token mixing mechanism (FATM) enhances the model’s ability to extract and utilize frequency characteristics, critical for processing time-frequency signals.
2. Neuromorphic Efficiency: The exclusive reliance on convolutional and MLP operations reduces computational complexity, making the architecture highly compatible with neuromorphic hardware for low-power applications.
3. Comprehensive Feature Learning: The Frequency Learner module effectively captures multi-scale spectral features using block-diagonal matrix operations, enabling a thorough analysis of signal characteristics.
4. Hybrid Design: By combining spiking neural networks (SNNs) with transformers, SWT capitalizes on the low-energy consumption of SNNs while benefiting from the superior representational power of transformers.

SWT is particularly well-suited for JPR tasks due to its exceptional ability to capture spectral features through its frequency-aware design. The integration of temporal modeling ensures that dynamic variations in jamming signals are retained and exploited for classification. Furthermore, the hardware-friendly design makes SWT ideal for real-time deployment in resource-constrained environments, such as jamming detection and classification systems. Its combination of efficiency, accuracy, and adaptability positions it as a promising solution for JPR and analysis.

Maximum class separation (MCS) as inductive bias in one matrix

This paper introduces MCS as a foundational geometric inductive bias for classification tasks. Unlike conventional methods that jointly learn class embeddings and feature representations, MCS employs a pre-constructed matrix P to encode maximally separated class vectors on a hypersphere. By fixing P, the model simplifies the optimization process, decouples class representation learning from feature alignment, and ensures consistent class separation throughout training. This approach not only enhances decision boundaries but also imposes natural geometric constraints through the hyperspherical arrangement of P, resulting in a more structured and discriminative feature space. More importantly, separation does not require optimization but can be resolved in closed form prior to training and integrated into the network, thereby effectively enhancing the model’s efficiency.

Fixed class representation matrix.

The fixed class representation matrix P is the backbone of the MCS framework. It is designed with the following key properties:

1. Dimensionality: , where C represents the number of classes, and C–1 represents the embedding dimensionality.

2. Unit Norm Constraint: Each column of P, corresponding to a class vector P_c, is normalized to unit length:

(3)

3. Equal Pairwise Similarity: All class vectors maintain uniform cosine similarity:

(4)

4. Zero-Centered Distribution: The mean of all class vectors is zero, ensuring symmetrical distribution:

(5)

To achieve these properties, P is recursively constructed, embedding each new class vector orthogonal to the current subspace. This recursive method guarantees that the new vectors maintain maximum separation on a hypersphere. The closed-form separation recursive process is shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Separation in closed form through recursion.

(a) illustrates the process of obtaining maximally separated vectors for three classes on , starting from two class vectors on a line. Similarly, (b) demonstrates the step from three to four classes on .

https://doi.org/10.1371/journal.pone.0325381.g003

Feature projection.

After extracting features using SWT, each input is transformed into a (C–1)-dimensional feature embedding using a feature mapping function :

(6)

The classification logits o_i are then computed via the dot product of with the pre-constructed matrix P:

(7)

This process aligns each feature vector with the predefined directions in P, with each logit o_i,c representing the similarity of the feature vector to a specific class.

Optimization objective.

The network is trained using a cross-entropy loss , which ensures that the feature vectors are aligned with the correct class vectors:

(8)

where y_i denotes the ground truth label for , and N is the number of training samples.

This loss minimizes the discrepancy between the network output and the correct class, implicitly driving to align with the target class vector .

Energy-based JP-OSR

The MCS framework can be extended for open set detection tasks by incorporating an energy-based model. The energy score for a sample is computed from the classification logits: The energy score E acts as a compact measure of uncertainty or confidence in the input belonging to the known classes:

(9)

Known samples tend to have lower energy scores because their logits o_i align with the trained class distributions. Unknown samples often produce dispersed or inconsistent logits, leading to higher energy scores, making a reliable indicator of unknown inputs. Algorithm 1 illustrates how we apply the energy-based model for JP-OSR.

Algorithm 1. Self-adaptive energy-based JP-OSR algorithm.

For in-distribution(ID) samples, the energy score E(x) should be minimized as much as possible. For out-of-distribution(OOD) samples, the energy score E(x) should be maximized to exceed a specific threshold . The energy loss for ID and OOD samples can be expressed as:

(10)

where: is the total number of ID samples in the training set. is the total number of auxiliary OOD samples. is the energy score of the i-th ID sample [27]. is the energy score of the j-th OOD sample. and are the predefined margins for ID and OOD samples, respectively. The first term penalizes ID samples whose energy scores exceed the margin . This ensures that the energy for ID samples remains low. The second term penalizes OOD samples with energy scores below m_out, ensuring that their energy scores are pushed higher than the threshold.

The total loss for joint classification and open set detection is given by:

(11)

where control the relative importance of the energy components.

A threshold on E(x) is used to distinguish ID from OOD samples.

(12)

Where C + 1 represents the class label for unknown JPs.

Energy-based models(EBMs) are particularly suitable for anomaly detection due to the following reasons:

1. Unified scoring mechanism: EBMs naturally assign low energy (i.e., high compatibility) to in-distribution data and high energy to out-of-distribution or anomalous data, providing a unified and interpretable scoring criterion.
2. Model-agnostic integration: EBMs can be integrated with both generative and discriminative models, and can leverage deep neural networks to model complex data distributions.
3. Training stability and generality: Unlike explicit generative models that require likelihood estimation or reconstruction, EBMs allow implicit density modeling without the need to fully specify the output distribution. Enhanced robustness: Empirical studies [4] have shown that EBM-based methods are more robust to adversarial and high-dimensional anomalies than reconstruction- or density-based methods.

Data acquisition

To evaluate the proposed method for JP-OSR, synthetic datasets were designed to simulate complex electromagnetic environments. Unlike traditional datasets, where jamming and signal phases are carefully isolated to eliminate cross-interference, the generated data here incorporates diverse JPs to emulate practical communication scenarios. This ensures the robustness and generalizability of the model across different JSR and jamming types. The parameter configurations of each synthetic jamming signal are shown in Table 1. In addition to our synthetic data, we incorporate the publicly available Radio Frequency Jamming Dataset from Kaggle, which provides real-world spectral scans collected via commercial WLAN spectral-scan NICs (Qualcomm Atheros QCA9880 or Doodle Labs NM-DB-3U) combined with the JamRF jamming toolkit. This dataset comprises three labeled scenarios-Deceptive Jamming(DJ), Constant Jammer(CJ), and Reactive Jammer(RJ)-across the 2.4/5 GHz bands.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Jamming signal configurations and parameters.

https://doi.org/10.1371/journal.pone.0325381.t001

To verify the effectiveness of the proposed method for JP-OSR, for the first three combinations based on the synthetic dataset, we selected six common types out of the nine jamming signals as known jamming signals, and the remaining three as unknown jamming signals. However, only two of the unknown jamming signals were included in the test set each time, while the remaining unknown jamming signal was used to validate the hyperparameters of the energy model, resulting in a total of three experimental groups. For the fourth combination, we first resampled the real-world data to match the format of the synthetic dataset, and then performed mixed training on the combined data. The specific divisions are shown in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Experimental data combination index.

https://doi.org/10.1371/journal.pone.0325381.t002

1. Known Jamming Classes

The following six jamming types are designated as known during training: Continuous Wave (CW), Multi-Tone Jamming (MTJ), Linear Frequency Modulation (LFM), Partial Band Noise Jamming (PBNJ), Pulsed Periodic Noise Jamming (PPNJ), and Frequency-Hopping Jamming (FHJ). Additionally, in the fourth combination the real-world dataset’s Deceptive Jamming (DJ) and Constant Jammer (CJ) samples are also included as known classes in training. These represent commonly encountered JPs with distinct frequency and temporal properties.

2. Unknown Jamming Classes

The following three jamming types are excluded during training and considered unknown: Noise Linear Frequency Modulation (NLFM), Pulsed Constant Frequency Jamming (PCFJ), and Tracking Jamming (TJ). Furthermore, the real-world dataset’s Reactive Jammer (RJ) is treated as an unknown class in the fourth combination. These jamming types introduce additional complexity, mimicking realistic scenarios where new or unseen JPs may emerge.

This configuration ensures comprehensive evaluation of the model’s ability to handle diverse scenarios, including both known and unknown interference types. Moreover, since we adopt the energy model as a tool for open set recognition in the later stages, a portion of OOD data is required for auxiliary training during the training phase. This OOD data must not overlap with the nine types of jamming signals. Therefore, we generate random noise with no significant similarity to actual jamming signals, such as Gaussian white noise and colored noise. These noise signals exhibit uniform spectral distribution and are independent of the modulation patterns or spectral characteristics of the target signals, effectively avoiding confusion with known jamming signals.

Experimental setup and hyperparameters

In all adaptive-threshold experiments we set the temperature T = 1.0 in the energy formula, the scaling factor to define per-class thresholds , and the EMA smoothing coefficient . We use a confidence cutoff p_i>0.9 on the softmax max-score to admit only high-confidence known-class samples for online threshold updates. These hyperparameters were chosen via grid-search on the validation splits (synthesized and real-world) to balance false-reject versus false-accept trade-offs, and held fixed for all “Combination 4” mixed-training experiments. In the SWT framework, the hyperparameter Blocks was uniformly set to 4, while in MCS, the hyperparameter was consistently fixed at 1.0.

The F1 score, calculated using Eq (13), incorporates the true negative rate (TNR) and the true positive rate (TPR). In the subsequent experiments, known jamming samples are labeled as positive, while unknown jamming samples are labeled as negative.

(13)

Open-set recognition performance

The confusion matrix for JP-OSR using the energy model is presented in Fig 4a. In this experiment, Combination 1 was selected, with each type of jamming signal comprising 500 test samples in the study. The results indicate that nearly all known jamming samples are correctly classified, with only a negligible number mistakenly identified as unknown. Importantly, no samples were misclassified as other known types of jamming. Furthermore, among the 1,000 samples from the two unknown jamming types, only 15 were misclassified. These results strongly validate the effectiveness and robustness of the proposed method.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Confusion matrix and accuracy of energy-based JP-OSR.

https://doi.org/10.1371/journal.pone.0325381.g004

Fig 4b illustrates the average accuracy curve of our proposed method across the first three experimental data combinations. As depicted in the figure, when the JSR exceeds –2 dB, the algorithm demonstrates robust recognition accuracy for all known JPs, achieving an average accuracy of 99.5%. At an JSR of -4 dB, while the accuracy is slightly lower, it still maintains a respectable average of 93.4%. Furthermore, the recognition of unknown JPs, despite some variability, consistently surpasses 95%, with an average accuracy of 96.4%. These results underscore the efficacy of our approach for JP-OSR.

To measure and report the computational performance of the proposed energy-based discrimination strategy, we conducted comparative experiments against several representative open-set anomaly detection baselines, including: OpenMax (based on extreme value theory) [18], MSP with ODIN (temperature-scaled logits and input perturbation), Mahalanobis Distance-based detection, G-OpenMax [28], and JC-GAN [25]. To ensure fairness, these methods employed the Spiking Wave Transformer (SWT) as the feature extraction backbone, with parameter settings aligned with defaults or tuning strategies from the original papers. Traditional one-class classifiers, including SVDD and OCSVM [29], as well as the Vision-Transformer-based OSR method ARPL [17] and reconstruction-based methods like VAEs [30], were also included for comprehensive comparison. The evaluation metrics used were TPR, TNR, Precision, and F1-score. Each experiment was repeated five times, and the average results are summarized in Table 3. The results show that the proposed method achieves the best overall performance across all metrics, particularly attaining the highest TNR, Precision, and F1-score. Although ZSL-JPR and JCGAN marginally outperform our method in TPR, ZSL-JPR exhibits sensitivity to outliers, resulting in a conservative threshold selection and lower TNR. Notably, the use of SWT as a unified feature extractor improves the robustness of all compared methods to some extent.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Comparison between the proposed method and four mainstream methods.

https://doi.org/10.1371/journal.pone.0325381.t003

To determine whether the observed performance differences are statistically significant, we performed paired t-tests on the F1-scores obtained from the five runs. The p-values for comparisons between the proposed method and each baseline are all less than 0.01, indicating that the improvements are statistically significant at the 99% confidence level.

Further comparative analysis reveals the reasons for the observed advantages. First, the SWT captures both temporal dynamics and frequency structures within a biologically-inspired spiking framework, allowing better feature representation even under low JSR conditions or in highly dynamic signal environments. Second, the proposed energy-based discrimination aggregates confidence scores across all output logits, creating a more stable decision boundary compared to traditional softmax-based approaches like OpenMax and MSP+ODIN, which are vulnerable when known and unknown distributions overlap. In contrast, methods relying on Mahalanobis distance or Gaussian assumptions (e.g., Mahalanobis, G-OpenMax) suffer when feature distributions deviate from unimodal Gaussian forms, leading to performance degradation. Reconstruction-based methods (e.g., autoencoders, VAEs) rely on the assumption that anomalies cannot be well reconstructed. However, such methods often suffer from generalization issues, where the model may still reconstruct anomalous samples well. Overall, these results confirm that the proposed method not only significantly improves detection performance under both synthetic and real-world scenarios but also establishes a robust and statistically validated framework for open-set jamming signal recognition.

Computational efficiency and latency

To evaluate the inference efficiency of different open-set recognition methods, we measured their per-sample latency, throughput (in frames per second), and model size. All evaluations were conducted on an NVIDIA RTX 3090 GPU with batch size set to 1, simulating real-time processing scenarios.

Unlike prior works that rely on CNNs (e.g., SVDD, JCGAN) or transformer-based backbones (e.g., ARPL), our method adopts a lightweight SWT coupled with MCS. This design enables efficient frequency-domain feature extraction with reduced computational cost.

As shown in Table 4, our method achieves a favorable balance between latency and throughput, with a significantly smaller model size (17.5 MB) compared to most deep learning-based methods. Although SVDD and OCSVM are slightly faster, they rely on much simpler or hand-crafted features and show limited scalability, whereas our method leverages a more expressive feature extractor while maintaining competitive efficiency. These results demonstrate the suitability of our method for real-time RF jamming detection in resource-limited deployment environments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Comparison of computational efficiency (Inference-only, batch size = 1).

https://doi.org/10.1371/journal.pone.0325381.t004

Threshold sensitivity analysis

To assess the stability of our self-adaptive threshold under varying jamming-to-signal ratios (JSR) and noise conditions, we performed a grid search over the scaling factor and the EMA coefficient . For each pair, we evaluated the F₁-score on test signals at JSR values of –4, –2, 0 and  + 2 dB with additive Gaussian noise at SNR = 20 dB.

Fig 5 shows a representative heatmap of F₁-scores at JSR = –2 dB and SNR = 20 dB, demonstrating that the region and yields consistently high F₁ (> 0.98). Table 5 reports the standard deviation of F₁ across all tested JSR values for each (at fixed ), confirming minimal performance variation when .

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Sensitivity of -score to adaptive-threshold parameters and at JSR = –2 dB, SNR = 20 dB.

https://doi.org/10.1371/journal.pone.0325381.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Standard deviation of -score across JSR  dB for each (with ).

https://doi.org/10.1371/journal.pone.0325381.t005

These results demonstrate that our adaptive-threshold mechanism is robust: when and , the F₁-score variation remains below 0.01 across a 6 dB JSR range, confirming stability under diverse interference and noise conditions.

Ablation study on SWT and MCS

To verify the effectiveness of SWT and MCS in the proposed method, we conduct experiments on three datasets, replacing only one component at a time. When SWT is not used, the CNN network from the literature [25] is used as a substitute. The experimental results are shown in Table 6. As can be seen from the table, when both components are used together, the F1-score is higher. However, when either component is missing, the F1-score decreases slightly, indicating that both SWT and MCS have a positive effect on the model’s performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Ablation experiments on the F1-score of the proposed method.

https://doi.org/10.1371/journal.pone.0325381.t006

The key hyperparameters in SWT are the number of blocks (denoted as Blocks), tested within the range [2, 4, 8], while in MCS, the primary hyperparameter is the hypersphere radius (denoted as ), tested within [0.1, 2.0]. When jointly optimized in Combination 1–3, the best F1-score was achieved with Blocks = 4 and . The experimental results clearly indicate that excessive block partitioning (Blocks > 4) significantly impairs local anomaly detection performance, while an excessively large hypersphere radius () adversely affects sensitivity to tail classes. Optimal system performance is achieved only through careful parameter tuning, with the best results obtained at Blocks = 4 and , which effectively balances these competing requirements.