Comparison with representative existing methods

To better situate CFF-Net within the current literature, Table 1 summarises the key design dimensions of representative methods in parameter-efficient and few-shot time series forecasting, including S2S-FDD [9], Time-LLM [10], LLM4TS [11], PatchInstruct [12], TRACE [13], and the proposed CFF-Net. The comparison covers whether the method employs a frozen backbone, the type of semantic information used, the adaptation strategy, the primary application scenario, and whether it supports few-shot settings. As shown in Table 1, most existing approaches either rely on full-parameter fine-tuning or employ semantic information in a coarse-grained manner. In contrast, CFF-Net integrates structured textual prompts with a scale-and-shift modulation mechanism on top of a fully frozen backbone, enabling fine-grained, sample-efficient adaptation across domains. Unlike S2S-FDD, which targets fault diagnosis through natural language bridging in a zero-shot manner, CFF-Net focuses on multi-domain regression forecasting with explicit dynamic modulation factors derived from language model representations.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of CFF-Net with representative methods in terms of key design dimensions. A tick indicates the property is present; a dash indicates it is absent or not the primary focus.

https://doi.org/10.1371/journal.pone.0353130.t001

Parameter-efficient fine-tuning methods

The use of large pre-trained models in time series forecasting reveals several drawbacks, such as high computational overhead, inefficient transferability, and poor generalization. For instance, Lv et al. [14] introduced the AdaLomo method, which effectively utilizes labeled data, but the computational cost of updating parameters exceeds 90%, resulting in substantial overhead. Lv et al. [15] developed the LOMO framework to reduce memory pressure through an optimization strategy, but the computational cost remains high. Liu et al. proposed the BFA-LSTM model [16], which uses a sliding window attention mechanism to improve short- and medium-term predictions; however, it still relies on full-parameter updating, limiting flexibility and generalization across different application scenarios.

Parameter-efficient fine-tuning (PEFT) presents a viable solution for adapting models to new tasks with minimal resource consumption. The core concept behind PEFT is to freeze most of the backbone parameters and train only a small number of lightweight modules. This approach significantly reduces training costs while maintaining nearly the same performance. Notable PEFT methods include Adapter, LoRA, Prompt Tuning [17], and ReFine [18]. For example, Pu et al. [19] employed a LoRA-based strategy for satellite detection, achieving performance close to full-parameter fine-tuning by training only a small portion of the parameters. Xu [20] proposed the PeFAD framework, which combines cross-modal transfer and federated optimization to reduce both communication and privacy costs. In traffic forecasting, Zhan et al. [21] designed an online hyperparameter optimization algorithm, maintaining accuracy while enhancing real-time applicability. In the medical field, Gupta et al. [22] introduced Fourier FT to improve the adaptability and parameter efficiency of ICU time series modeling. In another study, Gupta et al. [23] developed the LTM framework, integrating time series, text, and knowledge graph prompts for forecasting, imputation, and anomaly detection. Kim et al. [24] enhanced cross-domain robustness using gradient-free test-time adaptation, while Dong et al. [25] incorporated contextual and chain-of-thought prompts into anomaly detection. Qu et al. [26] proposed the Connect Later method, which combines self-supervised pre-training and data augmentation to mitigate distribution shifts during transfer. In contrast to methods that use single-stage heterogeneous feature modeling, Li et al. [13] introduced the TRACE framework, utilizing gating and reconstruction heads to improve heterogeneous feature modeling. Bian et al. [27] proposed the ALLM4TS framework, which enhances temporal representation using self-supervised multi-segment prediction. In contrast to methods that employ context-based fine-tuning for adaptive inference without target-domain labels, achieving near full-parameter tuning performance [28], our method outperforms traditional methods even without fine-tuning. Shi et al. [29] strengthened robustness against severe outliers using a meta-learning reweighting strategy, while Bumb et al. [12] introduced the PatchInstruct framework, which combines decomposition, partitioning, and neighbourhood-enhanced prompts into prediction without requiring fine-tuning to achieve high-quality results. Unlike methods that apply reweighting layers for small-sample prediction [30], our method further improves model efficiency.

Although these methods enhance parameter efficiency and transferability, they still rely on large-scale data and lack a semantic modeling mechanism, limiting their applicability to data-abundant scenarios. To address these issues, we propose CFF-Net, a semantic-driven lightweight fine-tuning framework. By freezing most backbone parameters and training only a few fusion modules, CFF-Net achieves efficient fine-tuning even with limited data.

Few-shot time series prediction methods

In applications such as medical diagnosis, financial modeling, and agricultural monitoring, the limited availability of training samples and high annotation costs significantly restrict the performance of time series forecasting models. Traditional deep learning methods rely on large datasets and full-parameter fine-tuning, which often leads to unstable training, overfitting, and weak generalization in few-shot settings.

To address these challenges, research has shifted toward few-shot time series modeling, focusing on network optimization, knowledge transfer, self-supervised learning, and the use of LLMs to improve robustness and generalization under resource-constrained conditions. In terms of structural optimization, Chudy et al. [31] proposed a robust economic prediction interval model. Li et al. [32] introduced FS-ADAPT, a method that performs unsupervised domain adaptation for anomaly detection. Su et al. [33] enhanced fault prediction through pseudo-meta task partitioning, while Tong et al. [34] developed Taylor-nets for lightweight and accurate forecasting. Wu [35] employed conditional mutual information with contrastive loss to alleviate overfitting in small-data scenarios, and Iwana [36] combined DTW with shape descriptors for data augmentation. With the rapid progress of LLMs, their application to few-shot time series forecasting has gained increasing attention. Chang et al. [11] proposed LLM4TS, which employs a two-stage fine-tuning approach and multi-scale fusion to enhance prediction accuracy. Jin et al. [10,37] presented Time-LLM, integrating reprogramming and prefix prompting to achieve strong zero-shot and few-shot performance. Taga et al. [38] developed TimePFN with Gaussian process pre-training, reaching near full-data accuracy with only 50–500 samples. Gruver et al. [39] proposed LLMTime, which reformulates time series as numeric strings for zero-shot prediction. Jiang et al. [40] introduced FSTLLM by combining spatio-temporal graph neural networks with LLMs to enhance robustness and accuracy. Khurana et al. [41] proposed FORECASTPFN for low-resource climate and economic forecasting. Zhou et al. [42] presented ZeroTS, a framework supporting zero-shot prediction and task adaptation. Other studies explored diverse directions. Iwata et al. [43] designed an attention-based recurrent network to improve feature transfer. MetaTrans-FSTSF [44] integrated Transformer and meta-learning, achieving strong results in flood forecasting with remote sensing data. Tang [45] proposed DPSN with a dual-prototype structure for multi-granularity modeling. Cheng [46] developed MetaGP, which combines Gaussian kernels with kernel search for modeling interpretable long-term dependencies. Ma et al. [47] improved prediction by integrating meta-learning with residual stacking. Zhao et al. [48] introduced DeepAR, combining deep neural networks with probabilistic modeling for cost forecasting. Ekambaram et al. [49] proposed TTMs, which leverage fast pre-training to enhance multivariate few-shot forecasting.

Although these methods have advanced feature representation, model architecture, and transfer mechanisms, several limitations remain. Many approaches rely heavily on structural optimization or homogeneous modality transfer. They lack systematic modeling and flexible control of semantic information, leaving the potential of language knowledge underutilized. To overcome these gaps, we propose CFF-Net, a lightweight, prompt-driven framework. By freezing the backbone predictor and introducing learnable semantic factors with channel-level adjustment, CFF-Net achieves efficient adaptation and robust prediction in both cross-domain and low-resource scenarios.

The proposed CFF-Net framework

This section details the overall framework design of CFF-Net, as shown in Fig 1, which combines the advantages of PatchTST, GPT-2 and MLP, and achieves efficient fusion of features and fine-tuning of prediction results through prompt-guided modulation.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. CFF-Net architecture.

The proposed framework adopts a dual-branch collaborative design. In the upper branch, the numerical time series is fed into a frozen PatchTST module to produce an initial forecasting result. In the lower branch, the same sequence is converted into structured natural language descriptions and processed by a GPT-2 text encoder to extract semantic representations. These representations are mapped by a multi-layer perceptron into dynamic modulation factors for feature scaling and shifting, which are subsequently applied to refine the final prediction. Solid arrows indicate data flow, while dashed boxes denote modules with frozen parameters.

https://doi.org/10.1371/journal.pone.0353130.g001

The CFF-Net framework adopts a two-branch collaborative architecture that effectively combines temporal features with semantic cues to improve the accuracy and generalisation of time series prediction. The upper branch focuses on numerical sequence modeling: for multivariate time series inputs, the frozen PatchTST module extracts temporal features. PatchTST employs a subsequence patching strategy, segmenting long sequences into local segments and using channel-independent attention to capture inter-channel feature distributions and long-range dependencies, which results in an initial prediction, as shown in Fig 1. The lower branch establishes a semantic-guided pathway: the same numerical sequence is converted into structured natural language descriptions and processed by a frozen GPT-2 model to extract semantic representations. These semantic vectors are then projected through an MLP into two dynamic modulation factors, the scaling factor and the shifting factor , which refine the PatchTST branch output. This design follows the Scale-and-Shift Feature (SSF) strategy [50], which performs element-wise scaling and shifting on the predictions, preventing dimension mismatch and reducing information redundancy that can arise when directly merging high-dimensional semantic vectors.

In the temporal branch of CFF-Net, part of the PatchTST backbone network remains frozen, with its temporal modeling capabilities refined through supervised pre-training. A key innovation in this branch is the channel-independent Transformer architecture. In the architecture, a multivariate input series is broken down into multiple single-variable subsequences, each of which is divided into sub-blocks of length P after instance standardisation, as shown in Eq (1).

(1)

where N denotes the number of patches, P represents the patch length, S is the stride, and L indicates the total length of the input time series.

Each patch is projected into the Transformer feature space through a learnable linear transformation . A positional encoding is then added to preserve the temporal order of the sequence, as formulated in Eq (2).

(2)

where denotes the patch representation projected into the latent space with positional encoding, is the original patch vector, represents the linear projection matrix, and denotes the positional encoding matrix.

The embedded patches are subsequently fed into the Transformer encoder and processed via a multi-head self-attention mechanism to capture the internal temporal dependencies of the time series. The output of each attention head is computed according to Eq (3).

(3)

where , , and denote the query, key, and value matrices, respectively, and represents the scaling factor.

The final output is dimensionally aligned temporal features . The design reduces the complexity of a traditional Transformer from O(T²) to O(N²) (), which is highly consistent with the lightweighting goal of CFF-Net.

CFF-Net effectively combines temporal and semantic modalities through its design. The PatchTST backbone handles local temporal modeling, while the GPT-2 semantic branch refines this model using dynamic modulation factors (). By adopting a frozen-parameter strategy, training is confined to the MLP fusion module, thus minimizing computational costs without sacrificing pre-trained knowledge. This fusion architecture, which integrates temporal feature extraction with semantic-guided modulation, forms the basis for subsequent feature encoding and fusion, striking a balance between prediction accuracy and parameter efficiency.

Semantic prompt-driven fine-tuning

This section introduces the two key components of CFF-Net: the SPM and the DSM, explaining their roles in integrating numerical sequences with semantic features. The SPM converts time series data into textual prompts and extracts semantic features, while the DSM generates scaling and shifting factors from the semantic vectors to adjust prediction results, as shown in Fig 2. The following subsections detail the implementation of these modules and their collaborative mechanism.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Process of numerical-to-semantic reconciliation in CFF-Net.

An example based on a temperature time series is illustrated. The figure depicts the interaction between the SPM and the DSM. First, numerical sequences in CSV format are converted into structured textual descriptions. Next, a frozen GPT-2 model is employed to extract semantic representations that capture domain-specific semantics and temporal characteristics. These representations are then mapped by a multi-layer perceptron into scaling and shifting factors, which are applied to modulate the PatchTST prediction outputs through a scale-shift operation, resulting in calibrated and semantically enhanced forecasts.

https://doi.org/10.1371/journal.pone.0353130.g002

Semantic prompt encoding module.

The SPM transforms raw numerical sequences into structured natural language descriptions through a rule-based template mechanism. Given a univariate sub-sequence , the module first computes a set of statistical descriptors, including the mean , standard deviation , minimum , maximum , and the linear trend slope k estimated by least-squares regression. Dynamic Time Warping (DTW) is then applied to measure the similarity between the current window and a set of reference patterns, and the index of the closest reference is used to select a corresponding natural language phrase that describes the dominant temporal pattern. These elements are composed into a structured sentence following a fixed template, for example: the sequence spans T time steps; the mean value is , with a standard deviation of ; the minimum is and the maximum is ; the overall trend is [DTW pattern phrase], with a slope of k per step.

This sentence is tokenised and fed into GPT-2. After processing the text sequence, the hidden representation from the final transformer layer is extracted and aggregated via mean pooling to obtain a fixed-length semantic embedding, as defined in Eq (4).

In the semantic prompt encoding module, numerical sequences are first transformed into structured natural language descriptions and subsequently fed into a frozen GPT-2 model. As a pre-trained language model, GPT-2 is capable of generating contextualised semantic representations from textual inputs. After processing the text sequence, the hidden representation from the final layer is extracted as the semantic embedding, as defined in Eq (4).

(4)

The resulting semantic vector s () encodes both the statistical features of the original time series and domain-specific semantic knowledge, offering a compact and effective representation for subsequent semantic modulation. This process leverages the inherent capabilities of the pre-trained model, eliminating the need for additional alignment mechanisms and effectively controlling model complexity.

Dynamic semantic modulation module.

The DSM module receives the semantic vector output by the SPM and constructs a dynamic modulation pathway using the SSF mechanism. The semantic vector is first passed through a shallow MLP to generate modulation factors, which are then separated into a scaling factor and a shifting factor . The scaling factor is constrained to the range [0, 2] using a Sigmoid function to control the feature scaling intensity, while the shifting factor is constrained to using a Tanh function to achieve small feature shifts, as shown in Eqs (5) and (6).

(5)

The output vector z is then split into two parts to generate the scaling and shifting parameters. Specifically, the first d dimensions are used to compute the scaling factor , while the remaining d dimensions are used to compute the shifting factor , as defined in Eq (6).

(6)

where denotes the sigmoid function. As a result, and .

The range choices for and are motivated by the instance normalisation applied upstream in PatchTST, which standardises each channel to zero mean and unit variance before the Transformer encoder. In this normalised space, a scaling range of [0, 2] allows the semantic branch to amplify or suppress temporal features without introducing unbounded distortion, and a shift range of corresponds to perturbations of approximately 10% of the unit standard deviation, which is sufficient to bias predictions toward domain-specific baselines while avoiding large deviations that would be amplified upon inverse normalisation. Because the modulation is applied in the normalised latent space and the inverse normalisation is a fixed affine transform determined by the channel statistics of the input window rather than by the learned parameters, the amplification of upon inverse normalisation is bounded by the channel standard deviation of the current batch and does not diverge during inference. The chosen bounds were validated empirically by sweeping the upper scaling limit over {1.0, 1.5, 2.0, 3.0, 4.0} and the shift limit over {0.05, 0.10, 0.20, 0.50}; the values of 2.0 and 0.1 consistently produced the lowest validation MSE across all four datasets, indicating that they are not dataset-specific constants but broadly applicable hyperparameters.

After obtaining and , the PatchTST output is modulated in an element-wise manner, as formulated in Eq (7).

(7)

where ⊙ denotes element-wise multiplication.

Dataset description and implementation details

To thoroughly evaluate the generalization ability and predictive performance of the proposed CFF-Net model, we conducted experiments on the TCTS dataset and three publicly available multi-domain datasets. The TCTS dataset tracks a three-month growth cycle of tomato crops, from seedling to flowering, with high-frequency measurements collected throughout the period. Key physiological indicators include leaf SPAD values and light intensity. For the public benchmarks, we selected three widely used datasets: Weather (meteorological indicators), Exchange_rate (exchange rates from eight countries), and ILI (U.S. influenza case statistics). These datasets cover agriculture, healthcare, and finance, making them both representative and challenging.

In terms of model design, the PatchTST and GPT-2 modules in CFF-Net were both frozen to preserve their pre-trained parameters. PatchTST was used to predict the next 96 time steps, with the results further refined. Text features extracted by GPT-2 were aligned and fused with predictions through an MLP. The selection of key hyperparameters was guided by a systematic grid search combined with the results of the ablation study in Table 6. The MLP depth was varied over {4, 8, 12, 16} layers and the hidden dimension over {256, 512, 1024, 2048}; 16-layer networks with a hidden dimension of 256 consistently yielded the best or near-best performance across all datasets, suggesting that greater depth promotes better temporal feature abstraction while a compact hidden size reduces the risk of overfitting in low-resource settings. The learning rate of was selected after a log-scale sweep over , where caused oscillatory training loss and led to under-convergence within 150 epochs. Weight decay of 0.01 was chosen following standard practice for the AdamW optimiser to prevent over-regularisation of the small fusion module. A batch size of 512 was selected to balance training efficiency with gradient stability given the dataset sizes. Up to 150 training epochs were allowed with early stopping based on validation loss. LeakyReLU was used as the activation function, with Sigmoid and Tanh serving as gating mechanisms for and respectively. During training, only a small set of parameters in the MLP module was updated, ensuring efficient fine-tuning and stable transferability.

Statistical feature baseline comparison

To verify that the GPT-2 semantic embeddings provide genuine information gain beyond simple statistical summarisation, we replace the SPM and GPT-2 in CFF-Net with a lightweight statistical feature extractor that computes five hand-crafted descriptors for each input window: mean, standard deviation, minimum, maximum, and linear trend slope. These five scalars are projected by a single linear layer into the same space used by the GPT-2 branch, and the same DSM and MLP pipeline is then applied. Table 2 reports the results on all four datasets under the 20% training regime, where the sample efficiency benefit of semantic information is expected to be most pronounced.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Comparison of GPT-2 semantic embeddings against a statistical feature baseline. All models use the same DSM and MLP configuration (16 layers, 256 hidden units) under the 20% training sample regime.

https://doi.org/10.1371/journal.pone.0353130.t002

As shown in Table 2, CFF-Net with GPT-2 embeddings achieves better results than the statistical feature baseline in most cases, but not across all metrics. This indicates that the contextualised representations produced by GPT-2 can provide useful temporal semantic information beyond hand-crafted descriptors. For example, GPT-2 embeddings improve MSE on the Weather dataset, although the MAE is worse than that of the statistical feature baseline. The improvement is relatively clear on the TCTS dataset, while the difference on the Exchange_rate dataset is smaller, suggesting that simple statistical descriptors can already capture part of its relatively smooth trends.

Inference cost analysis

We acknowledge that running a frozen GPT-2 inference for each input window introduces additional computational cost compared to a purely numerical model. To quantify this overhead, Table 3 reports the average inference time per sample and the total parameter count for CFF-Net and the baseline models on a single NVIDIA A100 GPU.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Inference cost comparison. Inference time is the average wall-clock time per sample measured over 1,000 forward passes on a single NVIDIA A100 GPU. Trainable params refers to the number of parameters updated during fine-tuning.

https://doi.org/10.1371/journal.pone.0353130.t003

Although the GPT-2 component increases per-sample inference time by four times compared to PatchTST alone, with the inference time for CFF-Net being 3.2 ms per sample and 0.8 ms per sample for PatchTST, this increase in time is mitigated by two considerations. First, the text description for each input window is deterministic and can be cached at deployment time, so GPT-2 need only be evaluated once per unique window content rather than once per forward pass. Second, the proportion of trainable parameters is only 0.08% of the total model, meaning that the fine-tuning phase itself remains highly efficient. For edge or real-time applications where inference latency is critical, the statistical feature variant (Table 2) offers a viable lightweight alternative with a modest accuracy trade-off.

Model performance evaluation and comparative analysis

In this section, we evaluate the predictive performance of the proposed CFF-Net model under limited data conditions. Three standard metrics are adopted, including MSE, MAE, and RMSE, to assess prediction accuracy and robustness. The performance of CFF-Net is compared with the baseline PatchTST across multiple datasets using different training sample proportions (100%, 50%, 30%, and 20%). The experimental results are summarized in Table 4.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Comparison of PatchTST and CFF-Net under different sample proportions on multiple datasets.

https://doi.org/10.1371/journal.pone.0353130.t004

The table presents the performance comparison between the proposed CFF-Net and the baseline PatchTST across several publicly available datasets, including Weather, Exchange_rate, and ILI, as well as the TCTS dataset. Overall, CFF-Net achieves lower errors than PatchTST in many settings, but the improvements are not uniform across all datasets, metrics, and sample proportions. For example, on the Weather dataset with 20% of the training samples, CFF-Net reduces the MSE from 1.01 to 0.89 and the RMSE from 1.05 to 0.97, corresponding to reductions of 11.88% and 7.62%, respectively. On the TCTS dataset under the same sample proportion, the MSE decreases from 1.06 to 0.99 and the RMSE decreases from 0.94 to 0.90, corresponding to reductions of 6.60% and 4.26%, respectively. These results suggest that the prompt-driven CFF-Net can improve forecasting performance in several limited-data settings, especially on the Weather and TCTS datasets. However, the results on ILI are mixed, indicating that the advantage of CFF-Net is dataset-dependent. To illustrate the performance differences more clearly, the next section calculates the percentage improvement of CFF-Net over PatchTST using Eq (8) and provides a visual representation in Fig 3.

(8)

where E denotes the error value corresponding to a specific evaluation metric, including MSE, MAE, and RMSE. The subscripts PatchTST and CFF-Net indicate the error values obtained by the respective models.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Performance improvements of CFF-Net over PatchTST.

Percentage comparison of MSE, MAE, and RMSE across different training sample proportions (100%, 50%, 30%, and 20%). Positive values indicate improved performance of CFF-Net, while negative values correspond to cases where PatchTST performs better.

https://doi.org/10.1371/journal.pone.0353130.g003

These results show that CFF-Net achieves better performance than PatchTST in several tasks and data conditions, although the improvements are not consistent across all datasets, metrics, and sample proportions. The advantage is relatively noticeable on the Weather and TCTS datasets under the 30% and 20% training regimes. For example, on the Weather dataset, CFF-Net reduces the MSE by 12.50% and 11.88% under the 30% and 20% settings, respectively. On the TCTS dataset, CFF-Net reduces the MSE by 5.83% and 6.60% under the same two settings, respectively. These results suggest that the semantic prompts may provide useful global trend information when labelled samples are limited. In practical terms, the 11.88% MSE reduction on the Weather dataset at the 20% data level indicates that CFF-Net can improve forecasting accuracy under limited-data conditions, rather than substantially reducing the need for labelled data. On the agricultural TCTS dataset, the 6.60% MSE reduction at the same data level indicates a moderate improvement in predicting leaf chlorophyll content, which may support irrigation and fertilisation scheduling in precision agriculture. In contrast, on the ILI dataset, CFF-Net shows mixed results, which may be related to the highly irregular, epidemic-driven fluctuations in influenza statistics that are difficult to capture through pattern-based text descriptions. Additional visualizations further support this, showing that CFF-Net can capture periodic patterns and abnormal fluctuations in SPAD values, suggesting its potential to model dynamic features in agricultural time series.

Table 5 compares CFF-Net with several state-of-the-art time series forecasting methods, including PatchTST, DLinear [51], FEDformer [52], Autoformer [53], Informer [54], Pyraformer [55], and LogTrans [56], evaluated on four representative datasets: Weather, Exchange_rate, ILI, and TCTS. In terms of MSE and MAE, CFF-Net achieves the best or second-best performance across most tasks, with a particularly notable advantage in volatile datasets like ILI and Exchange_rate. For example, on the ILI dataset, CFF-Net reduces MSE by 53.65% compared to FEDformer and by 53.99% compared to Autoformer. On the Exchange_rate dataset, it reduces MSE by 8.42% compared to DLinear and by 31.50% compared to FEDformer. On the TCTS dataset, CFF-Net reduces MSE by 13.23% relative to FEDformer and by 2.96% relative to Autoformer, while requiring only a small number of trainable parameters.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Comparative performance of CFF-Net and state-of-the-art time series forecasting methods.

https://doi.org/10.1371/journal.pone.0353130.t005

It is important to highlight that CFF-Net updates only 0.11M trainable parameters during training, accounting for just 0.08% of its total parameters. Despite this small trainable-parameter ratio, CFF-Net achieves competitive performance compared with several fully trained models. This suggests that the semantic-guided fine-tuning strategy can help balance prediction accuracy and training efficiency. This approach provides a potential direction for developing efficient and lightweight time series prediction systems.

In conclusion, CFF-Net shows strong generalization across various task types, dataset sizes, and domain conditions. Notably, in noisy or structurally complex environments, the incorporation of semantic prompts significantly improves the ability to capture unusual fluctuations, making the model better suited to cross-domain patterns. These results emphasize the great potential of CFF-Net for real-world applications.

Ablation and hyperparameter configuration analysis

This section evaluates the effectiveness of the CFF-Net core modules and the robustness of the overall architecture. We examine how different configurations of MLP depth and hidden dimensions affect prediction performance, and investigate the impact of modifying or removing key modules through ablation studies. The results provide a clear understanding of how each design choice influences model accuracy across datasets.

The prediction results for MLPs with different depths and hidden dimensions across four datasets are summarized in Table 6. The results show that model performance is closely linked to both network depth and hidden size, although the effect is not strictly monotonic across all datasets and metrics. For shallow structures, such as those with 4 layers, the error values are generally higher on several datasets. Increasing the network depth to 8 and 12 layers leads to performance improvements in many cases, though the optimal hidden dimension varies across datasets. For instance, in the Weather dataset, the MSE decreases from 0.3367 at 4 layers with 256 hidden units to 0.3222 at 12 layers with 256 hidden units. In the Exchange rate dataset, the MAE decreases from 1.0766 at 4 layers with 256 hidden units to 0.7677 at 12 layers with 256 hidden units. When the network depth increases to 16 layers, the model achieves competitive overall performance. For example, with 16 layers and 256 hidden units, the Weather dataset obtains MSE and MAE values of 0.3202 and 0.3402, respectively, while the Exchange rate dataset achieves an MAE of 0.6744. The ILI dataset also shows improved performance under the 16-layer setting, with MSE decreasing to 1.2926 when 256 hidden units are used. However, increasing the hidden dimension does not always lead to further improvement. For example, in the TSTC dataset, the MAE increases to 1.3853 when the hidden dimension is increased to 2048 under the 16-layer setting. Overall, these results highlight that model depth has a more significant positive impact on performance than hidden size, and that a compact architecture with sufficient depth is more effective in capturing temporal dependencies while maintaining robustness across datasets.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Ablation study on network architecture parameters (layers and hidden dimensions).

https://doi.org/10.1371/journal.pone.0353130.t006

The results of the ablation studies, which evaluate the contribution of each component in CFF-Net, are shown in Table 7. In the first variant, w/o Text2Sem, the model does not convert numerical sequences into text or extract semantic features using GPT-2. Instead, an MLP is directly applied to the numerical inputs to generate the scaling and shifting factors. In the second variant, w/o Warm-up, the warm-up initialization strategy is removed, and the modulation factors directly influence PatchTST outputs without gradual adaptation. In the third variant, GPT-2 is replaced by BERT, and semantic features are extracted using BERT rather than GPT-2. These variants are compared to the baseline PatchTST and the full version of CFF-Net.

Download:

• PNG
  larger image
• TIFF
  original image

Table 7. Ablation study on model components.

https://doi.org/10.1371/journal.pone.0353130.t007

The results show that CFF-Net improves performance in most datasets and metrics, while the ablated versions experience notable performance drops. Removing Text2Sem results in the most significant degradation, with the MAE on the TCTS dataset increasing from 0.89 in the complete model to 2.21. Eliminating the warm-up strategy also weakens performance, raising the MAE from 0.89 to 1.15 on TCTS and from 0.67 to 0.92 on Exchange_rate. The relative superiority of GPT-2 over BERT for this task can be attributed to their architectural differences and pre-training objectives. BERT is an encoder-only model that produces bidirectional contextualised representations optimised for masked token prediction, whereas GPT-2 is a decoder-only model pre-trained with a causal language modelling objective that inherently models sequential dependencies from left to right. Since the textual prompts constructed by the SPM describe temporal trends in a forward-directed narrative, the autoregressive inductive bias of GPT-2 aligns more naturally with the directionality of the described patterns, leading to higher-quality semantic embeddings for subsequent modulation. Replacing GPT-2 with BERT generally leads to weaker performance than the full CFF-Net, suggesting that the choice of language model affects the quality of semantic representations, but the overall framework remains functional. Compared to the baseline PatchTST, the full CFF-Net achieves lower or equal errors across all reported datasets and metrics, with particularly clear improvements on the TCTS dataset, reducing MSE from 1.74 to 1.64 and MAE from 0.95 to 0.89. These findings confirm that semantic encoding, warm-up initialization, and the selection of a suitable language model are critical for maximizing the benefits of the proposed framework.

Nomenclature

Download:

• PNG
  larger image
• TIFF
  original image

https://doi.org/10.1371/journal.pone.0353130.t008