Land-based traffic flow prediction

Traffic flow prediction is a mature and extensively studied field, particularly in the context of land-based transportation systems such as urban road networks. Research in this area has progressed from traditional statistical models to machine learning and, more recently, deep learning approaches designed to handle complex spatio-temporal dependencies. State-of-the-art methods for ground traffic often employ Graph Neural Networks (GNNs) [34] to explicitly model the fixed topology of road networks, effectively capturing the intricate spatial relationships between interconnected roads and intersections. This is necessary because ground traffic is characterized by high stochasticity and complex, grid-like spatial dependencies influenced by numerous factors like traffic signals, accidents, and local events. In ground transportation safety research, two studies on U.S. North Carolina data focus on vulnerable road users (VRUs) [35,36]: one develops a priority-based framework using 2014–2019 data to identify high-risk locations, noting changing impacts of factors like traffic control (declining influence) and daylight (growing influence) on crash severity; the other uses nine-year data and seasonal random parameter models to find seasonal variations in pedestrian crash factors (e.g., hit-and-run in spring, alcohol impairment in summer) and higher crashes in darker seasons. Furthermore, recent studies have demonstrated the extensive potential of advanced deep learning and stochastic optimization frameworks in addressing complex challenges within intelligent transportation systems, ranging from infrastructure planning and anomaly detection to data synthesis under scarcity [37–41].

In contrast, airspace traffic is more highly regulated, follows more structured long-range trends, and its network topology is more fluid and less rigidly defined than a physical road graph [42,43]. These fundamental differences necessitate the development of specialized models tailored to the unique dynamics of the aviation domain, rather than simply adapting land-based models. This review will therefore focus on methods specifically applied to or suitable for air traffic forecasting.

Short-term air traffic flow prediction

Short-term forecasting is essential for tactical air traffic management, focusing on real-time adjustments and safety assurance. In the early stages, traditional statistical models were widely used for this purpose. For example, Nieto et al. [44] employed Autoregressive Integrated Moving Average (ARIMA) models to effectively capture short-term traffic fluctuations in low-complexity environments. Similarly, the dynamic updating capability of Kalman filtering [45] has been applied in air traffic control to adjust predictions in real time by incrementally updating flight data, providing timely and accurate forecasts in response to immediate changes [46].

More recently, deep learning methods have been applied to short-term tasks. Lin [47] proposed a model based on ConvLSTM modules to accurately predict flow distributions at varying altitudes, while Jardines [48] developed a Convolutional Neural Network (CNN)-based model to predict thunderstorms, enhancing tactical air traffic planning. To handle the complex network topology of the airspace system in real time, some studies have used Graph Convolutional Networks (GCNs) to effectively model the intricate relationships between nodes in large-scale airspace systems [49]. In parallel, other advanced Transformer-based architectures, such as the Temporal Fusion Transformer (TFT), have been successfully applied to predict numerical airport arrival delays at a high temporal resolution, demonstrating strong performance with multi-factor inputs [50]. Valuable methodological insights can also be drawn from the adjacent domain of urban rail transit (URT), where researchers have developed sophisticated models to handle sharp, non-routine traffic fluctuations during events, holidays, or public health emergencies. These works demonstrate the efficacy of advanced deep learning architectures for such scenarios. For instance, Transformer-based models have been designed to explicitly separate and predict the “extra” passenger flow generated by large-scale events. Furthermore, Graph Neural Networks (GNNs) have been extended to not only capture spatial dependencies but also to integrate multi-frequency temporal patterns (e.g., real-time, daily, and weekly) and even incorporate physics-guided loss functions to enhance model interpretability. Other approaches have utilized multi-task learning frameworks to jointly predict related variables like station inflow and outflow, capturing their complex interactions [51–54]. While effective for immediate operational needs, these methods are primarily designed for tactical decision-making and often struggle to capture the broader patterns required for long-range strategic planning.

Long-term air traffic flow prediction

Long-term forecasting, which covers time scales of several days, is critical for strategic planning, including optimizing airline schedules and airspace resource allocation. This task requires models that can effectively capture underlying trends and seasonality. Traditional methods like the Holt-Winters technique have been used to forecast aggregated passenger data by decomposing the time series into trend and seasonal components [55].

To handle the non-linearities inherent in long-term data, machine learning and deep learning models have become the primary focus. Ensemble learning methods such as Random Forests have been used to analyze features like flight origins and destinations to predict air traffic flows and flight levels [12]. Deep learning models, which excel at handling complex long-term dependencies, have also been a major area of research [56–58]. For instance, Gui et al. [59] applied Long Short-Term Memory (LSTM) networks to airway traffic prediction, demonstrating strong performance, particularly when abnormal factors are considered.

A more recent and transformative trend is the application of Large Language Models (LLMs) to spatio-temporal forecasting. This emerging field aims to create spatio-temporal foundational models with strong generalization capabilities. For example, UrbanGPT [60] has been proposed to integrate spatio-temporal dependency encoders with the instruction-tuning paradigm, enabling LLMs to make accurate predictions even in data-scarce or zero-shot scenarios. Alongside the development of new models, efforts are also underway to create comprehensive benchmarks, such as STBench [61], to systematically evaluate the spatio-temporal knowledge, reasoning, and application capabilities of existing LLMs.

To further improve long-term accuracy, researchers have increasingly incorporated external variables, such as weather conditions and holidays, into deep learning models [62]. However, while these advanced models have shown progress, they face significant challenges. As the number of input features grows, the computational complexity and resource requirements escalate rapidly. Furthermore, for high-dimensional inputs and multi-layer neural networks, the total number of parameters can expand swiftly, increasing model complexity without guaranteeing better performance. There is an urgent need for a model that can efficiently and effectively learn the fundamental nonlinear variations, trends, and seasonal patterns from the airspace traffic data itself, forming the primary motivation for this work.

Overall structure

This paper introduces a deep learning model, IDCformer, specifically designed for long-term airspace traffic prediction. As depicted in Fig 1, the model builds upon the transformer architecture, incorporating multiple feature extraction techniques. The extracted features are divided into patches enriched with positional information, enabling stronger feature representation and positional awareness capabilities. IDCformer is composed of several modules, including the Trend and Seasonality Extraction (TSE) module, the PatchTST module with enhanced feature representation and positional awareness, and the Local Attention Transformer (LAT) module.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The overall architecture of the proposed IDCformer model.

https://doi.org/10.1371/journal.pone.0338949.g001

IDCformer aims to enhance the modeling capability for complex air traffic flow data through multi-module feature extraction and patch-based processing. Specifically, the processing workflow for the input sequence is as follows: First, the input sequence undergoes parallel processing, where linear transformations are applied, and the Trend and Seasonality Extraction (TSE) module captures trend and seasonal features. Next, the processed feature representations are segmented into equal-length patches using patching techniques and mapped to a high-dimensional space. Subsequently, a Transformer encoder is employed to extract global dependencies. Building on this, a local attention mechanism is utilized to strengthen local feature representations. Finally, the predicted air traffic flow is obtained through a linear layer.

Feature extraction module

The feature extraction module learns the linear relationship of the data through two sub-modules, Linear and TSE, on the raw input series respectively, and captures the characteristics of the flow changes in terms of global trends and seasonality.

The Linear module extracts the implicit information directly from the sequence level through a two-layer linear mapping, and uses the neural net layer to process the linear relationships in the original sequence. For linear transformation, we define the input sequence is linearly transformed as:

(1)

where is the weight matrix of the linear mapping and is the bias vector.

In time series analysis, trend and seasonality features typically manifest as correlations between neighbouring time points. Convolutional operations effectively learn smooth trends and seasonal patterns in time series due to their local connectivity and parameter-sharing properties. Fig 2 compares Recurrent Neural Network (RNN)-based methods and Convolutional Neural Network (CNN)-based methods, with Fig 2(a) clearly demonstrating the superiority of convolution in capturing trends and seasonal features.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Comparison between CNN-based Methods and RNN-based Methods.

(a) CNN-based Methods: Node w acquires and fuses information from its neighboring node p within a short time window at the current and preceding time steps to efficiently capture trends and seasonality. (b) RNN-based Methods: At the current time step, node w receives raw information from its neighboring node p and integrates latent representations from the previous time step to capture temporal features.

https://doi.org/10.1371/journal.pone.0338949.g002

As shown in Fig 2(a), the CNN-based method processes information in parallel. A convolutional filter aggregates weighted information from multiple neighbouring points within a local window simultaneously. This parallel, windowed view allows the model to directly perceive the local shape and structure of the time series, making it highly effective at identifying local slopes (trends) and periodic patterns (seasonality). In contrast, Fig 2(b) depicts the RNN-based method, which processes information sequentially. At each time step, it considers only the current input and its own past hidden state. While excellent for capturing long-term temporal dependencies, this point-by-point processing is less direct for identifying local shapes, as an RNN must infer a pattern by remembering a sequence of individual points, whereas a CNN recognizes the pattern of that sequence in a single operation. Therefore, for the specific task of extracting trend and seasonality, the convolutional approach is inherently more direct and efficient.

For sequential data, one-dimensional convolution simplifies computation by avoiding the need to process additional dimensional information, unlike two-dimensional convolution. Furthermore, as the convolution kernel slides along the temporal axis, the extracted features at each time step vary according to the local context at that specific position, inherently making the convolution operation position aware. Consequently, our TSE module employs one-dimensional convolution to capture long-term trends and seasonal components in traffic data. This process is represented as:

(2)

Where, is the convolution kernel, is the padding parameter and is the implicit position information carried by the 1D convolution operation. Specifically, to effectively capture both long-term trends and shorter-term seasonality, the TSE module consists of two parallel 1D convolutional layers, organized into a two-branch structure. The first branch, designed for trend extraction, employs a 1D convolutional layer with a larger kernel size (48) to smooth the sequence and capture low-frequency signals. The second branch focuses on seasonality, using a 1D convolutional layer with a smaller kernel size (16) to identify more local, periodic patterns. Following each convolutional layer, a GELU (Gaussian Error Linear Unit) activation function is applied to introduce non-linearity. The outputs from these two branches are then fused through element-wise addition, allowing the model to learn a combined representation of both trend and seasonal dynamics.

The final linearly transformed output with the combination of trend and seasonal components is:

(3)

Through the collaboration of these two modules, the model extracts richer features of flow changes, thereby effectively enhancing its ability to learn from data.

Enhanced feature representation and position-aware PatchTST

PatchTST is a Transformer-based model for long-term time series forecasting, introduced in recent years. By segmenting the original sequence into patches and leveraging the Transformer’s global attention mechanism, PatchTST effectively models long-range dependencies across patches, enabling accurate long-term time series predictions.

In this paper, we enhance the long-term prediction capability of PatchTST by strengthening the feature representation of Patch and injecting position information into it. Specifically, the core idea of the original PatchTST is to divide the original time series into a number of non-overlapping patches , then perform linear mapping (i.e., Patch Embedding) on each , and finally embed all the patches into the input Transformer encoder for global modelling. However, directly dividing the raw sequence into a number of patches would result in redundant information or noisy features, making it difficult for the model to effectively discriminate the correct feature representation. The enhanced PatchTST in this paper is to divide the high-level feature representation in Eq. (3) into patches with implicit position information:

(4)

Before doing patch division, the TSE module in Eq. (2) captures a more advanced and smoother representation of the temporal features, which allows the divided to have clearer information in the time dimension and smoother articulation between patches. Where each patch can be represented as:

(5)

Each patch is then mapped to a higher dimensional space via an embedding matrix:

(6)

Where, denotes the embedding matrix and is the bias term, after the above processing, the embedded feature sequence can be obtained:

(7)

This process involves segmenting the sequences, organizing them into multiple patches, and embedding them into sequences that the Transformer can learn from and subsequently pass to the Transformer encoder. The patch size is a critical hyperparameter that determines the granularity of the input to the Transformer. A size that is too large can average out important short-term patterns, while a size that is too small increases computational cost and may lose local contextual information. Therefore, in this paper, we adopt a patch size of 16, which is commonly used in relevant studies.

After embedding the patch, the original PatchTST applies learnable positional encoding to capture the temporal positions within the time series. Meanwhile, the convolutional property of the TSE module, as described in Eq. (2), inherently provides relative positional information for neighbouring moments. This process is equivalent to embedding linearly refined local positional signals. The integration of these positional encodings further enhances PatchTST’s ability to sense position and temporal patterns in traffic data, as represented by:

(8)

where denotes the final position information and denotes the position information of the learnable position code. This learnable positional encoding is implemented as an embedding layer, where each position in the sequence of patches is assigned a unique vector that is optimized during training. This positional information is then combined with the patch embedding via element-wise addition. For this operation to be possible, the embedding size for both the patch and the positional encoding must be identical, which is set to 128 in our model. capacity:e of embedding size affects the model’s capacity; a size too small may lead to underfitting, while one too large increase the risk of overfitting and computational cost. Therefore, to avoid model overfitting and save computational costs, the embedding dimension is set to 128.

The global features are then extracted using a Transformer encoder, which models the long-range dependencies in traffic data through a multi-head self-attention mechanism. Each patch is processed through multiple layers of Transformer encoders. By calculating the correlations between sequence elements, the self-attention mechanism generates a weight matrix that is used to compute a weighted sum of the value vectors, producing a new representation while emphasizing distinct subspace features. Furthermore, the Transformer encoder incorporates a feed-forward neural network, enabling independent linear transformations of the representations at each position, thereby enhancing the model’s representational capacity.

Local self-attention

Although a global self-attention mechanism is applied to patches to capture long-term dependencies and semantic relationships, the local associations between patches are insufficiently addressed. This limitation arises because the Transformer encoder primarily applies attention between patches, hindering the model’s ability to detect local fluctuations effectively. To overcome this, we design a local attention mechanism that employs window segmentation to enable finer-grained interactions at a local scale, building upon features that already contain global contextual information.

While our LAT module utilizes a self-attention mechanism, it is fundamentally different from the global attention in the main Transformer encoder in its scope, architectural role, and the data it processes. Unlike the main encoder’s attention, which operates globally across all patches to learn long-range dependencies, the LAT module is a subsequent, refining step that operates locally within fixed, non-overlapping windows to capture short-term fluctuations. Critically, it takes the globally aware feature representations from the main encoder as its input, rather than the initial embeddings. This hierarchical approach, which first models the global trend and then focuses on local details, enables the model to achieve long-term stability without compromising local accuracy.

Following the modelling of long-range dependencies in PatchTST, short-term fluctuations in airspace traffic data are captured through the local self-attention mechanism. With a daily data step of 96 and a patch size of 16, the window size is set to 6, corresponding to the time step of the preceding day. Within each window, the multi-head self-attention process is applied. The specific self-attention computation is as follows:

(9)

Where , , are Query, Key and Value matrices respectively. They are obtained by linear transformation. are the feature representations of the input window, and , , are the weight matrices of Query, Key, and Value, respectively. The self-attention calculation formula is as follows:

(10)

where is the scaling factor, usually equal to the dimension of the key vector, used to stabilize the gradient. The attention weight matrix is generated by computing the dot product of the query and the keys, which is then normalized by SoftMax. Finally, this attention matrix is used to generate a new feature representation by adding a vector of weights.

Local self-attention is primarily used for short-term temporal dependencies, thereby enhancing the model’s capacity to capture local patterns. Following the application of local self-attention, the model integrates the outputs with the global features extracted by the Transformer encoder via residual connections.

(11)

Where is the global feature representation, the residual linkage effectively combines global and local information, enabling the model to have both an understanding of global dependencies and a keen capture of local variations.

Finally, the model performs an aggregation operation of the fused feature sequences using average pooling to obtain unified feature representation:

(12)

where is the i element in the fused feature sequence. Finally the aggregated feature representation is mapped to the final prediction through a linear layer:

(13)

Where is the weight matrix of the output layer and is the bias term.

Through the above process, the model in this paper is able to make full use of the self-contained features of the input sequences themselves to capture the complex temporal dependencies and achieve multi-step prediction of airspace traffic.

Dataset and experimental settings

In this section, we use real-world airspace traffic data to evaluate the performance of the proposed model. The dataset comprises real-time ADS-B data collected at 15-minute intervals from 1 June 2018–30 June 2018 within a 300 km radius centered on Frankfurt Airport. This includes flight data for all flights within this airspace. Specifically, the data encompass information such as flight ID, flight altitude, flight heading angle, and the latitude and longitude of each aircraft. ADS-B data acquisition and transmission can be affected by various factors, including signal occlusion and multipath effects, leading to data loss. Therefore, interpolation was employed to impute missing data during the preprocessing stage.

The dataset was divided into training, validation, and test sets in the ratio of 7:1.5:1.5. The TSE module consists of two parallel 1D convolutional layers, organized into a two-branch structure. There are three layers of Transformer encoders, and the number of attention heads matches the number of local attention heads in the final part of the model, which is 4. In the subsequent LAT module, local attention is applied with a window size of 6 using non-overlapping windows. The resulting local features are then integrated with the global features from the main encoder via a residual connection, as described in the “Local self-attention” section. The batch size was set to 64 to ensure robust model performance on the dataset.

To ensure robust convergence and mitigate overfitting, we implemented a rigorous training strategy. The model was trained using the Adam optimizer with an initial learning rate of 1e-3. We employed a Cosine Annealing learning rate scheduler, which gradually decreases the learning rate to a minimum of 1e-6 over the course of training, facilitating smoother convergence in later epochs. The batch size was set to 64. The maximum number of training epochs was set to 100. To further prevent overfitting, we adopted an early stopping strategy with a patience of 10 epochs; training was automatically terminated if the validation loss did not improve for 10 consecutive epochs. Fig 3 presents the training and validation loss curves. As shown, both curves descend rapidly from an initial value of approximately 0.14 and converge to a low error range after 80 epochs. The consistent alignment between training and validation loss demonstrates the stability of the training process and the effectiveness of the regularization strategy. All experiments were conducted on a desktop computer equipped with a 13th Gen Intel® Core™ i7-13700H processor, 16 GB of RAM, and an NVIDIA GeForce RTX 4060 Desktop GPU.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Training and validation loss curves over 100 epochs.

https://doi.org/10.1371/journal.pone.0338949.g003

Evaluation metrics

We divide the whole airspace traffic data into three common airspaces according to altitude: high altitude airspace (above 6000m), medium altitude airspace (3000m ~ 6000m) and low altitude airspace (below 3000m). Consequently, the objective was to predict flight traffic not only across the entire airspace but also within these three specific airspaces to provide controllers with more detailed decision-making guidance. Additionally, long-term traffic prediction in the airspace involves multiple time steps; therefore, Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE) were employed to evaluate the model’s performance.

(14)

(15)

(16)

Where: denotes the sequence length of the airspace traffic data, is the ADS-B data value, and is the airspace traffic prediction value. To interpret these metrics, it is crucial to understand that the prediction target, , is a single scalar value: the total number of aircraft in a defined airspace. Therefore, the metrics directly quantify the error in this aircraft count prediction.

The practical significance of this is best explained using MAE, which represents the average absolute difference between the predicted and actual aircraft counts. For the 24-hour prediction (Table 1 in the subsequent “Model Comparison” section), the baseline HA model has an MAE of 106.98, meaning its prediction is, on average, wrong by about 107 aircraft. In stark contrast, our IDCformer model achieves an MAE of 45.19, meaning its average error is only about 45 aircraft. This improvement from an average error of 107 aircraft to 45 is practically significant. It elevates the prediction from a rough estimate to a reliable decision-support tool, enabling air traffic managers to proactively optimize sector configurations and staffing with much greater confidence, thereby enhancing operational safety and efficiency.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Performance of each model in entire altitude airspace, high-altitude airspace, mid-altitude airspace, and low-altitude airspace. Where dark colored numbers are the best performance and underlined numbers are the second best.

https://doi.org/10.1371/journal.pone.0338949.t001

Baseline methods

To comprehensively assess the model’s performance, five state-of-the-art baseline methods were selected for comparison, encompassing both machine learning and deep learning approaches:

1. (1). HA: Historical Average. It uses the average value of historical data to predict future values.
2. (2). SVR [63]: a support vector regression with linear kernel.
3. (3). LSTM [64]: Long Short-Term Memory. A specialized form of recurrent neural network, LSTM addresses the issues of gradient vanishing and explosion in traditional RNNs by introducing a gating mechanism, thereby effectively capturing long-term dependencies in sequences.
4. (4). Autoformer [65]: Based on the transformer architecture, it incorporates a series decomposition module and an auto-correlation mechanism.
5. (5). PatchTST [31]: Patch Time Series Transformer. A Transformer-based model that segments time series data into several patches, each containing a sequence of consecutive time steps and treated as a token.
6. (6). iTransformer [66]: An inverted Transformer model that performs inverted modelling of time series within an encoder-only framework, deeply exploring the correlations between variables through an attention mechanism.
7. (7). ModernTCN [67]: Modern Temporal Convolutional Network. An advancement of Temporal Convolutional Networks (TCN), ModernTCN enhances the receptive field of the convolutional kernel and adopts a lightweight design while maintaining performance.
8. (8). AMD [68]: An MLP-based framework that decomposes time series into multiple scales and adaptively synthesizes predictions. Its core is an Adaptive Multi-predictor Synthesis (AMS) block configured with several predictors.

These baseline models have undergone extensive research and practical validation, occupying a core position in the field of traffic flow prediction and demonstrating stable performance and strong adaptability in scenarios similar to air traffic. Incorporating them into comparative experiments allows coverage of a wide range of classical algorithmic approaches and advanced technical solutions. To ensure a fair comparison of air traffic flow prediction performance, the input sequence length of all baseline models is kept consistent with that of IDCformer.

Model comparison

Table 1 presents the performance of the proposed method alongside other baseline methods across different test datasets, illustrating the variation in error for each model in predicting various time steps within different airspace traffic datasets. The analysis focuses on the following aspects:

Superiority of Deep Learning Models in long-term prediction: A comparative analysis of classical deep learning models, such as LSTM, and SVR on several datasets reveals that the prediction error of LSTM is only greater than that of SVR for relatively short time steps. As the prediction horizon increases, the error of LSTM becomes significantly smaller than that of SVR, while the errors of other state-of-the-art deep learning models are considerably lower than that of LSTM. This indicates that deep learning models are better suited for long-term predictions of flight traffic across different spatial domains. They effectively learn complex fluctuation patterns from large volumes of ADS-B data, primarily due to the deeper network structures and more sophisticated iterative mechanisms employed in deep learning models compared to SVR.

Accuracy of IDCformer in long-term forecasting: We compared the proposed model with state-of-the-art deep learning approaches, including ModernTCN, PatchTST, AMD and iTransformer. As shown in Table 1, the proposed model consistently outperforms the competing models across all time scales. Specifically, compared with the second-best model, PatchTST, IDCformer achieves overall reduction of 4.12% in RMSE, 4.20% in MAE, and 4.87% in MAPE. These results demonstrate that the proposed model is better suited for predicting long-term airspace flight traffic, owing to its superior ability to effectively capture the dynamics of airspace traffic conditions.

The stability and statistical significance of IDCformer: To rigorously evaluate our model’s ability to learn in a stable and reliable manner from the overall traffic flow, we conducted both 5-fold time-series cross-validation and hypothesis testing on the entire altitude dataset. The cross-validation results, presented in –, demonstrate that the proposed IDCformer not only consistently achieves the lowest average error across all metrics and prediction horizons but also exhibits small error bars. This indicates that its superior performance is stable and robust when evaluated on different partitions of the data. Furthermore, to formally verify that this outperformance is not due to random chance, hypothesis testing was conducted against the top three baseline models. As shown in Table 2, the p-values from the Diebold-Mariano test are consistently below the 0.05 significance level. This provides strong statistical evidence that the IDCformer’s performance represents a significant and reliable improvement over other advanced methods for long-term forecasting of the entire airspace’s traffic.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Results of the Wilcoxon signed-rank test comparing IDCformer against advanced baseline models on the entire altitude airspace dataset. The values represent the p-values for each comparison. Bolded values indicate a statistically significant difference (p < 0.05).

https://doi.org/10.1371/journal.pone.0338949.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Comparison of RMSE for Various Models Under 5-Fold Time-Series Cross-Validation (where bar represents average errors and error bars denote standard deviations).

https://doi.org/10.1371/journal.pone.0338949.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Comparison of MAE for Various Models Under 5-Fold Time-Series Cross-Validation (where bar represents average errors and error bars denote standard deviations).

https://doi.org/10.1371/journal.pone.0338949.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Comparison of MAPE for Various Models Under 5-Fold Time-Series Cross-Validation (where bar represents average errors and error bars denote standard deviations).

https://doi.org/10.1371/journal.pone.0338949.g006

To comprehensively understand the interpretability of the proposed model, Fig 7 exhibits the integrated attention weight heatmap across different time periods of the day. This heatmap is derived by averaging the weights from the enhanced feature representation and position-aware PatchTST and the Local Attention module, providing a holistic view of the model’s focus. The vertical and horizontal axes denote the indices of query patches and key patches, respectively, where each patch corresponds to a 4-hour interval within the 96-step daily sequence (e.g., Patch 0 represents 00:00–04:00). Color brightness corresponds to attention weight magnitudes, with brighter colors indicating a higher degree of dependency and focus.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Integrated attention weight heatmap illustrates the intrinsic dynamics across different time periods of the day.

https://doi.org/10.1371/journal.pone.0338949.g007

The visualization reveals that the model captures intrinsic dynamics by identifying critical periodic structures rather than distributing attention uniformly. Specifically, the model assigns the highest attention weights (reaching 0.24) to Patch 4 (16:00–20:00), identifying the evening peak as a deterministic anchor for the day’s traffic pattern. Furthermore, a significant structural “look-back” mechanism is observed: at the end of the sequence (Patch 5), the model heavily attends to Patch 2 (08:00–12:00) with a weight of 0.23. This demonstrates that IDCformer effectively utilizes the daily periodicity of airspace traffic, querying historical morning peak data to calibrate predictions for the night, thereby validating its ability to learn complex intrinsic dynamics beyond simple temporal proximity.

Visual analytics

The MAPE effectively reflects the degree of fit between predicted and actual traffic flow. Given that MAPE values are generally higher for models in mid-altitude and low-altitude airspaces, we focus on these two airspaces to visualise the actual traffic flow alongside predictions from state-of-the-art models.

Figs 8 and 9 offer a detailed visual comparison of the prediction performance of our proposed model and the PatchTST baseline. The top two rows of each fig illustrate the individual performance of our proposed model and PatchTST, respectively. For each model, two subplots are provided: a time-series plot on the left compares the predicted traffic (bar chart) against the actual values (red dots) to visualize the temporal fit, while a correlation plot on the right maps predicted values against actual values, where the proximity of the scatter points to the 45-degree line of perfect prediction indicates accuracy. The bottom panel then provides a direct time-series overlay, visualizing the predictions of both models as line graphs against the real traffic data, with an inset offering a magnified view of a specific interval to highlight performance on more volatile segments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Plot of actual vs predicted flight traffic in Low-altitude airspace.

https://doi.org/10.1371/journal.pone.0338949.g008

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Plot of actual vs predicted flight traffic in Mid-altitude airspace.

https://doi.org/10.1371/journal.pone.0338949.g009

Figs 8 and 9 illustrate that flight traffic in the Mid-altitude and Low-altitude airspace exhibits significant fluctuations. These variations are primarily attributable to the impact of low-altitude fog, which reduces visibility and directly increases the intervals between take-offs and landings, thereby causing fluctuations in flight traffic data. In the Mid-altitude airspace, the influence of clouds differs from that at lower altitudes but remains substantial. Mid-altitude clouds are closely associated with weather systems, often heralding precipitation or storm systems. Such weather events prompt aircraft to alter flight paths or adjust altitudes, thereby introducing variability in mid-altitude airspace traffic.

Our model consistently demonstrates superior fit to actual traffic flow compared to the second-best model across all time intervals, with a notable advantage during periods of heightened prediction difficulty. This underscores the effectiveness of employing one-dimensional convolution to separately model trends and seasonality in air traffic flow. Additionally, the use of linear layers proves sufficient for capturing linear patterns within the data. These findings further validate the robustness of the proposed model in handling the complexities of dynamic airspace environments.

Ablation studies

To evaluate the contribution of each component in the airspace traffic prediction task, a series of ablation experiments were conducted on five IDCformer variants using the traffic data of the entire airspace. These experiments systematically dissected the impact of key modules within the model. The variants are defined as follows:

1. IDCformer-RLAT: LAT module removed.
2. IDCformer-RL: Linear layer removed.
3. IDCformer-RC: A set of Conv1d in TSE removed.
4. IDCformer-RTSE: TSE module removed
5. IDCformer-RP: Enhanced PatchTST removed.

The input variables for each model are listed in Table 3. The prediction evaluations are illustrated in Fig 10, demonstrating that each component contributes to the model’s predictive performance. Fig 11 depicts the mean error increase for each ablated model compared to the original model, showing that the removal of any component results in decreased model performance. Based on this, the following conclusions can be drawn:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Input and output variables of the IDCformer and each variant.

https://doi.org/10.1371/journal.pone.0338949.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Experimental results after removing different modules.

https://doi.org/10.1371/journal.pone.0338949.g010

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. The mean error increase of ablated models compared to IDCformer.

https://doi.org/10.1371/journal.pone.0338949.g011

1. Effectiveness of the Patching Technique and the Enhanced Position-Aware PatchTST: The results highlight the effectiveness of both the general patching technique and our specific enhancements. A comparison between IDCformer-RP (which removes the PatchTST module entirely) and the full model reveals that IDCformer-RP exhibits the poorest performance. This demonstrates that the patching technique itself is fundamentally important, as it efficiently encodes the structural features of the input data and provides essential contextual information for the Transformer encoder.Furthermore, to specifically evaluate the contribution of our proposed enhancements, we tested a variant where the Enhanced PatchTST was replaced by the original PatchTST. This variant also underperformed the full IDCformer model, confirming the value of our modifications. By applying patching to high-level feature representations and integrating implicit convolutional positional signals, our enhanced module provides the model with a more robust understanding of temporal context. This is particularly crucial in long-term forecasting as it helps prevent the loss of positional information over long sequences, leading to a significant improvement in prediction accuracy.
2. Effectiveness of TSE and LAT Modules: Effectiveness of TSE and LAT: As shown in Fig 10, a comparison of the errors among IDCformer-RTSE, IDCformer-RC, IDCformer-RLAT, and the full IDCformer model demonstrates that the model’s error increases significantly, whether TSE is incrementally removed or LAT is directly removed.This phenomenon arises because the convolutional operations within TSE facilitate hierarchical feature abstraction, aiding the model in recognizing higher-level patterns and traffic flow trends. Moreover, when the TSE module is removed, the positional information inherently encoded in the patches due to convolutional characteristics is also lost. This leads to ambiguity in sequence positional information, particularly for long-term predictions. The LAT module, on the other hand, effectively captures local dependencies and mitigates the influence of long-range noise. Consequently, the removal of either TSE or LAT substantially impairs the predictive performance of the model.
3. Effectiveness of the Overall Architecture: The full IDCformer model outperforms all variants, underscoring the pivotal role of its constituent modules. The high accuracy of IDCformer in long-term prediction is attributable not only to the superior performance of individual components but also to the synergistic integration of these components within the model architecture.

Impact of different input_window

For deep neural networks, the input window (i.e., look-back window) is a key parameter that significantly affects the model’s ability to learn essential information from historical data. Selecting an appropriate input window can significantly improve the model’s prediction performance. Fig 12 demonstrates the error performance of various models on the entire airspace under different input window sizes. This analysis aims to evaluate how different input window sizes affect the model’s prediction performance while maintaining a consistent output window (i.e., prediction horizon).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Performance of three error metrics (Y-axis) for different input_window (X-axis) on different output_window for Entire airspace traffic.

https://doi.org/10.1371/journal.pone.0338949.g012

Analysis by Different Metrics: Each row in Fig 12 represents a distinct error metric, specifically RMSE, MAE, and MAPE. It is evident from the figure that, irrespective of the metric utilized, the error of our proposed model consistently remains lower than that of the other state-of-the-art models as both input and output windows increase.

Analysis by Different Output Windows: Each column in Fig 12 corresponds to a different output window size. The performance of each model improves with increasing input window size only when the output window is set to 192. For output window sizes of 96, 288, and 344, the performance of each model peaks at an input window size of 192, after which it deteriorates as the input window size increases further. Consequently, an input window size of 192 is deemed optimal compared to other input window sizes. Additionally, we observed that the error of each model decreases when the input window is increased from 96 to 192. However, the prediction error increases sharply when the input window is expanded from 192 to 288 and 384. When the input window was set to 96, the models lacked sufficient information to accurately predict the changing patterns of flight traffic in future airspace. Conversely, an input window size of 384 introduced excessive redundant information into the model, leading to inaccuracies in predicting future traffic flows and thus diminishing model performance.

Efficiency analysis

To evaluate the suitability of the proposed model for the airspace traffic forecasting task, we use the entire altitude dataset to analyze the computational cost of the proposed model. The experiments select training time, inference time, and GPU memory usage as the evaluation metrics, conducted in a unified environment with a fixed batch size of 64 to ensure comparability.

The results in Table 4 indicate that the MLP-based AMD is the most computationally efficient, followed by the iTransformer. Our proposed IDCformer, owing to its more complex, multi-stage architecture that includes dedicated modules for feature extraction (TSE) and local refinement (LAT), exhibits moderately higher computational costs. However, these costs remain well within a practical range for real-world deployment. For critical applications such as airspace traffic management, predictive accuracy is paramount to ensure operational safety and efficiency. As demonstrated in the preceding sections, IDCformer consistently achieves the highest forecasting accuracy. Therefore, the modest increase in computational resources is considered a justifiable and acceptable trade-off for the significant and reliable gains in predictive performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Computational costs of different models on entire altitude dataset.

https://doi.org/10.1371/journal.pone.0338949.t004

The external information integration ability of IDCformer

Although the IDCformer proposed in this study is primarily designed to address the internal dynamics of airspace traffic data, the dynamic variations in flight traffic during actual air traffic operations are often influenced by multiple external factors. Therefore, this section focuses on evaluating whether IDCformer possesses the capability to effectively learn and integrate external information. Let the original feature vector at time , containing internal traffic variables (e.g., number of flights), be denoted as . The selected external features (e.g., average flight altitude, longitude, and latitude) at the same time step are represented by another vector, . These two vectors are then concatenated to form a new, augmented feature vector, :

(17)

Where denotes the concatenation operation. The model then uses the historical sequence of these augmented feature vectors, , as its new input to predict the future traffic count. This formulation allows the model to learn from both the intrinsic traffic dynamics and the external factors provided. The subsequent experiments analyze the impact of including these external variables.

In this study, four external factors—aircraft heading, longitude, latitude, and flight altitude—were selected as features to analyze airspace traffic data. The prediction error of IDCformer was calculated, and Fig 13 illustrates the impact of these factors on long-term traffic prediction performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Effect of adding different features on the error of predicting Entire airspace traffic.

https://doi.org/10.1371/journal.pone.0338949.g013

Characteristic Description and Rationale for Selection: Aircraft Heading represents the azimuth of flight relative to true north. The heading reflects the flight path of an aircraft, aiding in capturing the distribution characteristics of aircraft on different routes. Longitude and Latitude indicate the geographic coordinates in the east-west and north-south directions, respectively. These coordinates assist the model in capturing spatial traffic change patterns. Flight Altitude indicates the vertical distance of an aircraft relative to sea level. Changes in flight altitude directly affect route selection and airspace utilization. Incorporating these features is essential for a comprehensive understanding of the spatial and temporal dynamics of air traffic.

Impact of Adding Features: Analyzing Fig 13, it was observed that when the prediction duration is 72 hours (i.e., output window = 288), adding two external factors—flight altitude and heading angle—results in higher prediction errors compared to the original IDCformer model. Apart from this exception, the inclusion of any individual feature improved the model’s long-term prediction accuracy. This outcome highlights the model’s inherent capability to adaptively learn from external features. This adaptive learning is a direct result of the model’s hierarchical attention architecture. The global self-attention mechanism in the main Transformer encoder first assesses the overall importance of each external feature across the entire prediction horizon. Subsequently, the Local Self-Attention (LAT) module refines this understanding by focusing on smaller time windows, allowing the model to capture how an external factor’s influence might vary over shorter periods. This synergy between global and local attention enables the model to dynamically amplify the weights of consistently relevant features while suppressing those that introduce noise, thereby effectively learning from a complex, multi-feature environment.

Table 5 presents the prediction performance of the model after introducing different features, including the prediction error after combining latitude and longitude. This allows for a clear quantification of the performance improvements resulting from capturing various types of information. The abbreviations in Table 5 correspond to the following feature combinations: ‘Only F’ represents the baseline model using only traffic flow; ‘&H’ denotes the addition of aircraft heading; ‘&LA’ adds latitude; ‘&LO’ adds longitude; ‘&FA’ adds flight altitude; and ‘& LA & LO’ represents the combined addition of latitude and longitude. When adding heading as a feature, the prediction accuracy improved by only 2.05% on average compared to the baseline model. This limited improvement is attributed to the extensive coverage of the study area, the large number of flights, and the significant variation in heading angles, which may render heading less effective as a feature input.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Performance of the model in predicting flight traffic over the entire airspace after adding different features. Where dark colored numbers are optimal and underlined numbers are sub-optimal.

https://doi.org/10.1371/journal.pone.0338949.t005

In contrast, adding latitude and longitude features individually enhanced the model’s prediction accuracy by 4.04% and 5.91%, respectively. This suggests that changes in latitude and longitude significantly impact overall traffic flow, and our model effectively captures the relationship between spatial location and traffic flow changes. However, although flight altitude is a crucial parameter in aviation operations, its performance enhancement was relatively limited. This may be due to the high number of flights introducing randomness or insufficient correlation with traffic patterns, making it challenging for the model to extract effective information from the altitude feature to improve prediction accuracy.

When both geographic features, latitude and longitude, were input into the model, the improvement in predictive accuracy was not as pronounced as when they were added separately. This suggests that the design of flight routes leads to high correlation between longitude and latitude, resulting in multicollinearity between these features. Feature redundancy can impede the model’s ability to extract effective information and may introduce noise, thereby affecting prediction performance. Therefore, the correlation and information gain between features must be carefully considered during the feature selection process.

To succinctly quantify the contribution of each external feature and investigate the reasons for feature conflicts, we conducted a Pearson correlation analysis. Fig 14 presents the correlation matrix between the target variable (Traffic Flow) and external factors. The analysis yields two critical insights that explain the experimental results in Table 5:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Pearson correlation heatmap of external features.

https://doi.org/10.1371/journal.pone.0338949.g014

Weak Correlation of Heading: The correlation coefficient between Heading and Traffic Flow is relatively low (r = 0.12). This confirms that while aircraft heading contains some valid information (contributing to the minor 2% performance gain), it is a weak predictor compared to spatial features, limiting its potential for significant improvements.

Feature Entanglement and Interference: A moderate correlation (r = 0.44) is observed between Longitude and Latitude. More importantly, we observe consistent cross-correlations between auxiliary features (Heading/Altitude) and spatial features (ranging from 0.30 to 0.39). This indicates a complex entanglement among external factors. When all features are integrated simultaneously, this coupling introduces inter-feature interference, which hinders the model from isolating unique information gains from each variable. This explains why the “combined” performance plateaued compared to individual feature integration.

In summary, our experimental results demonstrate that although IDCformer is specifically designed to capture the dynamic variations inherent in traffic data, it exhibits a robust capability to effectively learn and integrate external information. When external factors are incorporated as feature inputs, IDCformer achieves notable improvements in long-term prediction performance. Furthermore, the experiments reveal that individual independent factors are more conducive to enhancing the accuracy of long-term air traffic flow forecasting.