1.1. The missing data challenge in environmental monitoring

Environmental monitoring systems, despite their critical role in protecting public health and informing policy decisions, are plagued by a persistent challenge – missing data [1]. These gaps in knowledge aren’t merely technical inconveniences; they represent real risks to communities exposed to air pollution.

Missing data remains a persistent and significant challenge within air quality monitoring networks. Junninen et al. [2] reported that the share of incomplete rows in air quality datasets ranged from 7.2% in Helsinki to 14.5% in Belfast, with individual pollutants showing missing data levels varying from approximately 5% to over 10%. These patterns reflect the typical heterogeneity and challenges in European urban air monitoring systems. Wang et al. [3] highlight the issue of missing data in hourly time‑series records for six air‑pollution indicators (PM_2.5, PM₁₀, O₃, NO₂, SO₂, and CO) collected across the Qinghai‑Tibet Plateau from 2019 to 2022. Among eight state‑controlled monitoring stations, the average share of missing values was about 5%, and most sites met the national requirement of at least 89% valid daily means per year. By contrast, eight province‑controlled stations showed far greater gaps, with roughly 22% of observations missing on average, peaking at 43.3%. Analysis of five PM_2.5 monitoring stations in Pavlodar, Kazakhstan, revealed even more severe challenges, with completeness rates ranging from just 23.3% to 57.5% over a 13-month period. These aren’t isolated cases – they reflect a systematic problem affecting monitoring networks worldwide [4,5].

The prevalence of data gaps reflects a heterogeneous set of mechanisms, including sensor malfunctions, routine calibrations, power outages, severe meteorological events, and disruptions to Wi-Fi connectivity, Internet service, or data-collection servers. Monitoring infrastructure is, paradoxically, most prone to failure precisely when pollution levels are highest and reliable measurements are most needed. For example, a study in Tehran, Iran, found that nearly 48% of hourly PM_2.5 observations were missing across 30 monitoring stations between 2012 and 2018, with the largest gaps occurring in winter due to power outages and equipment downtime during severe smog episodes [6]. Similarly, in Temuco, Chile, Quinteros et al. reported that official monitoring data for PM_2.5 contained between 50% and 95% missing values from 2009 to 2016, largely due to repeated sensor failures and limited technical support [7]. These cases highlight a broader pattern: the more extreme the pollution event, the more likely it is that the monitoring system will break down, creating critical blind spots in both public health protection and subsequent data analysis.

These patterns of data loss highlight not only the technical fragility of monitoring networks but also the importance of how missing data is subsequently addressed. While imputation methods have become increasingly sophisticated, relatively little attention has been paid to how raw data is prepared prior to imputation, and how such choices affect downstream analyses. This underscores the need to better understand the full imputation pipeline, from data quality filtering to model selection and validation.

1.2. Why imputation quality matters: From public health to policy

The consequences of poor gap-filling extend far beyond academic concern. Consider the practical dilemma facing air quality agencies every day: when PM_2.5 data is missing during a pollution episode, should they issue health warnings based on incomplete information? The wrong decision either direction – false alarms or missed warnings – erodes public trust and potentially costs lives. Empirical evidence suggests that missing data tends to increase significantly during episodes of severe air pollution. Mu, Rubin, and Zou (2022) demonstrate that monitoring systems are more likely to fail precisely during these critical periods – when timely public advisories are most needed. This highlights a serious risk of informational blind spots and the potential absence of adequate health guidance at times of deteriorating air quality [8].

Regulatory frameworks compound these challenges. The European Union’s Air Quality Directive (2008/50/EC) [9] and the U.S. National Ambient Air Quality Standards (NAAQS; 40 CFR Part 50, Appendix I; 40 CFR Part 58) [10,11] require 75–90% data completeness for monitoring results to be considered valid in compliance determinations. Stations falling below these thresholds are effectively invisible to regulators, creating blind spots in pollution monitoring. The economic implications can be substantial: regulatory monitoring stations may cost around $50,000 upfront and require continuous maintenance [12], while comprehensive monitoring stations can require up to $200,000 for construction and $30,000 annually for maintenance [13]. High maintenance and calibration costs can be prohibitive for many regions or municipalities. These ongoing operational expenses mean that data loss due to incomplete measurements represents a direct waste of public investment, as agencies continue to bear the full costs of station operation while losing the regulatory value of collected data.

Beyond immediate operational needs, incomplete data undermines long-term planning and research. Epidemiological studies linking PM_2.5 exposure to health outcomes require robust data collection methods, as meta-analyses demonstrate substantial evidence of adverse health associations that depend on consistent monitoring approaches [14]. Accurate prediction of air quality responses to emission changes is essential for developing effective control policies, yet incomplete training datasets can compromise prediction accuracy and mislead environmental decision-making [15].

1.3. Current approaches and the overlooked role of data preprocessing

Traditional approaches to addressing missing data in air quality monitoring have encompassed a diverse array of methodological frameworks, ranging from elementary interpolation techniques to sophisticated machine learning architectures. Despite substantial algorithmic advances over several decades, operational monitoring systems continue to exhibit suboptimal performance characteristics, with error rates that compromise data utility for regulatory and research applications.

Linear interpolation, while computationally efficient and widely implemented, demonstrates significant limitations for extended data gaps. Research has demonstrated that simple interpolation methods exhibit significant performance degradation for extended data gaps, with accuracy decreasing substantially as gap length increases [2]. Contemporary machine learning approaches, particularly Random Forest and XGBoost algorithms, have exhibited superior performance in missing data imputation tasks, demonstrating significant accuracy improvements compared to conventional statistical interpolation methodologies. Empirical evidence from ecological monitoring datasets indicates that appropriate model selection enables high-fidelity gap filling, with accuracy rates reaching 96.5% for perfectly matched cases, substantially exceeding the 88.5–91.8% accuracy achieved by traditional single statistical imputation methods [16]. Furthermore, comparative analysis of imputation techniques applied to environmental air quality monitoring data from Kuwait revealed that the missForest algorithm consistently yielded minimal error rates across diverse pollutant parameters and varying proportions of missing observations [17]. Nevertheless, these advanced methodologies exhibit an inherent limitation: their efficacy is contingent upon the availability of comprehensive training datasets, thereby creating a fundamental paradox wherein optimal performance depends on the very data completeness that missing data scenarios inherently lack.

Contemporary developments in deep learning architectures, particularly Long Short-Term Memory (LSTM) networks and transformer models, have yielded further performance enhancements. For example, Pak et al. [18] combined convolutional neural networks with LSTM in a “PM predictor” that leverages spatiotemporal mutual-information features to forecast next-day PM_2.5 in Beijing. That hybrid CNN–LSTM model achieved an RMSE of 2.997 µg/m³, MAE of 2.20 µg/m³ and MAPE of 0.039 – substantially outperforming a standalone LSTM (RMSE 4.764 µg/m³; MAE 3.612 µg/m³) and an MLP baseline (RMSE 20.278 µg/m³; MAE 14.982 µg/m³). Although state-of-the-art deep-learning models offer superior performance for imputing air quality data, their adoption is constrained by limited computational budgets and staff expertise in many operational monitoring networks [19]. Consequently, routine practices still favour simpler techniques – such as linear interpolation – to address missing data [20].

This focus on algorithmic optimization may represent a fundamental misallocation of research priorities. While the scientific community has concentrated extensively on methodological innovation, critical preprocessing considerations have received insufficient attention. Practitioners routinely implement data preparation decisions – including quality thresholds for data exclusion, protocols for handling incomplete records, and criteria for data reliability assessment – yet these preprocessing strategies remain largely unexplored in the research literature.

1.4. Research Gap: Bridging the divide between data reality and imputation methods

1.5. Research aim and objectives

This study aims to validate and extend the application of DynamicSeq2SeqXGB imputation method to multi-station environmental monitoring networks with extreme data sparsity, demonstrating its robustness and practical applicability in real-world conditions.

Specific objectives:

1. To validate DynamicSeq2SeqXGB performance across five monitoring stations in Pavlodar, Kazakhstan with completeness rates ranging from 23.3% to 57.5%
2. To investigate the impact of data compression strategies (full vs. selective) on imputation quality across different data quality scenarios
3. To quantify real-world imputation accuracy through controlled degradation experiments on high-quality reference data (Beijing Guanyuan station), simulating severe missingness patterns typical of compromised monitoring networks
4. To benchmark DynamicSeq2SeqXGB against standard imputation methods (linear interpolation, rolling mean, forward/backward fill) under both synthetic and real-world missing data conditions
5. To establish practical performance benchmarks and confidence intervals for PM_2.5 imputation when dealing with extreme data sparsity and extended temporal gaps.

The remainder of this manuscript is structured as follows. Section 2 provides a comprehensive literature review on missing data imputation methods for environmental monitoring, with emphasis on machine learning and deep learning approaches under data sparsity. Section 3 describes the datasets, preprocessing procedures, and the DynamicSeq2SeqXGB methodological framework. Section 4 presents the experimental results for both the Pavlodar monitoring network and the external Beijing validation. Section 5 discusses the practical implications, limitations, and broader relevance of the findings. Section 6 concludes the study and outlines directions for future research.

2.1. Classical statistical methods

Despite advances in machine learning, classical statistical methods remain highly competitive for short-term gaps and resource-constrained applications. Research by Belachsen and Broday (2022) demonstrates that weighted k-nearest neighbors (wkNNr) achieves R² = 0.82 and NMAE = 0.21 for very short gaps (0.5–3 hours) in PM_2.5 data from 59 Israeli monitoring stations, outperforming more complex approaches (iiET, Iterative Imputation with an Ensemble of Extremely randomized Trees) for this specific use case [22].

Kalman filtering has proven particularly effective for real-time air quality applications. Lai et al. (2019) demonstrated that the algorithm reduced RMSE by an average of 68.3% across six pollutants (SO₂, NO₂, CO, O₃, PM_2.5, PM₁₀) compared to traditional methods such as simple moving averages and ARIMA, using data from the Guangzhou monitoring station. Additionally, it enhanced sensor measurement accuracy by 27%. For PM_2.5 specifically, Kalman filtering achieved an 18.56% RMSE reduction (0.1047 vs. 0.1285), confirming its robustness for fine particulate correction [25].

ARIMA and spectral methods show context-dependent effectiveness. Moshenberg et al. (2015) found that Discrete Cosine Transform (DCT) and K-SVD methods significantly outperformed linear interpolation for batch missing data patterns, while linear interpolation performed better for randomly scattered missing values [26]. This highlights the critical importance of understanding missing data patterns in method selection.

In a recent benchmark study on Seoul’s urban PM_2.5 datasets, Lee et al. (2025) report that for short-term gaps (6 hours), forward fill achieved optimal imputation accuracy (RMSE = 4.76, MAE = 3.20, r = 0.97). Conversely, SARIMAX outperformed other methods for longer (24-hour) gaps, yielding RMSE = 9.37 and MAE = 6.85 while maintaining high correlation (r = 0.97). These results underscore the importance of tailoring imputation strategy to gap duration [27].

2.2. Machine learning approaches

Random Forest-based methods have demonstrated consistent effectiveness across a range of environmental data imputation tasks, offering a robust baseline for comparison in both structured and sparse settings. The MissForest algorithm, evaluated by Alsaber et al. (2021) on Kuwait environmental data, consistently outperformed k-nearest neighbors, MICE, BPCA, and PMM across multiple pollutants with missing rates ranging from 18.2% (O3) to 57.4% (PM10) [17]. The iterative imputation scheme using Random Forest’s intrinsic multiple imputation capabilities proved particularly effective for mixed-type environmental datasets [21].

XGBoost demonstrates exceptional performance in comparative studies. Zamani Joharestani et al. showed XGBoost achieving R² = 0.81, MAE = 9.93 µg/m³, RMSE = 13.58 µg/m³ on Tehran PM_2.5 data, outperforming both Random Forest and deep learning approaches [28]. The hybrid RF-XGBoost model by Lin et al. [29] pushed performance further, achieving R² = 0.93, RMSE = 12.49 μg/m³ through a two-stage approach combining RF for aerosol optical depth (AOD) gap-filling with XGBoost for PM_2.5 estimation.

Gradient Boosting methods show remarkable consistency across diverse geographical contexts. Makhdoomi et al. [30] demonstrated Gradient Boosting Regressor achieving R² > 0.96 on Mashhad, Iran data, with statistical significance confirmed through Wilcoxon signed-rank tests. The ensemble approach combining multiple ML methods (Random Forest, XGBoost, Extra Trees) in the MISE framework showed superior performance across Indian cities’ AQI data [31].

Support Vector Regression proves effective for complex temporal patterns. The Iterated Imputation and Prediction (IIP) algorithm combining Correlation Dimension Estimation with SVR showed small average percentage prediction errors on three European ozone concentration time series, demonstrating particular strength in capturing nonlinear environmental dynamics [32].

2.3. Deep learning architectures

In a comparative evaluation on the Beijing Air Quality dataset, Hua et al. (2024) found that the Self-Attention-based Imputation for Time Series (SAITS) model consistently achieved lower MAE than RNN-based Bidirectional Recurrent Imputation for Time Series (BRITS) and other deep learning baselines across varying missing rates, confirming its effectiveness for time series imputation. The improvements were notable both in terms of accuracy and training efficiency compared to RNN-based methods [21].

Graph Neural Networks demonstrate particular strength for spatiotemporal environmental data. The Graph Recurrent Imputation Network (GRIN) achieves >20% MAE improvements over state-of-the-art methods by exploiting spatial correlations between monitoring stations through bidirectional message passing [33]. This approach proves especially valuable for sensor networks where geographical relationships contain crucial information for imputation.

LSTM-based imputation methods have demonstrated strong performance in recovering missing PM_2.5 data, particularly through architectural innovations. For example, the Transfer Learning-based LSTM Iterative Estimation (TLSTM-IE) model achieved 25–50% higher imputation accuracy compared to baseline methods on long-interval missing sequences in NYC PM_2.5 datasets by transferring knowledge from complete to incomplete sequences [34]. Separately, a standard LSTM-based imputation approach also showed substantial improvements over mean and moving average methods in PM_2.5 prediction tasks, reducing RMSE by up to 55–65% across multiple pollutants in Hefei, China [35]. These results confirm the robustness of LSTM architectures in handling both short and extended gaps in air quality time series.

The BRITS-ALSTM model by Wang et al. (2023) demonstrated R² = 0.7454 with 43.29% missing data rates at extreme locations on the Qinghai-Tibetan Plateau, representing a significant improvement over traditional methods in challenging environmental conditions [3]. Autoencoder-based approaches achieve up to 65% RMSE improvements over univariate methods for discontinuous and long-interval gaps [24,36].

Generative approaches offer unique advantages for uncertainty quantification. GAIN (Generative Adversarial Imputation Networks) and its variants provide not only point estimates but also uncertainty bounds, crucial for environmental decision-making [35,37]. The DEGAIN architecture, an enhanced version of Generative Adversarial Imputation Networks, introduces deconvolutional layers that improve reconstruction fidelity. Through adversarial training, DEGAIN achieves superior performance over baseline models, including autoencoders and MICE, particularly on multivariate air quality datasets. This reinforces the growing evidence that GAN-based approaches represent a leading strategy for restoring complex environmental time series [38].

2.4. Performance evaluation and method selection criteria

2.5. Specialized approaches for extreme data sparsity

Extremely sparse datasets demand fundamentally different strategies. Research by Chen et al. [42] on the FTLRI (“First Five & Last Three Logistic Regression Imputation”) method shows that specialized temporal correlation models consistently achieve lowest MAE (4.46–5.68 range) at 40% missing rates, outperforming general-purpose algorithms. The FTLRI temporal correlation model specifically designed for high-missing-rate datasets demonstrates the value of purpose-built approaches.

Spatiotemporal methods prove crucial for extreme sparsity. The spatiotemporal convolutional autoencoder by Wardana et al. [36] achieves up to 65% RMSE improvement over univariate methods at 40% missing rates by exploiting correlations between nearby monitoring stations. This approach handles both short-interval and long-interval consecutive missing patterns effectively.

Tensor-based imputation techniques, such as Tucker decomposition, have demonstrated resilience under extreme data sparsity conditions – up to 90% missing – by leveraging multi-modal correlations. Tan et al. [43] report that their method successfully reconstructs entire missing days of traffic data, outperforming matrix-based approaches. These methods prove particularly valuable when entire days of environmental monitoring data are completely missing due to systematic equipment failures.

The BRITS-ALSTM architecture exhibits notable robustness in deployment scenarios characterized by extreme data sparsity. Wang et al. demonstrated that this approach maintains reasonable imputation performance (R² = 0.7454, MRE = 0.2534) even under the challenging conditions described earlier for high-altitude monitoring networks [3]. These results highlight the model’s capacity to operate effectively where conventional techniques – such as mean filling, k-nearest neighbors, matrix factorization, and multiple imputation – experience significant degradation or operational failure. The bidirectional recurrent structure with attention mechanisms proves essential for extreme sparsity scenarios.

2.6. Implementation and deployment considerations

Real-time processing requirements favor specific architectural choices. Kalman filtering provides optimal balance for streaming applications, while Transformer models like SAITS offer batch processing advantages [25]. Edge computing implementations demonstrated by multiple studies support up to 40% missing data rates with acceptable latency for operational monitoring networks [44].

Ensemble-based machine learning approaches, particularly combinations of support vector regression (SVR) and Random Forest, have demonstrated operational robustness in scenarios with substantial data sparsity. Faramarzzadeh et al. [41] report that such multi-model frameworks outperform traditional gap-filling techniques – including linear interpolation – when applied to daily precipitation datasets with missing rates ranging from 50% to 70%. However, these methods necessitate careful hyperparameter tuning and substantial computational resources to ensure stability and generalization across heterogeneous environmental conditions.

Model interpretability varies significantly across method categories. Classical statistical methods provide clear causal relationships, while deep learning approaches often function as black boxes. The N-BEATS inspired architecture addresses this limitation by explicitly decomposing time series into bias, slope, seasonality, and residual components with interpretable equations [45].

As model complexity increases, so does the demand for computational resources. Kim et al. [45] report that training deep neural networks for time-series imputation required a workstation equipped with 128 GB RAM and a single NVIDIA Titan X GPU (12 GB VRAM), with training times ranging from several hours depending on dataset size and missing rate. These resource constraints underscore the trade-off between model sophistication and inference efficiency – where simple architectures allow near real-time imputation, and deeper models necessitate batch processing strategies.

2.7. Summary

This comprehensive literature review reveals a mature field with clear performance hierarchies and practical guidance for method selection. Transformer-based approaches (particularly SAITS) represent the current state-of-the-art for most environmental monitoring applications, providing 12–38% improvements over previous methods while maintaining computational feasibility [21]. Classical statistical methods retain important niches for short-term gaps and resource-constrained deployments, while specialized high-sparsity methods become essential for extremely sparse datasets [2,46].

The field is moving toward ensemble approaches that combine multiple method strengths rather than seeking single optimal solutions. Future developments should focus on physics-informed methods that incorporate atmospheric dynamics, federated learning for privacy-preserving multi-network collaboration, and continual learning approaches that adapt to changing environmental conditions [37]. The convergence of high-performance computing and environmental monitoring creates unprecedented opportunities for accurate, real-time air quality data reconstruction across diverse deployment scenarios.

While significant progress has been achieved across all method categories, critical gaps remain for operational deployment in severely compromised monitoring networks. As identified in Section 1.4, existing approaches inadequately address extreme sparsity scenarios (<30% completeness) common in developing regions and harsh environments. This literature analysis confirms the need for specialized frameworks that can operate effectively under such challenging conditions.

3.1. Data

3.1.1. Data Source and Station Network.

This study utilized data from five air quality monitoring stations operated in Pavlodar, Kazakhstan, representing a comprehensive urban monitoring network covering different environmental and industrial zones. The monitoring stations were strategically positioned to capture spatial variability in air pollution across the city (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Pavlodar Air Quality Monitoring Network.

https://doi.org/10.1371/journal.pone.0338788.t001

A brief environmental characterization of the monitoring sites is provided to clarify their representativeness. The app_pspu station is located in the central urban area near educational and residential buildings with additional influence from a nearby railway corridor. The app_2pavlodar station represents a residential district situated closer to major industrial facilities, particularly the aluminum and energy complex. The app_center site is an urban-background location in the city center, characterized by high pedestrian activity, green corridors, and a dominant transport contribution. The app_metallurg station is positioned in a southern residential district located in closest proximity to metallurgical and energy facilities, making it the most industry-influenced site in the network. The app_zaton station represents a riverside residential microdistrict near the Irtysh floodplain with strong ventilation from the water–green corridor and industry located at moderate distances.

3.2. Data preprocessing

The data preprocessing pipeline consists of several sequential stages designed to transform raw sensor measurements into analysis-ready datasets (Fig 1). The workflow incorporates quality control measures at multiple levels, from minute-level duplicate removal to hourly-level outlier detection, ensuring data integrity while preserving temporal patterns essential for imputation model training.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Data preprocessing workflow for PM_2.5 monitoring network.

https://doi.org/10.1371/journal.pone.0338788.g001

3.2.1. Outliers detection.

A comprehensive two-stage outlier detection system was implemented to identify and handle anomalous values in PM_2.5 measurements while preserving legitimate pollution episodes.

Stage 1: Minute-level preprocessing. The first stage addressed fundamental data quality issues through systematic identification of anomalous values. Duplicate measurements sharing identical timestamps and station identifiers were removed to ensure data integrity. Physically impossible values were identified using absolute bounds criteria, where PM_2.5 concentrations below 0 μg/m³ or exceeding 1000 μg/m³ were flagged for removal. These extreme values typically indicate sensor malfunctions or calibration errors.

Stage 2: Hourly-level filtering. Secondary filtering employed a hierarchical series of three complementary filters designed to address different types of measurement anomalies

Filter 1: Station-specific upper limits

Station-specific thresholds were calculated using the formula (1):

(1)

where Q_0.995 represents the 99.5th percentile calculated after log-transforming x+1 to dampen heavy tails. This approach accounts for the highly skewed distribution of PM_2.5 concentrations while establishing station-specific upper bounds that reflect local pollution characteristics. Value of 900 μg m⁻³ serves as sensor-saturation filter. This threshold ensures removal of measurements beyond reliable instrument range.

Filter 2: Single-step spikes

Isolated measurement spikes were identified and removed using dual criteria (2):

(2)

This filter captures isolated jumps caused by dust bursts or electronic noise that do not persist in subsequent measurements, effectively distinguishing between measurement artifacts and genuine pollution episodes.

The hierarchical filtering approach systematically addressed different sources of measurement error while preserving legitimate atmospheric pollution events. Detected outliers were replaced with NaN values to maintain temporal structure while enabling subsequent imputation procedures (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Outlier detection and removal results by monitoring station.

https://doi.org/10.1371/journal.pone.0338788.t002

3.2.2. Data Quality Characteristics.

Following outlier detection and removal, the monitoring network exhibited substantial heterogeneity in data completeness, reflecting real-world challenges in environmental monitoring systems. Data completeness varied significantly across stations (Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Data quality characteristics and PM_2.5 concentration statistics for the five-station monitoring network in Pavlodar, Kazakhstan^1.

https://doi.org/10.1371/journal.pone.0338788.t003

The monitoring network demonstrated considerable variability in data quality, with mean data completeness of 46.3% across all stations. The complete dataset comprised approximately 50,667 hourly observations over the 15-month period (May 23, 2024 to July 19, 2025), with 23,471 valid measurements and 27,196 missing observations. The network exhibited significant spatial heterogeneity in PM_2.5 concentrations, with station-specific mean values ranging from 17.5 µg/m³ (app_zaton) to 84.8 µg/m³ (app_metallurg).

Data completeness patterns revealed two distinct station categories. Higher-performing stations (app_center, app_2pavlodar, and app_pspu) achieved 56–57% completeness. Lower-performing stations (app_metallurg and app_zaton) exhibited 23–37% completeness throughout the entire monitoring period.

The observed data completeness patterns reflect typical challenges in environmental monitoring networks, including equipment malfunctions, calibration periods, power outages, and communication failures [1,4]. Notably, station app_metallurg exhibited extremely low completeness (23.3%) representing the type of severely compromised monitoring infrastructure commonly encountered in operational networks. Despite these challenges, the dataset provides a realistic testbed for evaluating imputation methods under varying data quality scenarios, from moderately incomplete to severely sparse monitoring conditions.

3.2.3. Data compression strategies.

Due to the extreme sparsity of the monitoring data, model training was conducted on time series after removing missing values (NaN), effectively compressing the temporal sequences into dense arrays of valid observations suitable for machine learning algorithms. Synthetic gaps for training, testing, and benchmarking imputation methods were subsequently created within these dense datasets to simulate realistic missing data scenarios while maintaining ground truth values for performance evaluation.

Data compression in the research represents the process of removing rows with NaN from PM_2.5 time series to create dense training datasets. This technical operation transitions sparse time series with gaps to continuous arrays of valid observations. For example, if an original time series contains 10,000 hourly measurements with 4,000 missing values, compression yields an array of 6,000 consecutive valid observations without temporal structure.

Two distinct compression approaches were evaluated to assess the impact of data preprocessing strategies on imputation performance.

Selective compression uses a quality-first approach with strict filtering criteria applied before NaN removal. The method evaluates monthly data completeness against a 70% threshold and requires 24 consecutive valid hours. Only months meeting both criteria are retained, ensuring data quality while reducing dataset size. Stations must have at least three months above the 70% threshold to be included in the analysis.

Full compression adopts a quantity-first approach, directly removing all NaN values without preliminary quality filtering. This method preserves all available valid measurements regardless of their temporal context or surrounding data quality, maximizing the utilization of sparse monitoring data (Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Impact of compression strategies on data completeness.

https://doi.org/10.1371/journal.pone.0338788.t004

The results demonstrate fundamental differences between approaches. Selective compression achieved high completeness rates (>90%) in retained datasets but resulted in substantial data loss, with stations retaining only 21–45% of original observations. Most critically, app_metallurg was completely excluded due to insufficient monthly completeness. Full compression maintained perfect completeness (100%) while preserving all originally valid measurements, demonstrating universal applicability across all monitoring stations regardless of data quality characteristics.

The results demonstrate fundamental differences between approaches. Selective compression achieved completeness rates of 82.5–87.8% in retained datasets but led to significant monthly data exclusion. Stations with poor initial data quality experienced the greatest losses: app_zaton retained only 4 out of 15 months, while app_metallurg was completely excluded from selective compression due to failing quality criteria. Even stations with moderate completeness lost substantial temporal coverage, with app_2pavlodar retaining 6 months and app_pspu retaining 7 months. Only app_center, with the highest initial completeness, preserved 8 months. In contrast, full compression maintained perfect completeness (100%) while preserving all originally valid measurements, demonstrating universal applicability across all monitoring stations regardless of data quality characteristics.

Thus, selective compression aims to ensure high-quality, temporally coherent training data by retaining only months that meet predefined completeness and continuity requirements. In contrast, full compression aims to maximize data availability by preserving all valid observations regardless of temporal structure.

For clarity, the decision rules for both compression strategies are summarized below.

3.2.5. Feature Scaling for ML Models.

Feature scaling was implemented using the StandardScaler algorithm from scikit-learn to ensure optimal performance of machine learning algorithms. The standardization process applied the z-score transformation formula (3):

(3)

where μ represents the mean and σ the standard deviation of each feature. This transformation was applied to all numerical variables including PM_2.5 concentrations, meteorological parameters, and temporal features.

Standardization parameters (means and standard deviations) were estimated from the complete set of available non-missing observations in the dense time series prior to synthetic gap generation. Additional sensitivity tests with normalization fitted solely on the training subset yielded very similar performance, so the simpler global standardization scheme was retained without affecting the substantive conclusions.

The standardization procedure was essential for addressing the heterogeneous scales present in environmental monitoring data, where PM_2.5 concentrations (measured in μg/m³) exhibit substantially different ranges compared to meteorological variables such as relative humidity (percentage) and wind speed (m/s). The implementation ensured that no single variable would dominate the learning process due to scale differences, thereby improving algorithm convergence and prediction stability.

3.3. Benchmarking Methodology and Synthetic Gap Generation

The benchmarking methodology follows a systematic workflow to ensure fair comparison between developed Dynamic Sequence-to-Sequence model implemented using eXtreme Gradient Boosting (DynamicSeq2SeqXGB) and classical imputation methods (Fig 2). The process encompasses two parallel compression strategies, unified model training, synchronized testing on identical synthetic gaps, and comprehensive statistical evaluation across multiple experimental runs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Comprehensive benchmarking methodology workflow for imputation method comparison.

https://doi.org/10.1371/journal.pone.0338788.g002

3.4. Classical imputation methods

To establish performance benchmarks and validate the effectiveness of DynamicSeq2SeqXGB, four widely-used classical imputation methods commonly applied in environmental monitoring systems were implemented. These methods represent the current standard practice in operational air quality networks due to their computational simplicity and ease of implementation.

3.4.1. Linear Interpolation.

Linear interpolation estimates missing values by creating straight-line connections between adjacent known data points. The interpolated value is calculated as:

(4)

where t₁ and t₂ are the nearest available timestamps before and after the gap. This method assumes linear temporal trends and performs well for short gaps but degrades significantly with increasing gap length, as it cannot capture non-linear patterns or diurnal variations characteristic of atmospheric pollutant concentrations [50].

3.4.2. Rolling Mean.

The rolling mean method fills missing values using the average of a sliding window of available observations. A 24-hour window (12 hours before and after each missing point) was implemented to capture short-term temporal patterns while maintaining sensitivity to recent changes:

(5)

where n represents the number of available observations within the 24-hour window, and tᵢ are the timestamps of valid measurements [51]. While the 24-hour window effectively captures temporal patterns and provides stable estimates, this averaging approach may underestimate extreme pollution events.

3.4.3. Forward Fill.

Forward fill (also known as “last observation carried forward” or LOCF) propagates the most recent valid observation to fill subsequent missing values:

(6)

where t_last is the timestamp of the last available measurement before the gap. This method assumes temporal persistence of atmospheric conditions and performs reasonably well for short gaps but becomes increasingly unrealistic for extended periods, as it cannot account for natural temporal variations in pollutant concentrations [52]. LOCF is widely implemented in time-series databases and environmental monitoring systems as a computationally efficient gap-filling technique, particularly useful when data shows relatively stable trends.

3.4.4. Backward Fill.

Backward fill propagates the next available observation backward to fill preceding missing values:

(7)

where t_next is the timestamp of the first available measurement after the gap. This method assumes that future conditions can inform past missing values, which may be appropriate when data processing occurs retrospectively. However, it violates causality principles and may introduce artificial patterns when applied to real-time monitoring scenarios [52]. Studies have shown that backward fill can effectively handle downward trends in environmental time series, though it may introduce artificial jumps or dips in the data.

3.5 DynamicSeq2SeqXGB Model Architecture

The complete processing workflow of DynamicSeq2SeqXGB is illustrated in Fig 3, which demonstrates the sequence-to-sequence transformation from sparse environmental time series to gap-filled PM_2.5 data through adaptive context processing and multi-output prediction.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. DynamicSeq2SeqXGB sequence processing workflow.

https://doi.org/10.1371/journal.pone.0338788.g003

4.1. Data Quality and Completeness Assessment of the Pavlodar Monitoring Network

Comprehensive assessment of data quality across the five-station monitoring network revealed substantial heterogeneity in measurement completeness and gap patterns throughout the 15-month study period. The analysis identified significant variations in data availability both temporally and spatially, with important implications for imputation strategy selection. Fig 4 presents comprehensive gap pattern analysis across both temporal and statistical dimensions, providing critical insights into the operational challenges faced by each monitoring station.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Temporal patterns of data completeness and gap distribution across monitoring stations.

Panel A: Monthly data completeness rates for each monitoring station with 70% regulatory threshold indicated by horizontal dashed line. Panel B: Empirical cumulative distribution functions (ECDF) of gap lengths with 95% confidence intervals derived from bootstrap resampling of gap episodes. Critical thresholds marked at 5, 12, 24, 48, and 72 hours correspond to operational breakpoints for imputation strategy selection. Each station exhibits distinct gap duration profiles, with steeper curves indicating predominance of shorter interruptions and gradual slopes reflecting higher proportions of extended outages.

https://doi.org/10.1371/journal.pone.0338788.g004

Data completeness patterns varied considerably across monitoring stations, ranging from 23.3% (app_metallurg) to 57.5% (app_2pavlodar), with a network-wide mean of 46.3%. The temporal distribution of missing data exhibited distinct seasonal patterns, with winter months (December-February) showing decreased completeness rates across most stations, likely attributable to equipment malfunctions during severe weather conditions and increased maintenance requirements during high-pollution episodes.

Gap duration analysis revealed highly skewed distributions with substantial heterogeneity across monitoring stations. Mean gap lengths varied from 3.5 hours (app_pspu) to 32.1 hours (app_metallurg), with considerable variation in gap frequency from 127 gaps (app_center) to 370 gaps (app_pspu). Maximum gap durations demonstrated extreme variability, ranging from 307 hours at app_metallurg to 3,268 hours (136 days) at app_zaton. The gap pattern analysis indicates that while most stations experienced frequent short interruptions, extended outages substantially impact overall data availability and present significant challenges for conventional interpolation methods.

Station-specific characteristics demonstrated clear clustering into distinct operational categories. Higher-performing stations (app_center, app_2pavlodar, app_pspu) achieved 56–58% completeness with consistent monitoring patterns, while lower-performing stations (app_metallurg, app_zaton) exhibited 23–37% completeness with more erratic data collection. Station app_metallurg showed the most challenging operational profile with only 23.3% completeness despite having fewer total gaps (242), indicating prolonged equipment failures.

Two data compression strategies produced markedly different outcomes. Selective compression achieved 82.5–87.8% completeness in retained datasets while dramatically reducing data volume, with compression ratios ranging from 0.239 (app_zaton) to 0.524 (app_center). Most critically, app_metallurg was completely excluded due to insufficient monthly data quality. After selective compression, remaining gaps were substantially shorter (mean: 3.5–12.7 hours vs. original 11.7–35 hours), facilitating more effective imputation. Full compression maintained 100% data retention with universal applicability across all monitoring stations.

Pollution concentration patterns varied considerably across stations, with mean PM_2.5 concentrations ranging from 17.5 μg/m³ (app_zaton) to 84.8 μg/m³ (app_metallurg). After selective compression, retained high-quality data showed modified concentration profiles, suggesting potential systematic relationships between data availability and pollution episodes that warrant consideration in imputation strategy selection (Table 5).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Data quality characteristics and compression strategy outcomes for the Pavlodar monitoring network.

https://doi.org/10.1371/journal.pone.0338788.t005

A brief comparative analysis of station-specific outcomes shows that the stations with more stable temporal structure and moderate variability in PM2.5 (app_center, app_pspu) benefited most from selective compression, which provides longer uninterrupted segments for model training. In contrast, stations with irregular emission patterns or lower initial completeness (app_zaton, app_2pavlodar) showed larger improvements under full compression, likely because this strategy preserves all available valid observations, maximizing the usable signal in environments with higher variability or operational interruptions. The app_metallurg station, characterized by extremely sparse measurements, benefited exclusively from full compression, as selective filtering removed the station entirely from training.

4.2. Benchmarking performance under full compression strategy

Comprehensive evaluation of DynamicSeq2SeqXGB against standard imputation methods under full compression conditions demonstrated near-universal superiority across the monitoring network. In all experiments, DynamicSeq2SeqXGB was trained on multivariate input that combined PM_2.5 with meteorological covariates (temperature, humidity, wind speed and direction), which enhances the environmental interpretability of the imputed series. The benchmarking analysis, conducted across 25 gap length-station combinations, revealed consistent outperformance of the proposed machine learning approach, with 96% win rate (24/25 cases) and improvements averaging 43.4% over the best baseline methods (Table 6). Fig 5 shows percentage improvement in MAE for each station and gap length combination.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. DynamicSeq2SeqXGB performance comparison with baseline methods under full compression.

https://doi.org/10.1371/journal.pone.0338788.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. DynamicSeq2SeqXGB improvement over best baseline methods under full compression strategy.

Green bars indicate DynamicSeq2SeqXGB superiority; red bar indicates baseline method superiority. Horizontal dashed line at 0% represents performance parity.

https://doi.org/10.1371/journal.pone.0338788.g005

Performance overview established DynamicSeq2SeqXGB as the dominant imputation approach, achieving superiority in all cases except one instance at app_pspu for 48-hour gaps, where linear interpolation proved marginally superior (–28.3%). The consistent advantage across diverse operational conditions validates the effectiveness of the sequence-to-sequence architecture with adaptive context processing for environmental time series reconstruction under real-world monitoring constraints.

Station-specific analysis revealed distinct performance profiles while maintaining universal superiority. app_2pavlodar demonstrated the highest average improvements (52.0%), with peak performance reaching 64.6% for 24-hour gaps compared to rolling mean methods. app_zaton achieved substantial gains (50.2% average), with notable 60.0% improvement for 24-hour gaps over linear interpolation. app_metallurg showed the most variable performance (47.9% average), including the maximum single improvement of 80.9% for 72-hour gaps, while app_center (37.9% average) and app_pspu (29.0% average) demonstrated more moderate but consistent advantages.

Gap length dependencies exhibited clear temporal patterns favoring extended reconstructions. Extended gaps (72 hours) showed the highest average improvements (52.5%), with DynamicSeq2SeqXGB excelling in complex multi-day pattern reconstruction. Medium-term gaps (24 hours) achieved strong performance (50.7% average), effectively capturing diurnal cycle dynamics. 12-hour gaps demonstrated substantial improvements (47.9% average), while 48-hour gaps (33.2% average) and 5-hour gaps (32.7% average) showed more modest but still significant advantages.

Baseline method comparison revealed linear interpolation as the most competitive conventional approach, serving as the best baseline in the majority of cases. This dominance reflects linear interpolation’s effectiveness for capturing temporal trends across diverse gap lengths. Rolling mean methods proved competitive for medium to extended gaps at specific stations, while forward and backward fill methods achieved best performance primarily for short-term interruptions or specific station characteristics.

Performance magnitude analysis demonstrated substantial and practically significant advantages across the network. Improvements exceeded 50% in 40% of cases (10/25), indicating transformative reconstruction quality under optimal conditions. Five cases achieved improvements >60%, with app_metallurg’s 72-hour reconstruction representing the peak performance at 80.9% improvement over linear interpolation. The single case of underperformance (−28.3% at app_pspu, 48h) involved comparison with linear interpolation, suggesting that specific temporal patterns may occasionally favor simple trend-based approaches.

Methodological insights highlight the value of adaptive sequence-to-sequence processing for environmental time series reconstruction. The algorithm’s superior performance on extended gaps demonstrates its ability to maintain temporal coherence across multiple diurnal cycles, where conventional methods struggle with pattern degradation. The predominance of linear interpolation as the primary competitor underscores the challenge of beating well-established temporal trend estimation, making DynamicSeq2SeqXGB’s consistent superiority particularly noteworthy.

These results establish DynamicSeq2SeqXGB as a robust and superior alternative to conventional imputation methods for operational air quality monitoring networks. The near-universal improvements, particularly for challenging extended gaps, demonstrate that advanced machine learning approaches can deliver substantial and consistent value over traditional statistical methods when properly designed for environmental time series characteristics.

4.3. Benchmarking Performance under Selective Compression Strategy

Evaluation of DynamicSeq2SeqXGB under selective compression conditions revealed consistent superiority over classical imputation baselines, with the method achieving a 100% win rate (20/20 cases) and average improvements of 48.8% compared to the best baseline methods (Table 7). The analysis encompassed all stations except app_metallurg, which was excluded due to insufficient data quality for selective compression (compression ratio = 0.051).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. DynamicSeq2SeqXGB performance comparison with baseline methods under selective compression.

https://doi.org/10.1371/journal.pone.0338788.t007

Performance overview established DynamicSeq2SeqXGB’s universal dominance under selective compression, achieving superiority in all evaluated cases without exception. This perfect win rate contrasts with the 96% success rate observed under full compression, suggesting that selective preprocessing may enhance the method’s robustness across diverse operational conditions. The consistent advantage validates the effectiveness of quality-first data preparation strategies for environmental time series reconstruction.

Station-specific analysis revealed distinct performance profiles with all stations benefiting from selective compression. app_pspu demonstrated the highest average improvements (54.6%), with exceptional performance for 24-hour gaps (72.6% improvement over linear interpolation) and 12-hour gaps (64.9% improvement over linear interpolation). app_2pavlodar achieved substantial gains (52.0% average) with peak performance of 66.9% for 12-hour gaps, while app_center showed consistent improvements (42.4% average) across all gap lengths. app_zaton exhibited strong performance (46.1% average) with notable 55.0% improvement for 48-hour gaps over rolling mean methods.

Gap length dependencies under selective compression displayed favorable patterns across all temporal scales. 12-hour gaps demonstrated the strongest average improvements (58.6%), effectively capturing sub-daily cycle dynamics through enhanced data quality. 48-hour gaps showed strong performance (53.3% average), while 24-hour gaps achieved substantial improvements (51.3% average). 72-hour gaps (44.1% average) and 5-hour gaps (36.6% average) exhibited more modest but still significant advantages.

Baseline method comparison revealed linear interpolation as the predominant competitor under selective compression, serving as the best baseline in 65% of cases. This dominance reflects the method’s effectiveness when applied to quality-filtered data with enhanced temporal coherence. Rolling mean and backward fill methods achieved competitive performance for specific station-gap combinations, while forward fill proved optimal primarily for short-term interruptions.

Performance magnitude analysis demonstrated substantial improvements across the network. Improvements exceeded 50% in 45% of cases (9/20), with the maximum improvement reaching 72.6% for app_pspu’s 24-hour reconstruction. The minimum improvement of 16.3% occurred for app_center’s 5-hour gaps, indicating that even in less favorable scenarios, the method maintained consistent advantages over conventional approaches.

Methodological insights highlight the value of selective compression for enhancing reconstruction quality. The universal success rate suggests that quality-first preprocessing strategies create more favorable conditions for sequence-to-sequence learning by reducing noise and improving temporal pattern consistency. The particularly strong performance on short to medium-duration gaps (12–48 hours) reflects the method’s ability to leverage enhanced data quality for capturing complex atmospheric dynamics.

These results establish that selective compression enhances DynamicSeq2SeqXGB’s robustness across diverse monitoring conditions, achieving universal superiority over conventional methods. The findings support the implementation of adaptive preprocessing strategies that prioritize data quality when sufficient historical observations are available for effective model training.

4.4. Comparative analysis of compression strategies

Systematic comparison between full and selective compression strategies revealed distinct performance characteristics with implications for operational deployment in environmental monitoring networks. The analysis, conducted across 20 comparable station-gap combinations (excluding app_metallurg due to selective compression incompatibility), demonstrated that strategy selection significantly impacts reconstruction quality in a context-dependent manner.

Overall performance patterns showed both compression strategies achieving universal superiority over classical baseline methods when evaluated independently. However, direct head-to-head comparison revealed balanced competition with equal win rates – full compression achieving superiority in 50% of cases (10/20) versus 50% for selective compression (10/20). The comparative analysis provides critical insights into the relative effectiveness of quality-first versus quantity-first data preparation methodologies for machine learning-based imputation.

Strategy effectiveness patterns varied substantially across temporal scales and station characteristics (Table 8). For short-term gaps (5 hours), selective compression demonstrated advantages in 75% of comparable cases (3/4), suggesting that quality-filtered data provides superior context for brief interruptions. Medium-term gaps showed mixed patterns: 12-hour gaps exhibited balanced performance (50% each), while 24-hour gaps favored full compression in 75% of cases (3/4). Extended gaps displayed divergent patterns, with 48-hour gaps favoring selective compression (75%) while 72-hour gaps preferred full compression (75%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Direct comparison of compression strategies: MAE values and strategy superiority.

https://doi.org/10.1371/journal.pone.0338788.t008

Station-specific preferences emerged as the dominant performance determinant and are more pronounced than any gap-length-based trends, as shown in Fig 6. app_pspu consistently benefited from selective compression across all gap durations, with selective improvements ranging from 16% to 41%, indicating that strict quality filtering preserves temporally coherent structure essential for reconstruction at this station. In contrast, app_2pavlodar showed uniform preference for full compression, with performance gains between 8% and 54%, reflecting the benefits of retaining all available observations at a station characterized by frequent interruptions and irregular measurement patterns. app_zaton also demonstrated an overall tendency toward full compression – particularly for medium and long gaps (12–24 hours and 72 hours) – though selective compression remained advantageous for the shortest (5 h) and 48-hour gaps, indicating mixed temporal behavior. app_center displayed a hybrid pattern: selective compression provided small improvements for short interruptions, whereas full compression delivered the largest single-station gain in the dataset (+56% at 72 hours). These patterns illustrate that environmental variability, sensor stability, and data continuity differ substantially across stations and exert stronger influence on strategy effectiveness than gap duration alone.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Strategy performance advantage by station and gap length.

Heatmap showing percentage improvement of full compression over selective compression. Positive values (orange/red) indicate full compression advantage; negative values (blue) indicate selective compression advantage. Clear horizontal patterns demonstrate station-specific preferences that exceed gap-length-based trends, with app_pspu showing consistent preference for selective compression while app_2pavlodar favors full compression across all gap lengths.

https://doi.org/10.1371/journal.pone.0338788.g006

Direct comparison analysis across matched station-gap combinations provides quantitative evidence for strategy selection patterns (Table 8). The head-to-head comparison revealed perfectly balanced overall performance, with average improvements of 28.6% when the superior strategy was applied over the inferior alternative. However, pronounced station-specific variations indicate that universal strategy recommendations are inappropriate for operational deployment.

Performance magnitude assessment demonstrated substantial differences between strategies in optimal application scenarios. The largest observed improvement occurred at app_center for 72-hour gaps, where full compression achieved 56.0% superior performance over selective compression. The peak advantage for selective compression reached 48.4% at app_zaton for 24-hour gaps. These substantial differences underscore the critical importance of appropriate strategy selection for maximizing reconstruction quality.

Gap length dependency analysis revealed complex relationships between temporal scale and strategy effectiveness (Fig 7). Linear trend analysis shows median MAE for selective compression increases with gap length, while full compression maintains relatively stable performance across temporal scales. The relative performance gap between strategies varied significantly, with individual data points demonstrating substantial station-specific variations around the fitted trends, confirming that neither strategy shows consistent dominance across all gap lengths.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Imputation error (MAE) as a function of gap length under different compression strategies.

Linear trends show median MAE across four stations (excluding app_metallurg) for selective (red) and full (blue) compression approaches. Individual data points represent MAE values for each station at different gap lengths.

https://doi.org/10.1371/journal.pone.0338788.g007

Critical operational differences emerged in methodological robustness and universal applicability. Full compression maintained universal applicability across all monitoring stations, while selective compression proved incompatible with severely compromised stations (app_metallurg elimination). This distinction has profound implications for operational networks requiring consistent methodology across heterogeneous station types, where strategy failures could create critical monitoring blind spots.

Methodological insights highlight the fundamental trade-off between data quality and data quantity in machine learning applications. Selective compression’s quality-first approach creates more favorable conditions for temporal pattern recognition when sufficient high-quality data is available, as evidenced by app_pspu’s consistent preference. However, full compression’s quantity-first methodology proves more robust across diverse data quality scenarios, particularly for stations with moderate data quality issues where additional context outweighs quality concerns.

These findings establish that strategy selection should be informed by station-specific characteristics rather than universal temporal thresholds. The strong consistency of station preferences (app_pspu and app_2pavlodar showing 100% preference for their respective optimal strategies) indicates that local data quality profiles, sensor characteristics, and environmental conditions fundamentally influence reconstruction effectiveness more than temporal gap properties alone. For operational deployment, this suggests implementing adaptive strategy selection protocols based on historical station performance profiles rather than gap-length-based decision trees.

Notably, both compression strategies demonstrated substantial peak performance advantages in their respective optimal scenarios (56.0% for full compression, 48.4% for selective compression), suggesting that strategy selection potential is significant regardless of the chosen approach, with station-specific characteristics being the determining factor for optimal performance rather than inherent strategy superiority.

4.5. Correlation Analysis of Station Characteristics and Compression Strategy Performance

To understand the factors underlying compression strategy effectiveness, Pearson correlation analysis was conducted across nine variables derived from the four-station monitoring network (excluding app_metallurg due to selective compression incompatibility). The analysis examined relationships between station characteristics and compression strategy performance, where Improvement_Full_vs_Selective_% represents the percentage performance gain of full compression relative to selective compression, Homogeneity quantifies temporal consistency in PM_2.5 measurements, Mean_PM25 and Std_PM25 denote average concentration and temporal variability respectively, and Missing_Count indicates total missing observations per station.

The correlation matrix revealed seven statistically significant relationships out of 36 possible pairwise correlations (p < 0.05), providing quantitative insight into compression strategy effectiveness patterns. The most striking finding was the perfect negative correlation between data completeness and missing count (Completeness_% ↔ Missing_Count: r = −1.000, p < 0.001), confirming the mathematical relationship between these complementary measures of data availability.

Strong positive correlations emerged between pollution characteristics and reconstruction errors for both compression strategies. Full compression MAE showed near-perfect correlation with PM_2.5 temporal variability (Full_Compression_MAE ↔ Std_PM25: r = 0.997, p = 0.003), while selective compression MAE correlated strongly with mean pollution levels (Selective_Compression_MAE ↔ Mean_PM25: r = 0.996, p = 0.004). These relationships indicate that both strategies encounter greater challenges at stations with high pollution concentrations and temporal variability, reflecting the inherent difficulty of reconstructing complex atmospheric patterns.

The analysis revealed that strategy improvement patterns are driven primarily by temporal consistency rather than data availability. Improvement_Full_vs_Selective_% showed the strongest correlation with Homogeneity (r = 0.886), indicating that full compression performs particularly well at stations with consistent temporal patterns. Conversely, stations with high temporal variability (Std_PM25: r = −0.86) and reconstruction errors (Full_Compression_MAE: r = −0.859) showed reduced advantages for full compression, though these correlations did not reach statistical significance (p > 0.05).

Station-specific analysis revealed distinct pollution profiles that explain strategy preferences. app_pspu exhibited the highest mean PM_2.5 concentrations (64.3 μg/m³) with extreme temporal variability (σ = 161.3 μg/m³), while app_center and app_zaton demonstrated lower pollution levels (25.1 and 17.5 μg/m³ respectively) with moderate variability. app_2pavlodar occupied an intermediate position with moderate mean concentrations (33.8 μg/m³) and variability (σ = 38.6 μg/m³). These pollution characteristics align with the observed station-specific strategy preferences, where high-variability stations (app_pspu) consistently favor selective compression while more stable stations (app_2pavlodar) benefit from full compression’s additional contextual information.

The strong correlation between pollution levels and temporal variability (Mean_PM25 ↔ Std_PM25: r = 0.967, p = 0.033) confirms that high-pollution stations present inherently more complex reconstruction challenges. Both compression strategies show near-perfect correlation in their error patterns (Full_Compression_MAE ↔ Selective_Compression_MAE: r = 0.990, p = 0.010), indicating that while their absolute performance may differ, they encounter similar challenges at problematic stations.

These correlational patterns provide empirical evidence that strategy selection should be based on station pollution characteristics rather than data availability metrics. The lack of significant correlation between completeness and strategy improvement (Improvement_Full_vs_Selective_% ↔ Completeness_%: r = −0.087, p = 0.913) demonstrates that data quality filtering effectiveness is independent of baseline data availability, supporting adaptive strategy selection protocols based on pollution variability and temporal consistency rather than simple completeness thresholds. Complete correlation coefficients and significance levels are provided in Supplementary Table S3 in S1 File.

4.6. External Validation (Beijing, Guanyuan, 2016)

To assess the transferability and robustness of DynamicSeq2SeqXGB beyond the training environment, external validation was conducted using high-quality reference data from the Beijing PRSA dataset (Guanyuan station, 2016, https://doi.org/10.24432/C5RK5G) [54]. This cross-regional evaluation employed the full compression strategy exclusively, providing an independent assessment of the method’s performance across different geographical, climatic, and pollution characteristics. The validation used a controlled degradation approach to simulate realistic monitoring network failures with large gaps comparable to those observed in the Pavlodar network, examining whether patterns learned from Central Asian monitoring conditions remain effective in East Asian urban environments with distinct atmospheric dynamics and pollution sources.

The Beijing Guanyuan dataset for 2016 provided an optimal validation framework with 8,784 hourly observations and 98.41% initial completeness. A complex degradation pattern was systematically applied based on gap characteristics observed in the Pavlodar app_pspu station, combining long clustered outages with scattered single-hour missing values to realistically reproduce its empirical sparsity profile.

To ensure reproducibility and clarify the construction of this missingness pattern, we explicitly mirrored the empirical gap structure of the Pavlodar app_pspu station. First, we extracted the six longest real outage episodes from app_pspu and inserted gaps of identical durations (773, 383, 360, 193, 180, and 95 hours) at randomly selected positions in the Beijing time series, thereby reproducing its characteristic long clustered failures. We then added 2,408 single-hour missing values sampled uniformly at random across the remaining valid timestamps to capture the short, dispersed interruptions typical of this station. This procedure closely matches the observed gap-length distribution of app_pspu and produces a complex missingness profile combining long clustered outages and dispersed single-point gaps (visualized in Supplementary Fig S5 in S1 File). Because the objective of this experiment was to evaluate robustness under a realistic worst-case sparsity profile derived from an operationally degraded station, we did not incorporate additional synthetic missingness schemes such as purely random, seasonal, or independently clustered gaps. This controlled degradation reduced data completeness from 98.41% to 54.64%, creating 3,844 artificially missing observations for evaluation while preserving 4,800 dense data points for model training.

Following the established methodology, the DynamicSeq2SeqXGB model was trained exclusively on the 4,800 dense data points using air quality parameters (PM₁₀, SO₂, NO₂, CO, O₃), meteorological variables (temperature, pressure, dew point, precipitation, wind speed), and temporal features (hour, day of week, month). The XGBoost regressor employed 100 estimators with maximum depth of 6 and StandardScaler normalization.

The external validation demonstrated exceptional performance across all evaluation metrics, confirming the model’s ability to maintain high reconstruction quality under challenging operational conditions (Table 9).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. External validation performance on Beijing dataset (Guanyuan, 2016).

https://doi.org/10.1371/journal.pone.0338788.t009

The strong linear relationship between predicted and actual values is clearly demonstrated in the scatter plot analysis (Fig 8), which shows excellent agreement with the ideal prediction line and confirms the absence of systematic bias in model predictions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. External validation results showing predicted vs actual PM_2.5 concentrations and error distribution for DynamicSeq2SeqXGB model on Beijing dataset.

Panel A demonstrates strong linear agreement between predicted and actual values with R² = 0.921, with the red line representing model fit and the black dashed line indicating perfect prediction (y = x).

https://doi.org/10.1371/journal.pone.0338788.g008

The DynamicSeq2SeqXGB model achieved MAE of 8.50 μg/m³, RMSE of 18.84 μg/m³, and R² of 0.944, representing substantial improvement over baseline methods across 3,844 validation points. Linear interpolation achieved only R² = 0.303, highlighting the critical advantage of the machine learning approach for complex temporal pattern reconstruction. The model maintained consistent performance across the entire concentration range (3.0–639.0 μg/m³) with minimal systematic bias (−0.58 μg/m³), demonstrating robust handling of both background and pollution episode conditions. Some tendency toward underestimation was observed at extreme pollution levels (>400 μg/m³, bias: −75.74 μg/m³ for 31 points), though this represents a small fraction of the validation dataset (0.8%) and does not significantly impact overall performance metrics.

Error behavior across concentration levels was additionally examined to evaluate whether reconstruction accuracy deteriorates during pollution episodes. The model maintained stable performance throughout most of the concentration range (0–300 μg/m³), with MAE varying moderately between 6–10 μg/m³. A clear increase in error was observed only at the upper tail of the distribution: for concentrations above 400 μg/m³ (0.8% of observations), the model exhibited systematic underestimation with an average bias of –75.74 μg/m³. This pattern reflects the scarcity of extreme pollution events in the training data rather than structural model limitations. Importantly, the small number of such events means that these localized errors do not materially affect aggregate performance metrics (MAE, RMSE, R²), which remain robust across the full concentration spectrum.

The results confirm robust cross-domain transferability, with the model trained on Central Asian data (Pavlodar) successfully generalizing to East Asian conditions (Beijing) despite different pollution sources and meteorological patterns. Performance remained stable across diverse gap configurations spanning from 48 to 773 hours, demonstrating scalability to operational monitoring challenges comparable to real-world network failures. The maintained 68.1–79.8% improvement margins over baseline methods across different geographic regions and pollutant concentration ranges provide compelling evidence that the approach captures fundamental atmospheric dynamics rather than domain-specific artifacts, supporting practical deployment in diverse air quality monitoring networks facing severe data limitations.

5.1. Performance of DynamicSeq2SeqXGB and compression strategy selection

The consistently superior performance of DynamicSeq2SeqXGB highlights its ability to capture complex environmental dynamics and demonstrates clear advantages over conventional imputation methods. This near-universal superiority contrasts markedly with previous benchmarking studies where performance advantages typically ranged from 20–40% for specialized methods. For instance, while SAITS demonstrated 12–38% improvements over RNN-based approaches in controlled settings [21], and BRITS-ALSTM achieved R² = 0.7454 under extreme sparsity conditions [3], DynamicSeq2SeqXGB delivered consistent improvements averaging 48.8% across diverse operational conditions. The missForest algorithm, previously considered state-of-the-art for environmental data with 92.41% accuracy [39], is substantially outperformed by presented approach which maintains effectiveness even at 23.3% data completeness. Recent regional studies also demonstrate the competitive performance of gradient-boosted tree models for particulate matter prediction, including XGBoost-based PM₁₀ modeling in operational contexts [55].

The balanced performance between full and selective compression strategies (50% win rate each) reveals that optimal preprocessing depends critically on station-specific characteristics rather than universal quality thresholds. This finding challenges conventional data preprocessing paradigms that assume quality filtering consistently improves machine learning performance. The correlation analysis provides empirical evidence for this station-dependency, showing that strategy improvement patterns are driven primarily by temporal consistency (Homogeneity: r = 0.886) rather than data availability (Completeness: r = −0.087, p = 0.913).

Station-specific preferences demonstrated remarkable consistency, with app_pspu exclusively benefiting from selective compression across all gap lengths (improvements 16.2–41.1%) while app_2pavlodar consistently favored full compression (improvements 8.0–54.3%). This consistency suggests that local pollution characteristics and temporal patterns fundamentally influence optimal preprocessing strategies more than gap-specific properties.

The app_metallurg case exemplifies both the challenges and limitations of imputation under extreme sparsity. While full compression enabled reconstruction where selective compression failed entirely (due to 0% qualifying months), the resulting MAE values (29.71–32.23 μg/m³) were substantially higher than other stations (3.67–8.52 μg/m³), reflecting the inherent difficulty of accurate imputation at 23.3% completeness. This demonstrates that while DynamicSeq2SeqXGB can extract meaningful patterns from severely compromised datasets, reconstruction quality inevitably degrades as data sparsity increases.

The theoretical foundation for balanced strategy performance lies in fundamental trade-offs between data quality and quantity. Selective compression creates favorable conditions for pattern recognition by reducing noise, as evidenced by app_pspu’s consistent preference and strong correlation between pollution variability and reconstruction difficulty (Mean_PM25 ↔ Std_PM25: r = 0.967, p = 0.033). Conversely, full compression preserves temporal dependencies that may contain essential reconstruction information, particularly beneficial for stations with moderate data quality where additional context outweighs noise concerns.

The framework’s emphasis on station-specific strategy selection reflects operational priorities where consistent performance across heterogeneous conditions outweighs marginal accuracy gains in ideal scenarios. Rather than seeking universal optimization, the approach provides operators with clear, actionable guidance based on quantifiable station characteristics, simplifying deployment protocols while maintaining reconstruction effectiveness across diverse monitoring network conditions.

5.2. Theoretical implications of context-dependent strategy selection

The equal performance of full and selective compression strategies (50% win rate each) reveals fundamental principles about data preprocessing in extreme sparsity scenarios that merit theoretical consideration. Rather than validating universal preprocessing paradigms, these results demonstrate that optimal strategy selection depends on the complex interplay between data characteristics, temporal patterns, and reconstruction requirements.

The station-specific consistency of strategy preferences indicates that preprocessing effectiveness stems from alignment between data characteristics and algorithmic requirements rather than absolute data quality metrics. Correlation analysis supports this interpretation, showing that strategy improvement correlates with temporal consistency (Homogeneity: r = 0.886) while remaining independent of baseline data availability (Completeness: r = −0.087, p = 0.913). This pattern suggests that successful imputation depends on matching preprocessing approaches to underlying temporal structure rather than maximizing data cleanliness.

The theoretical foundation underlying these results centers on information preservation versus noise reduction trade-offs. Selective compression optimizes signal-to-noise ratios by eliminating low-quality observations, creating favorable conditions for pattern recognition when sufficient high-quality data exists. This approach proves particularly effective for stations with high pollution variability (app_pspu), where temporal inconsistency benefits from quality filtering. Conversely, full compression preserves temporal dependencies that may contain reconstruction-relevant information, particularly valuable when moderate noise levels are offset by increased contextual breadth.

These findings challenge the conventional assumption that preprocessing should universally optimize data quality before algorithmic application. Instead, the results suggest that preprocessing strategies should be viewed as integral components of the overall reconstruction system, where effectiveness depends on compatibility between data characteristics, preprocessing approach, and algorithmic architecture. The XGBoost ensemble’s demonstrated ability to handle mixed-quality inputs enables this flexibility, suggesting that robust algorithms may benefit more from comprehensive information access than from pre-filtered datasets. Comparative evaluations of ensemble and boosting architectures for long-term air-quality forecasting further support the advantages of tree-based ensembles in handling heterogeneous temporal patterns [56].

The implications extend beyond environmental monitoring to broader questions about data preparation in machine learning applications. The station-specific nature of optimal strategies indicates that preprocessing decisions should incorporate domain knowledge about data generation processes, temporal dynamics, and measurement characteristics rather than relying solely on statistical quality metrics. This perspective shifts focus from universal best practices toward adaptive, context-aware preprocessing approaches that optimize system-level performance rather than individual component quality.

However, these theoretical implications must be interpreted within the study’s limitations. The analysis encompasses only four stations with PM_2.5 data, limiting generalizability across different environmental parameters, geographic regions, and monitoring technologies. The observed patterns may reflect specific characteristics of atmospheric pollutant dynamics rather than universal principles applicable to other domains with sparse time series data. Further research across diverse applications would be necessary to validate broader theoretical claims about context-dependent preprocessing optimization.

5.3. Practical implications for Environmental Monitoring Networks

The practical significance of these findings centers on optimizing data utilization within existing monitoring infrastructure constraints rather than transforming fundamental operational paradigms. The demonstrated station-specific strategy preferences provide actionable guidance for operators facing decisions about data preprocessing approaches, though the implications require careful interpretation within the study’s empirical limitations.

The most immediate practical benefit lies in establishing evidence-based protocols for compression strategy selection. Stations exhibiting high pollution variability and temporal inconsistency (similar to app_pspu) benefit consistently from selective compression, while stations with moderate data quality and stable patterns (similar to app_2pavlodar) perform better under full compression. This station-specific approach contrasts with current practices that often apply uniform preprocessing protocols across entire networks, potentially suboptimizing reconstruction quality for individual sites.

The app_metallurg case illustrates both possibilities and limitations under extreme sparsity conditions. While full compression enabled reconstruction where selective compression failed entirely, the resulting reconstruction errors (MAE 29.71–32.23 μg/m³) substantially exceeded those achieved at other stations (3.67–8.52 μg/m³). Whether such reconstruction quality constitutes “meaningful” data depends critically on intended applications – these error levels might suffice for broad trend identification but would be inadequate for regulatory compliance or detailed exposure assessment. The cost-benefit analysis of maintaining severely compromised stations requires evaluation against specific operational requirements rather than assuming universal value.

The framework’s emphasis on station-specific optimization reflects operational realities where heterogeneous infrastructure conditions necessitate adaptive rather than standardized approaches. However, this customization introduces implementation complexity, requiring operators to characterize station-specific temporal patterns and maintain differentiated preprocessing protocols. The additional analytical overhead must be weighed against potential reconstruction improvements when determining operational feasibility.

Regarding broader applicability to other environmental parameters, the current evidence base remains limited to PM_2.5 measurements from Central Asian monitoring conditions. While the methodology’s reliance on temporal pattern recognition suggests potential transferability to other continuously monitored pollutants (SO₂, NO₂, O₃), parameter-specific validation would be essential given different measurement characteristics, temporal dynamics, and typical concentration ranges.

Beyond pollutant-specific considerations, applicability across different geographical contexts also warrants attention. Although the empirical analysis in this study is centered on Pavlodar, the DynamicSeq2SeqXGB framework is not tied to region-specific features. The model operates on general temporal properties of air-quality time series, including diurnal structure, short-term temporal dependencies, and characteristic gap patterns, and relies only on routinely collected pollutant concentrations and basic meteorological variables. As a result, its applicability is primarily determined by data sparsity and the availability of core covariates rather than by local emission profiles. External validation on the high-quality Beijing Guanyuan PM2.5 time series, implemented through controlled degradation of an originally near-complete dataset, further supports this generality: despite substantial differences in climatic conditions and urban emission regimes between Beijing and Pavlodar, the model maintained high reconstruction accuracy. This indicates that the proposed approach can be transferred to other regions with comparable monitoring configurations, provided that similar temporal resolution and essential meteorological information are available.

The computational requirements of DynamicSeq2SeqXGB relative to classical methods represent another practical consideration. While the performance improvements often justify increased processing costs, resource-constrained monitoring programs must evaluate this trade-off based on their specific accuracy requirements and computational capabilities. The method’s integration with existing data management systems would require careful planning to ensure sustainable implementation.

In practical terms, the computational burden of the proposed approach is modest. Because the encoder is based on gradient-boosted decision trees rather than deep recurrent or attention-based architectures, full model training for a single station typically completes within several minutes on a standard CPU (Intel i7–class). Once trained, inference is extremely lightweight: predictions for a given context–gap window are produced within milliseconds, enabling on-the-fly reconstruction without GPU resources or specialized hardware. This efficiency makes DynamicSeq2SeqXGB suitable for real-time or near–real-time deployment in municipal monitoring systems, where processing capacity is often limited and continuous data delivery is required.

The framework demonstrates most value for monitoring networks facing chronic data quality challenges where conventional approaches consistently underperform. For well-maintained networks with minimal data gaps, the added complexity may provide marginal benefits over simpler interpolation methods. The optimal application domain appears to be intermediate scenarios where data sparsity creates reconstruction challenges but sufficient temporal structure remains for machine learning approaches to extract meaningful patterns.

5.4. Study limitations and methodological considerations

Several considerations warrant discussion regarding the scope and generalizability of these findings. The correlation analysis, conducted across four monitoring stations, provides meaningful insights into station-specific strategy preferences while acknowledging that larger networks would further strengthen statistical inferences about the relationships between environmental conditions and optimal preprocessing approaches.

The focus on PM_2.5 reflects its critical importance for public health assessment and regulatory compliance. While this concentration provides a solid foundation for method development, extension to other atmospheric pollutants would benefit from parameter-specific validation, particularly for species with distinct temporal dynamics such as photochemical oxidants or traffic-related pollutants. The demonstrated cross-regional transferability between Pavlodar and Beijing suggests promising potential for broader geographic application, though validation in diverse climatic conditions would strengthen confidence in global applicability.

The 15-month study period captured multiple seasonal cycles and diverse pollution episodes, providing a robust foundation for method evaluation. This temporal coverage encompasses the typical range of atmospheric conditions encountered in operational monitoring, though longer-term studies would provide additional insights into method stability across inter-annual variability and extreme pollution events.

Methodological considerations inherent to the synthetic gap approach are addressed through systematic controls. Gap patterns reflect diverse failure scenarios commonly encountered in operational networks, while context availability assumptions align with realistic deployment scenarios where complete network failures are uncommon. The temporal splitting methodology preserves chronological dependencies, ensuring that evaluation reflects realistic prediction scenarios rather than retrospective interpolation.

The computational requirements of DynamicSeq2SeqXGB represent a practical consideration for implementation planning. While substantially more efficient than deep learning alternatives, the XGBoost-based approach requires greater processing resources than classical interpolation methods. This trade-off between accuracy and computational efficiency aligns with broader trends toward sophisticated environmental data analysis and can be managed through appropriate system design.

The emphasis on MAE as the primary evaluation metric provides interpretable error assessment directly relevant to air quality applications. The complementary use of RMSE, R², and cross-regional validation strengthens confidence in the performance assessment, while the observed patterns remain consistent across multiple evaluation approaches.

These considerations collectively indicate that while the current study provides strong evidence for DynamicSeq2SeqXGB’s effectiveness within environmental monitoring applications, implementation benefits from local validation and adaptation to specific operational requirements. The method’s demonstrated robustness across diverse data quality conditions and geographic settings provides a solid foundation for broader deployment in operational monitoring networks facing data reconstruction challenges.

5.5. Future research directions

The demonstrated station-specific optimization principles warrant extension across broader environmental monitoring contexts. Validation of DynamicSeq2SeqXGB for other atmospheric pollutants (O₃, NO₂, SO₂, CO) would establish whether the adaptive context processing mechanisms generalize beyond PM_2.5 temporal dynamics. Each parameter presents distinct challenges: O₃’s photochemical cycles, NO₂’s traffic-related patterns, and SO₂’s industrial emission profiles may require modified hyperparameter configurations or alternative context window strategies.

Geographic expansion represents the most immediate research priority. Testing across diverse climatic zones – tropical monsoon regions, arctic conditions, coastal marine environments – would determine whether the Beijing-Pavlodar transferability extends to fundamentally different atmospheric regimes. These studies should focus on regions with established monitoring networks suffering chronic data quality issues rather than well-maintained systems.

Multi-station imputation architectures could enhance reconstruction for severely compromised sites by incorporating spatial correlations between monitoring locations. However, this extension requires addressing substantial technical challenges: synchronizing multi-dimensional input processing, managing variable network topologies, and scaling computational requirements. The potential benefits must be weighed against increased architectural complexity and data requirements.

Long-term performance assessment under extended deployment conditions would address questions about model stability during extreme pollution episodes, seasonal transitions, and gradual changes in local emission patterns. Particular attention should focus on performance degradation over time and requirements for model retraining or adaptation to evolving environmental conditions.

Investigation of alternative ensemble architectures beyond XGBoost – including CatBoost, LightGBM, or hybrid approaches combining gradient boosting with neural sequence models – may yield computational or accuracy improvements. These comparisons should emphasize operational feasibility rather than marginal performance gains unsuitable for practical deployment.

The methodology’s applicability to other environmental time series with similar sparsity challenges – hydrological monitoring, soil sensor networks, biodiversity tracking systems –represents a logical extension within the environmental domain. These applications share characteristics of irregular data availability, temporal dependencies, and the need for robust reconstruction under challenging field conditions.

These directions should proceed with systematic validation rather than assuming transferability, focusing on domains where the method’s specific strengths – handling extreme sparsity, adaptive context processing, station-specific optimization – address documented challenges in existing approaches.