Point-forecast accuracy metrics.

The Mean Absolute Error (MAE) measures the average magnitude of forecast errors and is robust to occasional large deviations:

(1)

The Root Mean Squared Error (RMSE) penalizes larger errors more strongly, which is relevant for lumpy demand patterns where occasional high-demand events may occur:

(2)

To provide scale-aware percentage-based error reporting, the Mean Absolute Percentage Error (MAPE) and Weighted Mean Absolute Percentage Error (WMAPE) are also reported. Since MAPE is undefined for zero observations, it is computed only for instances with :

(3)

where is the number of observations with non-zero demand. The WMAPE mitigates the instability of MAPE in sparse series by scaling absolute error by total demand volume:

(4)

The Mean Absolute Scaled Error (MASE) is additionally used because it is scale-free and well suited for intermittent demand. MASE scales the forecast error by the in-sample MAE of a naïve one-step (random-walk) forecast. A value indicates the model outperforms the naïve benchmark:

(5)

where is the actual demand at time , is the forecast, and is the number of observations.

Finally, the Coefficient of Determination () is reported to summarize goodness-of-fit in terms of explained variance:

(6)

where is the mean of the observed demand.

Probabilistic forecast evaluation.

Since the proposed ZIG MC framework generates a full predictive distribution (rather than a single point estimate), probabilistic evaluation is necessary to assess both calibration (statistical consistency) and sharpness (concentration of uncertainty). The continuous ranked probability score (CRPS) is used as a proper scoring rule for probabilistic forecasts. CRPS measures the squared distance between the predicted cumulative distribution function (CDF) and the empirical CDF of the observation:

(7)

where is the predictive CDF and is the observed demand. is an indicator function equal to 1 if and 0 otherwise. Lower CRPS values indicate better probabilistic forecasting performance, reflecting both improved calibration and sharper predictive distributions [38].

Single-stage feature-based regression models.

These benchmarks generate a single point forecast and do not explicitly model demand occurrence and magnitude separately.

• K-nearest neighbors (KNN) regressor: A non-parametric, similarity-based forecasting method widely used in supply chain applications. For a new spare part, KNN identifies the most similar parts in the training set based on Euclidean distance in the standardized feature space and predicts demand as the average of their observed demand values [39].
• Bayesian time-weighted manifold k-nearest neighbors (BTM-KNN): An enhanced baseline developed in this study to strengthen the classical KNN approach. The method incorporates three extensions: (i) dimensionality reduction of the feature space using Principal Component Analysis (PCA), (ii) temporal decay weighting to emphasize more recent observations, and (iii) Bayesian inverse-variance weighting to account for uncertainty in neighbor contributions [40].
• Support vector regression (SVR): The regression counterpart of Support Vector Machines, SVR estimates a function that fits the data within a predefined error margin () while maximizing flatness. This margin-based formulation provides robustness to outliers, which is advantageous in highly variable and intermittent demand settings [41].
• Ridge regression: A regularized linear regression model that incorporates an L2 penalty on the regression coefficients. Ridge regression mitigates multicollinearity among predictors by shrinking coefficients toward each other, improving numerical stability and generalization performance [42].
• Lasso regression: A sparse linear regression model that applies an L1 penalty to the loss function. Unlike Ridge regression, Lasso can shrink less informative coefficients exactly to zero, thereby performing implicit feature selection in high-dimensional feature spaces [43].
• CatBoost regressor: A single-stage Gradient Boosted Decision Tree (GBDT) model specifically designed to handle heterogeneous tabular data and categorical features. This benchmark provides the most direct comparison to the proposed ZIG MC framework, as CatBoost is also used as the underlying learner within ZIG MC. Comparing these two models isolates the performance gains attributable to the two-part probabilistic structure rather than the base learner itself [44].

Two-part and zero-inflated statistical models.

These benchmarks explicitly separate demand occurrence from demand magnitude, which is critical for semi-continuous and zero-inflated demand patterns.

• Zero-inflated (ZI) two-part model: A standard hurdle-type approach for zero-inflated data. The model consists of (i) a logistic regression to estimate the probability of non-zero demand occurrence and (ii) a regression model applied only to positive demand observations to predict demand magnitude. This baseline enables isolation of the specific contributions of the Gamma distributional assumption and Monte Carlo simulation employed in the proposed ZIG MC framework [45].
• Tweedie regression model: A single-stage Generalized Linear Model (GLM) characterized by a variance power parameter , allowing it to represent distributions ranging from Poisson () to Gamma (). For spare-part demand, the Compound Poisson-Gamma regime () is used, which naturally accommodates a point mass at zero while maintaining a continuous distribution for positive demand values. The Tweedie model serves as a strong statistical benchmark for handling semi-continuous demand within a unified modeling framework [46].

Probabilistic deep learning model (DeepAR).

DeepAR is a state-of-the-art deep learning architecture based on recurrent neural networks (RNNs) designed for probabilistic time-series forecasting. While DeepAR is traditionally applied to longitudinal demand histories, it is adapted here to the cold-start setting by conditioning the global RNN on the vector of part-level attributes. Unlike deterministic regression models, DeepAR outputs a full predictive distribution, enabling a direct comparison with the probabilistic forecasts produced by the ZIG MC framework [47].

Together, these benchmarks allow evaluation across three complementary dimensions: (i) single-stage point forecasting performance, (ii) classical two-part and zero-inflated statistical modeling, and (iii) modern probabilistic forecasting capability. This comprehensive benchmark suite ensures that the performance gains of the proposed ZIG MC framework are not merely due to model complexity or choice of base learner, but rather to its explicit treatment of zero-inflation, distributional modeling of demand magnitude, and Monte Carlo-based uncertainty quantification.

Part 1: Probabilistic event model (classifier).

First, a binary classifier is trained to model the probability of a demand event occurring. A binary target variable is created for each data point :

(8)

The classifier is trained on the full dataset to learn the mapping from a part’s features to its probability of having non zero demand. This output is the event probability, .

(9)

Part 2: Demand magnitude model (regressor).

Second, a regression model is trained to predict the magnitude of demand, conditional on an event occurring. This model is trained only on the positive subset of the data where . This prevents the zero values from biasing the model’s estimate of a typical sale size.

The positive demand is assumed to follow a Gamma distribution, , which is well suited for right skewed, positive only data [24]. The regressor is trained to predict the conditional mean of this distribution:

(10)

With the mean related to the Gamma parameters by , the scale parameter for each part can be derived by fixing the shape parameter (e.g., ) and using the model’s prediction:

(11)

For any new part x_i, the model now provides the full parameterized distribution for a positive sale: .

Part 3: Probabilistic forecasting via monte Carlo simulation.

The final step is to synthesize these two outputs to generate a probabilistic forecast. Monte Carlo Simulation (MCS) is a computational technique that uses repeated random sampling to approximate the behavior of complex systems or, in this case, to determine the resulting distribution from a probabilistic model.

It is important to distinguish this simulation approach from others, such as Discrete Event Simulation (DES). DES is used to model complex systems and processes (e.g., an entire warehouse operation with queues and resources). In contrast, Monte Carlo simulation is the correct tool for this problem, as its purpose is to model uncertainty and risk. The simulation is the process of repeatedly sampling from the probability distributions defined by the two-part model to build a full picture of the possible outcomes [48].

For a single new part x_i, the simulation runs N times (e.g., N = 1000). For each simulation j ∈ {1,..., N}:

1. Define Input Distributions: The ZIG model provides two probabilistic inputs: from the classifier and the distribution from the regressor.
2. Randomly Sample and Evaluate: A random number is drawn.
   1. If , the event is “no sale,” and the simulated demand is .
   2. If , the event is “sale,” and a demand value is drawn from the parameterized Gamma distribution: .
3. Repeat and Analyze: This is repeated times to form an empirical distribution of possible futures .

`By the Law of Large Numbers, the average of this empirical distribution converges to the true expected value .

(12)

This mean value is used as the point forecast for comparison against the benchmark models. However, the true, actionable output is the full distribution , which can be used to calculate percentiles (e.g., the 95th percentile) for setting service level-based inventory policies.

While Zero-Inflated Poisson (ZIP) models are commonly used for discrete count data, their applicability in the present context is limited by the structural assumptions of the Poisson distribution. In particular, the Poisson formulation constrains the variance to be equal to the mean, which restricts its ability to capture the over-dispersion and heavy-tailed behavior frequently observed in spare-part demand. In practical industrial settings, especially when demand is aggregated over time or expressed in cost or variable quantity units, the positive-demand component often exhibits continuous, right-skewed characteristics that deviate significantly from simple count processes.

In contrast, the Zero-Inflated Gamma (ZIG) formulation provides greater flexibility by modeling the positive-demand magnitude using a continuous Gamma distribution, allowing independent control over mean and variance. This enables more accurate representation of variability and tail behavior in intermittent demand. Additionally, the Gamma-based formulation integrates naturally with the Monte Carlo simulation framework adopted in this study, facilitating the generation of full predictive distributions for downstream probabilistic evaluation and inventory decision-making. For these reasons, the ZIG model is considered more appropriate than ZIP for the present cold-start forecasting setting.

To ensure reproducibility despite data access constraints, comprehensive methodological details are provided in the Supporting Information. These include a full list of engineered features and their definitions, pseudo-code for the part-level nested cross-validation and cold-start simulation protocol, and the final optimized hyperparameters for all benchmark models evaluated in this study.

Justification of the gamma distribution for positive demand: The magnitude of non-zero demand is modeled using a Gamma distribution due to its well-established suitability for non-negative, right-skewed data with variance that increases with the mean. Empirical inspection of the positive-demand observations in the dataset revealed a strongly right-skewed distribution with occasional large realizations, a characteristic commonly reported in spare-parts and service-component demand. The Gamma family provides sufficient flexibility to capture this behavior while maintaining analytical tractability and compatibility with generalized linear modeling frameworks frequently used in intermittent demand analysis.

From a generative perspective, intermittent spare-part demand is often conceptualized as a compound process in which demand occurrence is governed by a Bernoulli mechanism and positive demand sizes arise from a continuous, skewed distribution. The Gamma distribution is therefore a natural choice for modeling the conditional magnitude component within a two-part formulation.

Justification for using a fixed shape parameter: In the baseline configuration, the Gamma shape parameter was fixed at . This decision was motivated by two practical considerations specific to the true cold-start setting. First, reliable estimation of both shape and scale parameters requires sufficient positive-demand observations for each part, which are unavailable for newly introduced items. Estimating a part-specific shape parameter under extreme sparsity can lead to unstable likelihood surfaces and overfitting. Fixing the shape parameter provides numerical robustness and ensures consistent distributional behavior across parts. Second, fixing reduces model variance while preserving sufficient flexibility through the scale parameter, which remains conditioned on part attributes. This strategy is commonly adopted in sparse-data regimes to stabilize probabilistic predictions.

To verify that this assumption does not bias the results, a comprehensive sensitivity analysis was conducted by varying the shape parameter across a wide range (), while holding all other components fixed. The results1, demonstrate that:

• Point forecast accuracy (MAE and MASE) remains highly stable across the entire range.
• Probabilistic performance (CRPS) improves from highly skewed regimes and stabilizes for moderate values of .
• The baseline choice lies within a broad region of near-optimal performance.

These findings confirm that the predictive gains of the proposed framework arise from its two-stage probabilistic structure rather than from fine-tuning of distributional parameters. For settings where richer historical data are available, the shape parameter could alternatively be estimated using method-of-moments or maximum likelihood procedures, representing a natural extension of the current framework.

Overall point-forecast accuracy.

The Table 1 summarizes the mean performance of all benchmark models on the held-out cold-start test set, while Figs 1 and 2 provide a visual comparison across complementary evaluation metrics.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparative model performance on the cold-start tests set (mean values).

https://doi.org/10.1371/journal.pone.0350729.t001

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Comparative performance of models on the cold-start test set: MAE and RMSE & MAPE and WMAPE.

https://doi.org/10.1371/journal.pone.0350729.g002

As shown in Fig 2 (MAE and RMSE), the proposed ZIG MC framework consistently achieves the lowest absolute and squared error values among all models. This improvement is particularly meaningful in cold-start settings, where errors are dominated by incorrect handling of zero-demand periods and occasional large, lumpy orders.

The reduction in MAE from 6.0368 (single-stage CatBoost) to 5.6532 (ZIG MC) corresponds to a 6.4% improvement, achieved without changing the underlying learning algorithm. This isolates the performance gain to the two-stage probabilistic structure, rather than model capacity or tuning advantages.

The Fig 2 (MAPE and WMAPE) further highlights the structural benefits of ZIG MC. Single-stage models such as Ridge and CatBoost exhibit extremely high MAPE values (>95%), reflecting systematic penalties when predicting non-zero demand during true zero-demand periods. In contrast, ZIG MC explicitly models demand occurrence, leading to a 38% reduction in MAPE relative to CatBoost. This confirms that the proposed framework is substantially better aligned with the zero-inflated nature of intermittent spare-part demand.

To further assess the consistency of model performance across cold-start partitions, Fig 3 presents box-plot comparisons of MAE, MAPE, and WMAPE across the five outer validation folds. Unlike tabulated mean values, the box-plots provide a direct visualization of median error, interquartile range, and fold-wise variability. The results show that the proposed ZIG MC framework consistently achieves lower error distributions with reduced spread across folds, indicating robust generalization to previously unseen parts. Each box represents the interquartile range (IQR), with the median shown as a horizontal line and whiskers extending to 1.5 × IQR. Lower values indicate better performance. The proposed ZIG MC model demonstrates consistently lower error distributions and reduced variability compared with benchmark models, confirming its robustness under true cold-start conditions.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Cross-fold distribution of forecasting errors under cold-start validation.

Box-plots summarize model performance across the five outer validation folds for (a) MAE, (b) MAPE, and (c) WMAPE.

https://doi.org/10.1371/journal.pone.0350729.g003

Scale-free accuracy and robustness across demand magnitudes.

While MAE and RMSE provide absolute error magnitudes, Mean Absolute Scaled Error (MASE) offers a scale-free comparison relative to a naïve benchmark, making it particularly suitable for heterogeneous spare-part portfolios. As illustrated in Fig 4 (MASE), ZIG MC achieves the lowest mean MASE value (0.8667) among all evaluated models. A MASE below 1 indicates that the model consistently outperforms a naïve random-walk forecast. Importantly, several widely used baselines (e.g., SVR and Tweedie) exceed this threshold, indicating inferior performance even relative to simple heuristics [49].

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Comparative performance of models on the cold-start test set: R² & MASE.

https://doi.org/10.1371/journal.pone.0350729.g004

The Tweedie GLM, despite being theoretically well suited for semi-continuous demand, performs poorly in terms of MASE. This suggests that a single-stage parametric formulation struggles to capture the heterogeneous, feature-driven demand patterns characteristic of cold-start spare parts.

Model stability and cross-fold consistency.

The Table 2 reports the standard deviation of all metrics across outer cross-validation folds, while Figs 1 and 2 visually reinforce performance consistency. ZIG MC demonstrates low variability across folds, particularly for MAPE, WMAPE, and MASE. As shown in Figs 1 and 2, its performance advantage is not driven by isolated folds but is systematically maintained across different cold-start partitions. This robustness is critical for operational deployment, where new parts continuously enter the system. In contrast, models such as SVR and DeepAR exhibit high variability, indicating sensitivity to the specific composition of the training set and limited generalization under extreme sparsity.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Comparative model performance on the cold-start test set (standard deviation values).

https://doi.org/10.1371/journal.pone.0350729.t002

Single-stage vs. two-stage modeling.

The comparison between single-stage CatBoost and ZIG MC provides the clearest insight into the source of performance gains. Both models use the same gradient-boosted decision tree architecture; however, CatBoost predicts demand magnitude directly, whereas ZIG MC decomposes the problem into demand occurrence and demand magnitude. As observed in Fig 2, the single-stage CatBoost model frequently incurs large percentage errors due to false-positive predictions during zero-demand periods. ZIG MC mitigates this by explicitly modeling the probability of demand occurrence, thereby reducing structural bias in intermittent regimes [50].

The Zero-Inflated two-stage baseline confirms this insight: separating occurrence and magnitude alone yields substantial gains over single-stage models. However, ZIG MC further improves performance through its Gamma-based magnitude modeling and Monte Carlo simulation, which better capture skewness and tail behavior in non-zero demand.

Performance of probabilistic and neural benchmarks.

The DeepAR model performs significantly worse than all other benchmarks across MAE, MAPE, and MASE, as clearly shown in Figs 2–4. This degradation arises from DeepAR’s reliance on temporal demand histories, which are unavailable in true cold-start scenarios. Conditioning solely on static attributes limits its ability to infer demand dynamics, particularly under extreme zero inflation.

These results highlight a key insight: state-of-the-art neural probabilistic models are not inherently suited to cold-start forecasting unless substantial historical context is available. In contrast, ZIG MC is explicitly designed for feature-only prediction under sparsity, giving it a decisive advantage in this setting.

Beyond point accuracy: Motivation for probabilistic evaluation.

Although ZIG MC outperforms all benchmarks on point-forecast metrics, these results understate its primary contribution. As shown in Figs 2–4, point accuracy alone cannot characterize uncertainty, tail risk, or service-level implications.

ZIG MC generates a full predictive distribution, enabling direct evaluation using probabilistic scoring rules and risk-based metrics. This capability is examined in the next subsection using CRPS, where ZIG MC is compared directly against DeepAR in a fully probabilistic setting (Fig 5).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Comparative probabilistic performance of ZIG MC and DeepAR on the cold-start test set using Continuous Ranked Probability Score (CRPS).

https://doi.org/10.1371/journal.pone.0350729.g005

To verify that the observed performance improvements are not due to random variation across cross-validation folds, a Diebold-Mariano (DM)-style paired statistical test was conducted on the fold-wise forecast errors. For each outer fold, the loss differential between the proposed ZIG MC model and the strongest competing benchmark (single-stage CatBoost) was computed using the absolute error loss. The null hypothesis of equal predictive accuracy was rejected at conventional significance levels, indicating that the reduction in forecast error achieved by ZIG MC is statistically significant and systematic, rather than arising from fold-specific effects. This result provides additional confidence that the observed gains reflect a genuine improvement in modeling intermittent, zero-inflated cold-start demand.

Although the ZIG MC framework achieves consistent and statistically significant improvements in MAE, MAPE, and WMAPE, the corresponding improvement in R² is marginal. This behavior is expected in highly zero-inflated demand settings, where the total variance of the target variable is dominated by the contrast between zero and non-zero demand rather than by fine-grained differences in forecast accuracy. As a variance-based metric, R² primarily reflects how well a model explains overall dispersion, but it is largely insensitive to improvements in correctly predicting zero-demand events and reducing systematic false positives.

Consequently, models that differ substantially in operational accuracy and percentage error may still exhibit similar R² values. This limitation underscores the importance of complementing R² with absolute, percentage-based, and probabilistic metrics when evaluating intermittent-demand forecasts [51]. To verify that the observed performance improvements are not attributable to random variation across cold-start folds, formal paired statistical significance testing was conducted. The results, confirm that the improvements achieved by ZIG MC over all benchmark models are statistically significant.

CRPS comparison of probabilistic forecasting models.

The Table 3 reports the mean and standard deviation of CRPS values for the probabilistic benchmark models evaluated on the cold-start test set.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Comparative probabilistic model performance on the cold-start test set.

https://doi.org/10.1371/journal.pone.0350729.t003

As illustrated in Fig 5, the proposed ZIG MC framework achieves a substantially lower CRPS than DeepAR, with both a lower mean value and markedly reduced variability across folds. This indicates not only superior average probabilistic accuracy, but also stable calibration behavior across different cold-start splits [52]. The magnitude of the CRPS gap is notable: ZIG MC reduces CRPS by nearly an order of magnitude relative to DeepAR. Such a difference cannot be attributed to random variation and instead reflects fundamental differences in how uncertainty is modeled under cold-start conditions.

Decomposition of probabilistic performance: Calibration vs sharpness.

CRPS can be conceptually decomposed into two interacting components: (i) calibration, reflecting whether observed outcomes are statistically consistent with the predicted distribution, and (ii) sharpness, reflecting the concentration of the predictive distribution. The superior CRPS performance of ZIG MC arises from improvements along both dimensions.

Calibration of zero-demand probability: A defining characteristic of intermittent spare-part demand is the high frequency of true zero-demand periods. ZIG MC explicitly models this phenomenon through a Bernoulli occurrence component, allowing it to place a discrete probability mass at zero demand. When zero demand is observed, the predicted CDF exhibits a sharp jump at zero, closely matching the empirical distribution.

In contrast, DeepAR represents demand as a continuous distribution without an explicit zero-mass component. Under cold-start conditions, this leads to systematic misallocation of probability mass, with excessive density assigned to small positive demand values. As a result, when true demand is zero, the predicted CDF deviates substantially from the empirical step function, incurring a large CRPS penalty. This calibration mismatch explains a significant portion of the CRPS gap observed in Fig 5.

Sharpness and tail behavior for positive demand: Beyond zero-demand calibration, ZIG MC also demonstrates superior sharpness for positive demand outcomes. The Gamma distribution used for modeling demand magnitude naturally captures the right-skewed, heavy-tailed nature of spare-part demand. When combined with Monte Carlo simulation, this produces predictive distributions that are concentrated around plausible demand levels while still allowing for rare, high-impact events.

DeepAR, by contrast, tends to generate overly diffuse predictive distributions under feature-only cold-start conditions. Lacking historical temporal context, the model compensates by inflating variance, resulting in overly conservative uncertainty estimates. This diffusion reduces sharpness and increases CRPS, even when the mean forecast is approximately correct.

Cold-start limitations of sequence-based probabilistic models.

The poor probabilistic performance of DeepAR highlights an important methodological insight. While DeepAR is highly effective in settings with rich temporal histories, it is fundamentally designed to exploit shared sequential patterns across time series. In a true cold-start scenario where no demand history exists and only static part attributes are available; this inductive bias becomes a limitation.

As shown in Fig 5, DeepAR’s predictive distributions are poorly aligned with observed outcomes under extreme sparsity and zero inflation. This result underscores that probabilistic deep learning models are not universally superior, and that model suitability must be evaluated relative to the data-generating process and operational constraints [53]. In contrast, ZIG MC is explicitly designed for feature-driven cold-start forecasting, making no assumptions about temporal continuity and directly modeling the two dominant sources of uncertainty: demand occurrence and demand magnitude.

Implications for inventory risk and service-level decisions.

From a practical perspective, improved probabilistic calibration has direct implications for inventory management. Lower CRPS indicates that predicted demand quantiles more accurately reflect realized demand variability, enabling reliable estimation of service levels, stock-out probabilities, and safety stock requirements.

In particular, the accurate representation of tail behavior achieved by ZIG MC allows decision-makers to select stocking policies based on quantile forecasts (e.g., 90th or 95th percentiles) with greater confidence. Over-dispersed distributions, such as those produced by DeepAR in this setting, can lead to systematic overstocking, while poorly calibrated zero-demand probabilities increase the risk of unnecessary inventory accumulation. By achieving substantially lower CRPS, ZIG MC provides a more trustworthy probabilistic foundation for risk-informed inventory planning in cold-start environments.

In summary, the probabilistic evaluation confirms that the advantages of ZIG MC extend well beyond improved point accuracy. The framework delivers:

• Superior calibration of zero-demand events
• Sharper and more realistic magnitude distributions
• Robust probabilistic performance across cold-start folds
• Clear operational relevance for inventory risk management

These results demonstrate that ZIG MC is not only a more accurate forecaster, but also a fundamentally better probabilistic model for intermittent, zero-inflated spare-part demand under cold-start conditions.

Reliability of quantile forecasts.

Beyond accuracy, it is essential that predicted quantiles are reliable, meaning that empirical coverage matches nominal probabilities. Fig 7 presents reliability (calibration) plots comparing ZIG MC and DeepAR.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Reliability plot comparing probabilistic calibration of ZIG MC and DeepAR on the cold-start test set.

The dashed diagonal represents perfect calibration.

https://doi.org/10.1371/journal.pone.0350729.g007

ZIG MC closely follows the diagonal reference line across the entire probability range, indicating well-calibrated quantile forecasts. Minor conservative deviations at high quantiles are observed, which are operationally desirable, as they reduce the risk of stock-outs under demand uncertainty. In contrast, DeepAR exhibits systematic miscalibration across nearly all quantiles, with empirical coverage consistently exceeding nominal levels. This behavior reflects over-dispersed predictive distributions, where uncertainty is inflated to compensate for missing temporal information.

Statistical significance analysis of forecasting performance.

While improvements in accuracy and probabilistic calibration are evident from the reported metrics, it is essential to establish whether these gains are statistically significant and not attributable to random variation across validation folds. To this end, a paired Diebold-Mariano-style statistical significance analysis was conducted using fold-wise loss differentials under the nested cold-start evaluation protocol.

The Diebold-Mariano framework evaluates the null hypothesis that two competing forecasting models exhibit equal expected predictive loss. By operating on paired fold-level errors, this approach explicitly accounts for dependency induced by shared train-test splits and provides a robust basis for comparative inference. Fig 8 presents a global pairwise statistical significance heatmap comparing all evaluated models based on fold-wise loss differentials. Each cell reports the p-value associated with the null hypothesis of equal predictive performance between a pair of models, with darker colors indicating stronger statistical significance.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. Pairwise statistical significance heatmap (p-values) comparing all benchmark models under the cold-start evaluation protocol.

https://doi.org/10.1371/journal.pone.0350729.g008

The heatmap reveals that the proposed ZIG MC framework is statistically superior to all classical single-stage regression baselines, including KNN, BTM-KNN, SVR, Ridge, Lasso, and CatBoost. In nearly all such comparisons, the null hypothesis is rejected with high confidence (p < 0.01), indicating that the observed improvements in MAE, MASE, and related metrics are systematic and reproducible.

Importantly, ZIG MC also exhibits statistically significant improvements relative to two-part statistical baselines, namely Tweedie regression and conventional Zero-Inflated models. The uniformly low p-values in these comparisons demonstrate that the gains achieved by ZIG MC are not merely a consequence of adopting a hurdle-type structure, but arise from the specific integration of learned demand occurrence modeling, Gamma-based magnitude representation, and Monte Carlo distributional inference [55].

The heatmap further reveals clusters of statistical similarity among certain benchmark models. For example, Ridge and Lasso regression exhibit high mutual p-values, reflecting their shared linear structure and similar inductive biases. Such patterns confirm that the statistical testing behaves consistently with known relationships among model classes. To directly address the reviewer’s concern regarding comparison with state-of-the-art probabilistic forecasters, a focused significance analysis was performed between ZIG MC and DeepAR. The results are summarized in Fig 9.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 9. Pairwise statistical significance heatmap comparing ZIG MC and DeepAR.

https://doi.org/10.1371/journal.pone.0350729.g009

The resulting p-value (≈ 0.02) indicates that the null hypothesis of equal predictive performance is rejected at the 95% confidence level. This finding confirms that ZIG MC’s superiority over DeepAR is statistically significant, despite DeepAR’s substantially greater model complexity. The result highlights a key methodological insight: probabilistic deep learning models optimized for sequential learning do not necessarily translate to superior performance under true cold-start conditions characterized by extreme sparsity and zero inflation. The same way, analogous analyses were also conducted for MASE and CRPS. These results, provided in the Supplementary Information, exhibit consistent significance patterns, further reinforcing the robustness of the conclusions across both point and distributional evaluation criteria. Taken together, the statistical significance analyses demonstrate that the performance gains of ZIG MC are not metric-specific artifacts, but reflect a genuine and statistically defensible improvement in forecasting quality.

Scientific and operational implications.

The combined evidence from quartile loss analysis, reliability plots, and pairwise statistical testing establishes that the probabilistic advantages of ZIG MC are structural rather than incidental. The framework produces quantiles that are accurate, calibrated, and statistically distinguishable from those of competing models.

Importantly, these improvements are most pronounced at operationally critical quantiles, directly translating into superior service-level adherence and reduced shortage risk, as demonstrated in the inventory simulation analysis. The results therefore confirm that ZIG MC is not merely a statistically superior forecaster, but a decision-relevant probabilistic model specifically suited for intermittent, zero-inflated spare-parts demand under cold-start conditions.

Probabilistic forecast analysis.

A key advantage of the ZIG MC framework is its ability to generate a full, tailored predictive distribution for each part, rather than a single point estimate. This flexibility demonstrates the true output of the model, which is analyzed for three sample parts from the test set in Fig 11.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 11. Example predictive distributions from the ZIG MC model for three different cold start test parts.

This demonstrates the model’s flexibility in generating unique, tailored forecasts based on part features (a) highly intermittent, (b) Sporadic/Lumpy, (c) More Regular.

https://doi.org/10.1371/journal.pone.0350729.g011

The model demonstrates its ability to learn from the part features and generate appropriate, and unique, demand profiles. In Fig 11a, the model predicts a part that is highly intermittent, with the vast majority of the probability mass at zero and a small tail of low volume positive demand. Fig 11b shows a prediction for a part with more sporadic, lumpy demand; the probability of a zero-demand period is still high, but the model also predicts a wider, more varied distribution of positive sales. Finally, Fig 11c shows a part that the model predicts will have more regular, non-lumpy demand, with a lower probability of zero and a tighter, more normal looking distribution of positive sales.

This demonstrates that the model is not just producing a single, biased average. It is capturing the underlying process and uncertainty of the demand for specific items, a capability that single point forecast models like K Nearest Neighbors lack [57]. This provides a far more powerful tool for inventory management.

Sensitivity to the Gamma shape parameter.

In the proposed ZIG MC framework, the magnitude of non-zero demand is modeled using a Gamma distribution. In the baseline configuration, the shape parameter is fixed to a constant value in order to ensure numerical stability, interpretability, and robustness under true cold-start conditions, where historical demand information is unavailable. Given that this represents a non-trivial modeling assumption, a detailed sensitivity analysis was conducted to verify that the forecasting performance of the framework is not unduly dependent on the specific choice of the Gamma shape parameter.

Accordingly, the Gamma shape parameter was varied across a wide and practically relevant range (), while all other model components including feature representations, learning algorithms, and hyperparameter settings were held fixed. For each value of , the ZIG MC model was re-evaluated using the identical part-level cold-start validation protocol. The resulting mean performance values in terms of MAE, CRPS, and MASE are reported in Table 4.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Sensitivity analysis of the ZIG MC framework with respect to the Gamma distribution shape parameter .

https://doi.org/10.1371/journal.pone.0350729.t004

As shown in Table 4, the point-forecast accuracy metrics (MAE and MASE) remain highly stable across the entire range of Gamma shape parameters. Variations in MAE are confined to a narrow interval, and MASE values consistently remain well below unity, indicating superior performance relative to a naive benchmark regardless of the choice of . This stability demonstrates that the predictive accuracy of the ZIG MC framework is not driven by fine-grained tuning of the Gamma distribution, but instead arises from the structural separation of demand occurrence and demand magnitude inherent to the two-stage formulation.

The stabilization behavior is further illustrated in Fig 12b-c, where both MAE and MASE exhibit only minor oscillations as increases. Notably, no monotonic trend is observed for these point-based metrics, reinforcing the conclusion that the ZIG MC framework is robust with respect to the Gamma shape parameter when evaluated using traditional accuracy measures.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 12. Sensitivity of ZIG MC performance to the Gamma shape parameter on the cold-start test set: (a) CRPS showing rapid improvement for low-to-moderate values of followed by stabilization, (b) MAE, and (c) MASE, both exhibiting minimal variation across the full range of shape parameters.

https://doi.org/10.1371/journal.pone.0350729.g012

In contrast, the probabilistic accuracy metric CRPS exhibits a systematic and interpretable sensitivity pattern, as reported in Table 4 and visualized in Fig 12a. At very low values of (e.g., ), the Gamma distribution is highly skewed, resulting in overly dispersed predictive distributions and elevated CRPS values. As increases from low to moderate values, CRPS decreases sharply, reflecting improved probabilistic calibration and better alignment between the predicted cumulative distribution function and the observed demand outcomes [58].

This improvement continues until approximately , beyond which CRPS enters a clear stabilization regime. For larger values of , further reductions in CRPS are marginal, indicating diminishing returns from additional distributional smoothing. This stabilization behavior, clearly visible in Fig 12, demonstrates that probabilistic forecast quality is robust across a broad and practically reasonable range of Gamma shape parameters.

Importantly, the baseline choice of lies well within this stable operating region, where both point-forecast accuracy and probabilistic calibration are strong. As evidenced by Table 4, this choice yields competitive MAE and MASE values while achieving a low CRPS, providing a balanced trade-off between distributional flexibility and robustness.

Overall, the combined evidence from Table 4 and Fig 12 confirms that the ZIG MC framework is insensitive to the specific choice of the Gamma shape parameter across a wide range. These results validate the use of a fixed shape parameter in cold-start settings and confirm that the reported performance gains are driven by the proposed two-stage probabilistic structure rather than by fragile distributional tuning.

Monte Carlo convergence analysis.

The proposed ZIG MC framework relies on Monte Carlo simulation to construct the full predictive distribution of demand. It is therefore essential to verify that the reported probabilistic and point estimates are not artifacts of sampling noise, but instead represent stable numerical quantities. To this end, a convergence analysis was conducted by examining the stability of the mean demand estimate as a function of the number of Monte Carlo samples , varied over several orders of magnitude from to .

The Fig 13 illustrates the convergence behavior of the mean predicted demand as the number of Monte Carlo simulations increases. At very low simulation counts (), noticeable variability is observed in the estimated mean, reflecting stochastic sampling effects inherent to Monte Carlo methods. In this regime, individual realizations of the Gamma-distributed demand magnitude exert a disproportionate influence on the estimated expectation.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 13. Convergence of the ZIG MC mean demand estimate as a function of the number of Monte Carlo simulations.

https://doi.org/10.1371/journal.pone.0350729.g013

As the number of simulations increases, these fluctuations diminish rapidly, consistent with the Law of Large Numbers. The stabilization trend is quantified in Table 5, which reports the average predicted demand across the cold-start test set for increasing values of . All Monte Carlo simulations were performed using a fixed random seed (seed = 42) to ensure reproducibility across different simulation sizes.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Convergence of the ZIG MC mean prediction as a function of the number of Monte Carlo simulations . Reported values correspond to the average predicted demand across the cold-start test set. Uncertainty is quantified using the standard deviation across repeated Monte Carlo runs, with ±2 standard deviation intervals indicating estimator stability.

https://doi.org/10.1371/journal.pone.0350729.t005

As shown in Table 5, the estimated mean demand increases gradually at low values of and stabilizes beyond approximately . Beyond this point, further increases in the number of simulations yield negligible changes in the estimated mean. At higher simulation counts (), the mean prediction converges fully and remains effectively constant even as the simulation size increases by several orders of magnitude.

This convergence behavior confirms that the Monte Carlo simulation used in the ZIG MC framework produces numerically stable and reproducible estimates of the predictive distribution. Importantly, it demonstrates that the probabilistic results reported in this study are not sensitive to Monte Carlo randomness, and that computational uncertainty has been effectively controlled through an adequate choice of simulation size.

Based on this analysis, a simulation size of was selected for all reported experiments, as it provides a reliable balance between numerical precision and computational efficiency while ensuring stable estimation of both point forecasts and distributional quantities.

Sensitivity of inventory decisions to probabilistic forecasts.

To assess the practical implications of probabilistic robustness, a service-level-based inventory simulation was conducted using the predictive distributions generated by the proposed ZIG MC framework and the probabilistic benchmark DeepAR. Inventory policies were derived by selecting demand quantiles corresponding to target service levels of 80%, 90%, and 95%, consistent with common service-level-driven replenishment practices in spare parts management.

Table 6 reports the resulting operational performance metrics, including fill rate, stock-out rate, average backlog, and average inventory levels for both models on the cold-start test set. To complement the tabular analysis and provide a clearer visualization of trade-offs, Fig 14 illustrates the relationship between target service level and key inventory outcomes.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Inventory performance comparison between ZIG MC and DeepAR across different target service levels, evaluated on the cold-start test set.

https://doi.org/10.1371/journal.pone.0350729.t006

Download:

• PNG
  larger image
• TIFF
  original image

Fig 14. Inventory performance trade-offs across target service levels for ZIG MC and DeepAR on the cold-start test set: (a) average inventory, (b) average backlog, (c) stock-out rate, and (d) fill rate.

ZIG MC consistently achieves higher service performance with lower shortage-related penalties across all service levels.

https://doi.org/10.1371/journal.pone.0350729.g014

As shown in Fig 14a, the average inventory level required to achieve higher service targets increases for both models, as expected. At lower service levels (80%), DeepAR maintains slightly lower average inventory than ZIG MC; however, this apparent inventory efficiency is misleading when examined in conjunction with service and shortage metrics.

The consequences of this under-provisioning are evident in Fig 14b and c. Across all service levels, ZIG MC exhibits substantially lower average backlog and lower stock-out rates than DeepAR. At the 80% service level, DeepAR experiences more than twice the backlog of ZIG MC, indicating frequent underestimation of demand variability. Even as service targets increase to 95%, DeepAR continues to exhibit significantly higher backlog and stock-out rates, reflecting persistent miscalibration of its predictive distribution under cold-start conditions.

In contrast, the ZIG MC framework demonstrates a consistent and controlled reduction in backlog and stock-outs as service levels increase, indicating that its probabilistic forecasts scale appropriately with risk tolerance. This behavior reflects superior calibration of both zero-demand probability and positive-demand tail behavior [59].

The resulting service performance is summarized in Fig 14d, which shows that ZIG MC achieves markedly higher fill rates across all service levels. Notably, ZIG MC attains a fill rate exceeding 95% at the 95% service target, closely matching the intended policy objective. DeepAR, by comparison, fails to achieve comparable fill rates even at higher inventory levels, underscoring the disconnect between its probabilistic outputs and realized demand outcomes in the cold-start setting.

Taken together, the combined evidence from Table 6 and Fig 14a-d demonstrates that improved probabilistic calibration translates directly into superior operational outcomes. While DeepAR may appear competitive in terms of average inventory alone, this comes at the cost of significantly increased stock-outs and backlog. The ZIG MC framework, by contrast, achieves higher service levels with more efficient and reliable inventory utilization, validating its practical value for risk-aware spare parts planning.

Summary of sensitivity and robustness findings.

The comprehensive sensitivity and robustness analysis provides strong empirical evidence that the proposed ZIG MC framework is stable, reliable, and well-suited for cold-start spare part demand forecasting.

First, the sensitivity analysis with respect to the Gamma shape parameter demonstrates that the forecasting performance of ZIG MC is largely insensitive to distributional parameter selection. As shown in Table 4 and Fig 12, point-forecast accuracy metrics (MAE and MASE) remain highly stable across a wide range of shape parameters, while probabilistic accuracy (CRPS) improves rapidly from highly skewed distributions and stabilizes over a broad operating region. Importantly, the baseline choice of lies well within this stable region, confirming that the reported performance gains do not rely on delicate distributional tuning but instead stem from the two-stage modeling structure itself.

Second, the Monte Carlo convergence analysis confirms the numerical reliability of the probabilistic estimation procedure. As demonstrated in Table 5 and Fig 13, the estimated mean demand converges rapidly with increasing simulation size and stabilizes beyond approximately . Beyond this point, further increases in the number of simulations yield negligible changes in the estimated quantities, indicating that the reported probabilistic and point forecasts are not artifacts of sampling noise. This validates the choice of simulation size and confirms that computational uncertainty has been effectively controlled.

Third, the inventory simulation study establishes that improvements in probabilistic calibration translate directly into superior operational decisions. As shown in Table 6 and Fig 14, ZIG MC consistently achieves higher fill rates, lower stock-out rates, and reduced backlog across all service-level targets when compared with the probabilistic benchmark DeepAR. While DeepAR occasionally appears competitive in terms of average inventory alone, this comes at the cost of significantly degraded service performance, highlighting the risks of relying on poorly calibrated probabilistic forecasts in cold-start settings.

Taken together, the expanded evaluation provides consistent evidence that the performance improvements of the proposed ZIG MC framework are not limited to a single metric or experimental configuration. The model demonstrates superior performance across point accuracy measures (MAE, WMAPE, MASE), probabilistic scoring rules (CRPS), calibration diagnostics, and inventory-oriented decision metrics. Furthermore, statistical significance testing and sensitivity analyses confirm that these gains are robust to data partitioning, distributional assumptions, and Monte Carlo sampling variability. These results collectively substantiate that the proposed framework offers a structurally more appropriate representation of intermittent, zero-inflated demand under cold-start conditions, thereby supporting the conclusions drawn in this study.