3.1 Forecasting the predictors

We adopt a unified procedure that extracts forward-looking information from each predictor by producing a one-step-ahead conditional mean forecast and its one-step-ahead conditional residual variance as a measure of forecast uncertainty. We compute the variance to quantify the state-dependent precision of the mean forecast and carry these quantities forward to the next stage; how they are used is described later.

Let denote the K predictor series. For each predictor, we estimate a flexible ARIMAX () model, proposed by [52]:

where is the differencing operator of order d_k; and are the lag polynomials for the autoregressive (AR) and moving average (MA) components, respectively; is the intercept; is a vector of exogenous variables with coefficient vector , and is the residual term. This provides the one-step-ahead forecast, .

To quantify the time-varying forecast uncertainty, we model the conditional variance of the ARIMAX residuals using a GARCH () process:

where is the conditional variance and is its one-step-ahead forecast implied by the recursion. The pair constitutes the forward-looking signals passed to the next stage.

Predictors differ in persistence, seasonality, and noise, so bespoke models might lift fit for a few series. Our aim, however, is not to maximize first-stage fit but to produce comparable reliability scores in a consistent and fair way. Using a unified ARIMAX–GARCH modeling framework across all series ensures that cross-predictor differences in out-of-sample R² reflect signal quality rather than model flexibility or researcher discretion.

3.2 Feature construction and dimension reduction

Stage 2 maps the predictor information available at time t into a feature vector X_t and uses it to forecast the next period equity premium r_t+1. The key input to this stage is the collection of predictors described in Section 4.1, together with the forward-looking quantities generated in Stage 1. In what follows, we define the candidate feature pools and the optional processing steps—screening and dimension reduction—that we use to obtain parsimonious inputs for learning.

We begin with two base feature pools. The Past pool uses lagged predictor information only and is given by

The Combined pool augments each lagged predictor with a forward-looking pair produced in Stage 1. Specifically, for each predictor k, Stage 1 provides a one-step-ahead conditional mean forecast and a one-step-ahead conditional residual volatility proxy . We collect these into a predictor-level feature triplet

and stack them across predictors to form

The Combined pool is designed to encode not only the current level of each predictor but also its anticipated movement and the uncertainty around that movement, allowing the model to exploit forward-looking information without introducing additional ad hoc state variables.

Economically, this augmentation is motivated by the idea that the current level of a predictor need not fully summarize the state relevant for next-period equity premia. Two periods with similar z_k,t can imply different required returns if the same predictor is expected to deteriorate further in one case but to stabilize or reverse in the other. The conditional mean forecast therefore provides a parsimonious summary of the near-term direction in which the macro-financial state is likely to move, while captures how precisely that transition can be read at the forecast origin. In this sense, the uncertainty term is useful not only as a reliability measure for the generated mean forecast, but also as an indicator of local fragility or regime instability, under which the mapping from predictors to expected returns may differ. The Combined pool is thus intended to capture both where the economy currently is and where it is likely headed. This economic interpretation also motivates the admission rule introduced below: when the first-stage forward-looking signals are weakly forecastable, adding them is more likely to inject noise than to sharpen the return forecast.

Because the forward-looking components are generated forecasts, their quality can vary substantially across predictors. To prevent noisy first stage forecasts from diluting the signal, we impose a predictor-level admission threshold when forming the Combined pool. Let denote the out-of-sample predictive fit of the Stage 1 model for predictor k, computed using the benchmark definitions in Section 3.3. If , we drop only the forward-looking pair from predictor k while always retaining the lagged level z_k,t. This rule keeps the baseline information set intact and uses Stage 1 only when it delivers sufficiently reliable forward-looking content. The resulting post-threshold working pool therefore consists of all lagged predictor-levels together with only the admitted forward-looking columns.

To manage redundancy and improve parsimony, we optionally apply screening and/or dimension reduction to the base pools. First, we consider SHAP-based screening: we fit a preliminary model on the training data, compute feature attributions via TreeSHAP, and rank predictors by mean absolute SHAP importance. Implementation details are provided in Section 4.2.2. We then retain only the top N predictors and carry forward the corresponding features from the chosen base pool. Let denote the resulting SHAP-screened feature vector.

Second, we apply linear dimension reduction to the working pool using either PCA or PLS. Let F denote the feature matrix over the training window formed from the chosen working pool (either directly from or , or from the SHAP-screened ), and let r denote the corresponding vector of target equity premia. PCA constructs orthogonal components that maximize the variance of the projected features by selecting loading vectors that solve

PLS instead targets predictive directions by maximizing squared covariance with the equity premium, choosing according to

In both cases, the constraints normalize the loadings and ensure that the extracted score vectors are mutually uncorrelated across components. Let denote the resulting m-dimensional representation at time t, constructed by projecting the feature vector from the working pool onto the selected directions.

Overall, depending on the specification under study, the Stage 2 input is taken from

Given the input representation X_t, Stage 2 then fits a forecasting function and produces the one-step-ahead equity premium forecast

3.3 Performance evaluation

Our primary measure of forecast accuracy is the out-of-sample R² () in the sense of [3]. Let r_t+1 denote the target at time t + 1 and the corresponding one-step-ahead forecast formed using information available at time t. We define

where is the historical average return estimated through period t, and denotes the out-of-sample test set disjoint from training and validation. Positive values indicate that the predictive model reduces mean squared error relative to the historical mean benchmark.

In addition, when evaluating one-step-ahead forecasts for highly persistent series as in our Stage 1 predictor forecasting, it is often more conservative to benchmark against a random-walk forecast. In that case, the out-of-sample statistic is computed as

so that performance is assessed relative to the no change prediction r_t+1 = r_t. Throughout this paper, we use the historical mean benchmark for equity premium forecasting and for return or growth type targets, and the random walk benchmark for most persistent predictor-level targets in Stage 1. An exception is monthly inflation, for which predictor-level out-of-sample reliability is evaluated relative to a trailing 12-month average formed recursively from the most recent available observations and lagged by one period to preserve real-time timing. We adopt this benchmark because monthly inflation is persistent but noisy at short horizons, so a recent-year average provides a more meaningful and less noise-sensitive real-time baseline than a one-month no-change forecast.

Additionally, we also report the relative root mean squared error (RRMSE) as a supplementary scale-based measure of forecast accuracy. Let denote the relevant benchmark forecast. We define

Values below one indicate that the model outperforms the benchmark in root mean squared forecast error, while values above one indicate worse forecast accuracy. The benchmark used in RRMSE follows the same rule as for : The historical mean for equity premium forecasting and return or growth-type targets, and the random-walk forecast for highly persistent predictor-level targets.

To connect accuracy to market states, we also compute tail-conditional out-of-sample R². For a quantile level , let

and define the downside and upside index sets

The corresponding conditional R² statistics are

Both metrics evaluate the reduction in squared forecast errors within either left-tail (downside) or right-tail (upside) months of the test period, relative to the same benchmark as above. Economically, a positive indicates improved accuracy precisely in adverse market states–relevant for downside risk management–whereas a positive reflects better accuracy in favorable states and stronger upside participation.

4.1 Data

Our sample spans January 1952 to December 2024. The equity premium is measured as the monthly market return minus the risk-free rate. Primary results are reported for the S&P 500, with robustness checks on the CRSP value-weighted index. Our baseline follows the 17 predictors of [1], augmented with five variables highlighted as promising in more recent work [2]: a composite index of technical indicators (tchi), tail risk (tail), average stock correlation (avgcor), the output gap (ogap), and growth in personal consumption expenditures (gpce). A key inclusion criterion is the availability of a continuous series back to at least 1952, ensuring a sufficiently long sample for robust out-of-sample evaluation.

To support economic interpretation and to stabilize later attribution exercises, we organize the predictors into six families following [1,2]—Valuation; Rates, Term Structure & Credit; Macroeconomic; Equity Issuance & Financing; Market Risk & Comovement; and Technical. This classification is also the one used later when we aggregate importance measures at the group-level.

Because our two stage design constructs forward-looking signals at the predictor-level, all subsequent steps rely on a real-time predictor panel that respects publication lags and appropriately aligns mixed frequencies. We describe these preprocessing and alignment rules—together with the Stage 1 forecasting procedure built on top of them—in Section 4.2.1.

4.2 Implementation details

This section describes the empirical implementation of our framework in a real-time setting. Building on the data described in Section 4.1, we summarize the preprocessing required to avoid look-ahead, the construction of the feature sets used in the forecasting experiments, and the training and evaluation protocol adopted throughout.

4.2.1 Stage 1: Preprocessing and forecasting each predictor.

Data preprocessing and mixed-frequency alignment

Stage 1 is implemented on a real-time predictor panel that respects publication lags and aligns mixed frequency series without look-ahead. We first shift each predictor by a fixed release lag based on its native frequency—one month for monthly series, three months for quarterly series, and one year for annual series—before any further transformation or standardization. Because each predictor is forecast in turn using the remaining variables as candidate exogenous regressors, frequency alignment is performed relative to the target variable in each forecasting model: regressors observed at the same frequency enter under the same lagged timing; when the target is higher frequency, lower frequency regressors are carried forward using their most recently released observation until the next release; and when the target is lower frequency (e.g., quarterly or annual), higher frequency regressors are aggregated to the target periodicity as described below.

Concretely, the aggregation step follows standard stock-flow conventions so that the mapped regressor preserves the variable’s economic interpretation. In particular, variables naturally interpreted as end of period state quantities—such as interest rate levels and spreads (e.g., lty, tbl, tms, dfy) and valuation ratios (e.g., d/p, d/y, e/p, d/e, b/m), as well as indices such as tchi and ogap—are mapped to the period-end value. Variables representing within-period conditions such as average states or risk—such as inflation and market risk/comovement measures (e.g., infl, svar, avgcor, tail)—are mapped to the period average. Cumulative flow series are mapped to the period sum (e.g., ntis), and return type series are mapped to period returns via geometric compounding (e.g., dfr, ltr).

To limit obvious redundancy in Stage 1, we did not impose a blanket exclusion rule at the level of the six broad predictor families in Table 1. Instead, the candidate exogenous set for each target predictor was restricted only in a limited, pair-specific way, excluding variables that were regarded as mechanically overlapping or as near-equivalent representations of the same underlying quantity. This mainly applied to closely related measures such as alternative valuation ratios or tightly linked rate/spread variables. Accordingly, the family classification in Table 1 should be interpreted as descriptive and used for economic organization, rather than as the formal criterion determining admissible exogenous regressors in Stage 1. The empirical implementation uses a single admissible exogenous regressor for each target predictor; Table 2 reports the selected regressor for each series.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Predictor families, variable names, and native frequencies.

https://doi.org/10.1371/journal.pone.0341578.t001

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. ARIMAX specifications and in-/out-of-sample R² for each target variable. For each series we report the selected exogenous predictor, the optimal ARIMAX order (p^m, d, q^m) – where p^m denotes the number of autoregressive lags, d the order of differencing, and q^m the number of moving-average lags – and the resulting in-sample () and out-of-sample () values.

https://doi.org/10.1371/journal.pone.0341578.t002

Forecasting model and expanding window refits

The objective of Stage 1 is to construct, for each predictor z_k, a forward-looking pair consisting of the one-step-ahead conditional mean and the conditional residual volatility for its forecast uncertainty. We implement this by estimating an ARIMAX model for the conditional mean and a GARCH model for the residual variance on the real-time dataset described above, maintaining the exclusion map throughout. The procedure is run on an expanding window with annual refits. We reserve January 1952 through December 1971 as an initial training period and then produce the first one-step-ahead forecasts for 1972, repredicting the models each year using the expanding sample available at that point.

To stabilize the mean component, the ARIMAX orders (p^m, d, q^m) are selected once at the initial calibration using the Akaike information criterion (AIC) from a compact grid—p^m and q^m ranging from 0 to 2, and d equal to 0 or 1—and then held fixed. In contrast, the GARCH orders (p^v, q^v) are reselected via AIC at each annual refit, allowing the volatility dynamics to adapt over time; for this, the GARCH order p^v is set to 1 or 2, and the ARCH order q^v ranges from 0 to 2. This unified template is intentionally parsimonious: our aim is not to maximize first stage fit for any single predictor, but to generate comparable one-step-ahead forecasts across series so that cross-predictor differences primarily reflect signal quality rather than model flexibility.

From the resulting expanding window forecasts, we compute a predictor-level out-of-sample as a reliability score, using the benchmark definitions in Section 3.3. These scores are subsequently used to screen the forward-looking components when constructing the Combined feature set.

5.1 Stage 1 forecastability and reliability screening

Before turning to equity premium forecasting results, we first summarize how forecastable the underlying macro–financial predictors are in real time, since this stage determines which forward-looking signals are admitted into the combined feature pool. In Stage 1, we fit a parsimonious ARIMAX specification with a single admissible exogenous regressor for each predictor z_k to generate one-step-ahead conditional mean forecasts , and pair them with a one-step-ahead residual uncertainty proxy ; for each target series, we identify the single best-performing exogenous regressor through the cross-prediction procedure described in Section 4.2.1.

Table 2 reports, for each predictor, the selected exogenous variable, the ARIMAX order (p^m, d, q^m) determined by the AIC, and the resulting in-sample and out-of-sample fit statistics, where out-of-sample performance is evaluated using the benchmark rules in Section 3.3. Supplementary assessment of the baseline Stage 1 ARIMAX–GARCH specification is reported in S1 Appendix.

A clear dispersion emerges across predictors. Several series—such as investment i/k, payout d/e, and market-wide risk/comovement measures including svar and avgcor—exhibit materially positive out-of-sample performance in this first stage exercise, whereas selected valuation, issuance, and macroeconomic variables such as b/m, ntis, and gpce display little to no incremental predictive content, with near zero or negative; rate, term-structure, and credit variables show mixed results.

We carry these predictor-level values forward as reliability scores that discipline the construction of the combined feature set in Stage 2. Concretely, when forming the combined pool, we apply the threshold to each predictor’s Stage 1 reliability and drop only its forward-looking pair when the reliability falls below , while always retaining the contemporaneous lag z_k,t. This design makes the role of forward-looking information transparent: it is included only when the underlying predictor demonstrates adequate real-time forecastability, and it allows the subsequent empirical analysis to isolate the incremental value of admitting these generated signals relative to using lagged predictors alone.

5.2 Main Results and Portfolio Performance

This section reports our main Stage 2 equity premium forecasting results and evaluates their economic value. We summarize statistical accuracy using out-of-sample R² measures and translate each one-step-ahead forecast into a constrained mean-variance portfolio to assess realized performance.

Following [3], at each time t, the portfolio weight allocated to the risky asset is determined by:

where is a backward-looking variance estimate computed from a rolling 5-year window, is the risk aversion coefficient, and and impose no short selling and leverage constraints. Robustness to alternative volatility estimators used in this portfolio-scaling rule is reported in S4 Appendix. Portfolio performance is reported on an annualized basis. Let and denote the annualized mean excess return and volatility of the portfolio. We report the Sharpe ratio , and the certainty equivalent return

We also report maximum drawdown

where W_t is the cumulative wealth. To emphasize downside risk, we compute the Sortino ratio with

and we quantify trading intensity by annualized turnover

Throughout Tables 3–5, “Past” denotes models that use lagged predictors only, whereas “Comb.” denotes the combined feature set that augments these lags with Stage 1 forward-looking signals admitted under the predictor-level reliability threshold . “SHAP-PCA/PLS” indicates that predictors are first pre-screened by SHAP importance and then reduced by PCA or PLS before forecasting; is the minimum individual Stage 1 out-of-sample R² required for a predictor’s forward-looking forecast to enter the combined set. We first assess the economic value of the forecasts via portfolio outcomes and then examine statistical predictive accuracy.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Portfolio performance by feature representation and estimation method. The table reports Sharpe ratio, Sortino ratio, certainty equivalent return (CER) with risk aversion , maximum drawdown (MDD), and turnover for various forecasting strategies. The reliability threshold refers to the minimum individual predictor out-of-sample R² from Stage 1 required for its forward-looking forecast to be included in the combined feature set.

https://doi.org/10.1371/journal.pone.0341578.t003

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Net of transaction cost portfolio performance by feature representation and estimation method. All entries are computed after deducting proportional transaction costs of 25 basis points per unit of turnover. The table reports Sharpe ratio, Sortino ratio, certainty equivalent return (CER) with risk aversion , maximum drawdown (MDD), and turnover for various forecasting strategies. The combined feature set admits forward-looking signals only for predictors whose Stage 1 out-of-sample R² exceeds the reliability threshold .

https://doi.org/10.1371/journal.pone.0341578.t004

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Out-of-sample R² for the equity risk premium forecasts by feature representation, estimation method, and predictor level threshold . The table reports the in-sample R² and three out-of-sample R² measures relative to the historical mean benchmark. The threshold refers to the minimum individual predictor out-of-sample R² from Stage 1 required for its forward-looking forecast to be included in the combined feature set. and denote out-of-sample R² in downside (left-tail) and upside (right-tail) months, as defined in Section 3.3.

https://doi.org/10.1371/journal.pone.0341578.t005

Table 3 provides a direct economic lens on the Stage 2 forecasts by comparing the implied mean-variance portfolios to standard benchmarks. A simple buy-and-hold allocation delivers a Sharpe ratio of 0.46 with an MDD of 0.50, while the conditional CAPM and FF3 benchmark strategies exhibit notably lower risk-adjusted performance and comparable or larger drawdowns.

Our primary focus is the incremental benefit of augmenting lagged predictors with the admitted forward-looking signals relative to using lagged predictors alone. The evidence in Table 3 suggests that such gains are possible, but not uniform across representations or threshold choices. In the raw feature block, a moderate positive threshold performs better than both the past-only baseline and the weakest screen: at , the combined specification raises the Sharpe ratio from 0.2719 to 0.3507 and lowers MDD from 0.5909 to 0.5469 relative to Past, although it still remains below buy-and-hold.

PCA offers a weaker investment profile overall: Combined variants are sometimes mildly better than Past in Sharpe or CER, but drawdowns remain high and the gains are not consistent across . By contrast, PLS-based representations benefit more from selective admission. The strongest gross PLS combined case occurs at , where Sharpe, Sortino, and CER all exceed the past-only PLS specification and MDD falls from 0.6561 to 0.4595.

SHAP screening sharpens this pattern. SHAP-PCA combined at improves both Sharpe and downside protection relative to its Past counterpart, while SHAP-PLS delivers the strongest gross performance in the table. The specification reaches a high performance overall, and the and variants also remain stronger than the past-only SHAP-PLS baseline on risk-adjusted return measures. Overall, Table 3 suggests that forward-looking augmentation can yield economically meaningful gains, but mainly in selected representations and thresholds rather than uniformly across all combined designs.

Table 4 complements this picture by evaluating net of transaction cost portfolio performance at the representative interior threshold after deducting proportional transaction costs. The raw combined specification still improves on raw Past in Sharpe, CER, and MDD, whereas PCA remains essentially unchanged. For PLS, the combined design continues to reduce drawdown materially but no longer improves Sharpe or CER relative to PLS Past. SHAP-PCA combined remains competitive, improving on its Past counterpart in Sharpe, Sortino, CER, and MDD, and SHAP-PLS combined remains the strongest net of transaction cost combined specification, with a Sharpe ratio of 0.5653 and a CER of 0.0530, while also reducing turnover relative to past-only SHAP-PLS. S4 Appendix additionally reports the corresponding net of transaction cost results over the full -grid, as well as sensitivity checks under 10 and 50 basis point transaction cost assumptions.

From a practitioner’s perspective, these results suggest that the additional modeling complexity is justified only when it survives a joint screen based on risk-adjusted return, drawdown control, and trading intensity after costs. Under that lens, the most compelling specification in the main text is SHAP-PLS at , not because the two-stage framework uniformly dominates simpler strategies, but because this specification remains competitive after costs on the metrics practitioners are most likely to care about. By contrast, other combined designs appear more mixed: some improve drawdown without clearly improving Sharpe, while others show only modest net gains once turnover is taken into account. The practical value of the framework should therefore be viewed as selective rather than universal, with the strongest case arising when forward-looking signals are paired with disciplined admission and supervised screening.

A well-known intuition from [3] is that even small gains in out-of-sample R² can deliver large economic value for a mean-variance investor. By contrast, [53] emphasize that minimizing a forecasting loss need not maximize a portfolio objective, so higher out-of-sample R² need not translate into a higher Sharpe ratio. Our evidence reflects both sides of this wedge: some specifications that rank favorably by aggregate do not deliver commensurate gains in realized portfolio performance, while some economically attractive variants show only modest aggregate improvements in statistical fit. With this distinction in mind, Table 5 summarizes the statistical forecasting evidence more directly.

Table 5 summarizes predictive performance in Stage 2 across feature families, dimension-reduction choices, and the predictor-level admission threshold . It reports in-sample R² (), aggregate out-of-sample R² relative to the historical mean benchmark (), the tail-conditional measures and , and RRMSE. A first takeaway is that forward-looking signals are not free: under the weakest admissible screen (), several combined specifications remain weak or negative in aggregate out-of-sample fit. A moderate positive threshold can stabilize performance in some blocks, but not mechanically. In the non-SHAP PCA block, aggregate moves from −0.0015 under Past to 0.0015 at and 0.0024 at under the combined feature set. In the PLS block, Past yields 0.0021, while the combined set improves to 0.0062, 0.0040, and 0.0039 at , 0.10, and 0.15, respectively, before turning negative again at . Thus should again be interpreted as a selectivity device that trades off signal quality against information retention, rather than as a monotone tuning parameter.

Second, SHAP screening yields the strongest and most consistent performance. SHAP-PCA Past already produces positive aggregate (0.0041), and all of its combined variants remain positive, with the strongest aggregate fit at (0.0157) and RRMSE as low as 0.9921. SHAP-PLS performs even better overall: The past-only version reaches 0.0153, while the combined set peaks at 0.0212 for and 0.0163 for , with the specification also delivering the lowest RRMSE in the table (0.9893). These patterns are consistent with the view that target-aligned SHAP screening removes weak or redundant inputs before the representation step.

Finally, the tail-conditional diagnostics show that predictive gains are state dependent. PCA and SHAP-PCA combined specifications tend to lean more toward upside accuracy, as reflected in positive alongside negative at several thresholds. By contrast, combined PLS specifications often load more heavily on downside months: for example, PLS with combined feature set at yields but . SHAP-PLS is more mixed across thresholds, with positive in both tails but strongly skewed toward downside accuracy (, ). These state-conditional patterns help explain why aggregate forecast fit and portfolio outcomes need not move one-for-one, especially for path-dependent objects such as maximum drawdown.

5.3 Additional validation and robustness checks

5.3.1 Forecast significance tests.

Table 6 complements the -based comparisons with formal forecast significance tests. We use the one-sided Diebold–Mariano (DM) test as our primary loss based test throughout. For comparisons against the historical mean benchmark, we additionally report the Clark–West (CW) statistic. The reason is that the historical mean is a parsimonious benchmark, whereas our competing specifications are richer conditional forecasting models estimated recursively from larger feature sets. In such settings, the larger model can be penalized in raw MSPE comparisons by estimation noise even when it contains incremental predictive information, which is precisely the case for which the CW adjustment was developed [54].

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Diebold–Mariano (DM) and Clark–West (CW) test statistics for one-step-ahead equity risk premium forecasts. The left block reports one-sided benchmark relative test statistics against the historical mean forecast. The Pairwise block reports one-sided DM test statistics comparing the Combined specification with the matched Past specification within the same method and threshold. denotes the difference in out-of-sample R² between the Combined and Past specifications. Superscripts *, **, and *** denote statistical significance at the 10%, 5%, and 1% levels, respectively, based on one-sided tests.

https://doi.org/10.1371/journal.pone.0341578.t006

Accordingly, the left block of Table 6 reports benchmark relative DM and CW test statistics, where larger positive values indicate stronger evidence that the reported specification improves upon the historical mean benchmark. The right block reports the paired comparison between the Combined and matched Past specifications within each method and threshold; there, , and positive values of both and the DM statistic favor the Combined specification.

The benchmark-relative results in the left block of Table 6 show that the DM evidence is selective rather than broad-based. The clearest joint support from both DM and CW appears in the SHAP-screened specifications, most notably for SHAP-PCA with and for the SHAP-PLS Past specification. More generally, benchmark-relative support is visibly stronger under CW than under DM. This pattern is most apparent for the PLS and SHAP-PLS families, where several Combined specifications receive positive CW support even when the corresponding DM statistic does not reach conventional significance. We interpret this asymmetry cautiously: It suggests that some richer conditional specifications contain incremental information relative to the historical mean benchmark, while the raw loss-differential evidence remains modest.

The right block of Table 6 asks a stricter question: whether augmenting past predictors with admitted forward-looking signals improves upon the matched Past specification within the same representation. Here the evidence is more limited. The clearest result is the unreduced specification at , where the Combined design delivers a positive together with a statistically significant DM statistic. PCA at and also yields positive values and positive DM statistics, but these remain below conventional significance thresholds. For PLS, SHAP-PCA, and SHAP-PLS, the incremental gains are either small or unstable across thresholds. Taken together, the pairwise results indicate that the incremental contribution of forward-looking augmentation is selective rather than pervasive.

This pattern is consistent with the broader literature on aggregate equity premium prediction. A long line of work emphasizes that out-of-sample market return predictability is difficult to establish reliably, and that when predictability is present it is often weak and unstable in economic magnitude and statistical significance [1–3,20,55]. In this context, the limited pairwise DM significance in Table 6 should not be interpreted as evidence against the usefulness of the approach. Rather, it underscores the difficulty of extracting robust real-time signals from a single noisy aggregate return series. Our contribution is therefore not that forward-looking augmentation uniformly dominates past-only information, but that under disciplined admission and representation choices it can isolate weak but economically relevant predictive content, especially in specifications that also perform well in the downside-sensitive portfolio exercises. This interpretation is also in line with the view that poor out-of-sample R² does not by itself rule out return predictability, and that even modest predictive improvements can matter economically [3,56].

5.3.2 Sigma ablation study.

To make the role of the Stage 1 uncertainty channel explicit, Table 7 reports a direct sigma ablation for the combined feature representation at the representative interior admission threshold . The table compares each specification with and without the uncertainty proxy , and reports the difference in performance between the specification with and the corresponding specification without . We focus on in the main text to keep the ablation compact and directly interpretable, while the full -grid of the sigma ablation study is reported in S5 Appendix.

Download:

• PNG
  larger image
• TIFF
  original image

Table 7. Sigma ablation in Stage 2 for the combined features set at . Throughout the table, denotes the difference in performance between the specification that includes the Stage 1 uncertainty proxy and the corresponding specification that excludes it. Panel A reports out-of-sample prediction metrics, and Panel B reports gross portfolio performance. Positive values of , , , , Sharpe, Sortino, and CER indicate better performance with , whereas negative values of RRMSE and MDD indicate improvement because forecast error and maximum drawdown are reduced.

https://doi.org/10.1371/journal.pone.0341578.t007

Panel A shows that the contribution of is not uniform across representations, but it is clearly not redundant. The cleanest comparison is the non-SHAP block, since it isolates the incremental role of the uncertainty proxy without additional screening effects. In the raw combined specification, adding sigma raises and , lowers , and modestly improves RRMSE. Thus, even in the simplest non-SHAP specification, the uncertainty proxy improves forecasting performance primarily in adverse market states rather than uniformly across the test sample.

The PCA block, by contrast, is essentially unchanged. All prediction metrics remain close to zero, indicating that once the combined signal is compressed along variance-maximizing directions, the uncertainty channel adds little incremental information. This stands in clear contrast to the PLS block, which delivers the strongest forecasting gains without SHAP screening. There, sigma inclusion increases , , and , but reduces while improving RRMSE. The asymmetry between and suggests that helps the second-stage learner identify states in which predictor-level forecasts are less precise and the mapping into the equity premium becomes more fragile or more nonlinear. In that sense, the uncertainty proxy functions as a state dependent reliability signal, which is precisely the role envisioned in Stage 1.

The SHAP-screened specifications reinforce this interpretation, but we treat them as complementary rather than primary evidence. Both SHAP-PCA and SHAP-PLS show positive changes in , and in both cases the improvement is again more pronounced on the downside than on the upside. These results are consistent with the non-SHAP evidence, but they should be interpreted with some caution because sigma inclusion may interact with screening and feature selection.

Panel B shows that the economic implications broadly mirror the statistical evidence. In the raw specification, sigma inclusion produces modest but consistently positive improvements in Sharpe, Sortino, and CER, while also reducing maximum drawdown. The PCA block again remains economically negligible. The strongest non-SHAP portfolio gains arise in the PLS block, where adding sigma improves the Sharpe ratio, the Sortino ratio, CER, and maximum drawdown, although turnover increases. Thus, in the representation where the gains are most visible, those gains also translate into a more attractive investment profile.

At the same time, the turnover results do not move in a single direction across methods. In particular, the PLS specification exhibits a higher turnover when sigma is included, whereas the raw and PCA specifications do not. This is useful for interpretation: The portfolio gains associated with are not simply the result of mechanically smoother portfolio weights or lower trading activity. Rather, they reflect an informational improvement in the signal entering the portfolio decision.

The SHAP-based portfolio results are again stronger in magnitude, with sizable gains in Sharpe, Sortino, CER, and drawdown control for both SHAP-PCA and SHAP-PLS. However, as in Panel A, we interpret these as robustness evidence rather than as the cleanest sigma comparison. The main takeaway from Table 7 is therefore not that sigma is universally helpful across all feature-processing choices. Instead, the evidence indicates that the Stage 1 uncertainty proxy has a distinct and economically meaningful role: its contribution is concentrated in downside-state forecasting, and it becomes most valuable when the combined signal is represented in a disciplined supervised low-dimensional form, especially PLS.

This interpretation is broadly in line with [41], who show that international economic policy uncertainty measures contain out-of-sample predictive information for U.S. excess returns, suggesting more generally that uncertainty related signals can be most useful when they are incorporated through a disciplined summary representation.

5.4 Interpretability: What drives predictability?

In this section, we study which economic channels are most closely associated with equity premium predictability over time by summarizing predictor importance at the group-level. Using the six predictor families defined in Section 4.1 (see Table 1 for the complete mapping), we compute mean absolute TreeSHAP attributions on the validation set and aggregate them within each family to obtain a time-varying measure of group-level importance. This aggregation helps mitigate multicollinearity induced attribution dilution among closely related predictors and yields a more stable, interpretable diagnostic of the channels emphasized by the forecasting model.

This group-level diagnostic also helps rationalize why SHAP-based screening improves out-of-sample performance. Because screening ranks predictors by their validation-based TreeSHAP contributions, it prioritizes variables that the fitted model relies on most when producing real-time forecasts, while de-emphasizing redundant signals within the same economic family. Consequently, the downstream forecasting stage is effectively guided toward the channels that carry predictive content at the forecast origin, which is consistent with the performance improvements documented in Tables 3–5.

Several broad patterns emerge. During the dot-com boom–bust (2000–2003), valuation signals are especially prominent, while issuance and financing variables are more visible than in later years, which is consistent with work linking valuation ratios and issuance waves to subsequent return reversals [57,58]. In the mid-2000s expansion, macroeconomic information gains relative importance, particularly around consumption–wealth conditions, while rates and credit variables remain an important backbone [18,59]. During the global financial crisis (2007–2009), the emphasis shifts more clearly toward rates/credit and market risk/comovement, in line with evidence on credit spreads, intermediary balance sheets, and tighter funding conditions [60,61]. Through much of the 2010s, market risk/comovement remains persistently important, and technical signals become more visible toward the late 2010s, which is consistent with the role of broad financial-cycle conditions and trend-sensitive information under prolonged monetary accommodation [21,22,62]. The 2020 pandemic shock again raises the relative importance of market risk/comovement together with rates/credit [63,64], while the 2022–2024 inflation-tightening period is characterized by renewed macroeconomic and rates/credit importance, with market risk/comovement remaining elevated rather than disappearing [65].

This group-level view also helps make the role of SHAP-based screening easier to understand. The screening step is meant to keep the model focused on the predictors that are most informative at the forecast origin, while avoiding an unnecessarily wide set of nearby substitutes that often carry overlapping information. Read in that way, Fig 1 is not saying that one family causes the equity premium in a structural sense. Rather, it shows which kinds of signals the fitted forecasting system is leaning on most heavily in different periods. That is also why the family-level summary is useful: Even when the importance of individual variables moves around within a family, the broader economic channel can still remain visible in an interpretable way. Additionally, S6 Appendix examines, for a representative SHAP-PLS specification, how the composition of the annually selected SHAP-ranked predictor bundles evolves over time.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Time-varying group-level importance of equity premium predictor families.

Each bubble corresponds to one of the six predictor families defined in Table 1 in a given calendar year. For each year, group-level importance is computed from mean absolute TreeSHAP values on the validation set and then aggregated within family. Bubble area reflects that family’s share of total annual group-level importance across the six families, so larger bubbles indicate that the fitted forecasting model relied more heavily on that family in that year.

https://doi.org/10.1371/journal.pone.0341578.g001