acceptodds
Under review as a conference paper at ICLR 2027

Learning from Zeros: Regime Aware Sparse Forecasting

Abstract

Sparse time series, where a large fraction of observations are exactly zero, are common in retail and supply chain forecasting. Conventional methods for this multi-horizon panel setting address only one facet of the challenge at a time, such as smoothing zeros away, decomposing occurrence and magnitude, or routing series by a fixed sparsity threshold. We argue that the temporal arrangement of zeros carries information about an underlying state and should be modeled as a continuous latent regime shared across the forecast horizon. We show theoretically that marginal objectives cannot identify how sparse events are jointly organized across a forecast window, that a shared latent regime can induce this dependence, and that the independent per-horizon regimes underlying common architectures cannot express it regardless of training. We propose a regime-aware framework built on this insight, combining a sparsity-aware encoder with a context-conditioned latent regime to produce a joint predictive distribution over Negative Binomial outputs. Across a large-scale proprietary e-commerce dataset and three public benchmarks, our method achieves consistent improvements over existing approaches, with joint-level calibration metrics confirming our theoretical predictions.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, probabilistic forecasting, sparse demand forecasting, epistemic uncertainty

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_z9p2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-horizon probabilistic forecasting for zero-inflated count series. To capture persistent runs of zeros (e.g., stockouts), the authors share a single context-conditioned latent variable zz across all forecast horizons to shift the Negative Binomial parameters, and train with the joint trajectory likelihood. They show theoretically that shared latents can represent cross-horizon dependence while independent per-horizon latents cannot. Experiments report quantile WAPE on one proprietary dataset and three public benchmarks, with joint calibration mostly checked on synthetic data.

Critical strengths

The motivation is practical and the method is easy to deploy. Standard multi-horizon heads treat zero occurrence independently at each step, whereas sparse demand often comes in runs of zeros. Sharing one latent variable across horizons is a simple way to couple the trajectory. The experiments are also broad, covering proprietary data, three public benchmarks, a sparse-demand baseline (SPADE-S), and a fine-tuned Chronos2 model.

Critical weaknesses

The link between zero patterns and a latent "regime" is not actually tested. The paper claims that zz captures a latent state from the arrangement of zeros. But zz is just a context-conditioned Gaussian noise variable with no posterior inference or KL term, making the model a continuous mixture of Negative Binomials. Nothing shows that zz encodes a state rather than just adding flexibility to the output distribution. Also, Sec. 4.1 only compares against a predictor that sees the overall zero rate, which does not apply to real sequence baselines. Without a direct test, the result just shows that a shared mixing variable gives a small WAPE gain.

Cross-horizon dependence is not compared against baselines that also model dependence. The theory and synthetic ablations only compare a shared latent with per-horizon independent latents. However, several baselines in the main tables (DeepAR, Chronos, and the copula models) already capture cross-horizon dependence. On real data, the paper does not compare joint calibration against these models, nor does it evaluate total demand over the lead time (the quantity that actually depends on cross-horizon covariance in inventory decisions). The main tables only report per-horizon WAPE, where differences are small.

Questions for authors

(Weakness 1) Does the model actually use the arrangement of zeros rather than just the zero rate? A simple check: shuffle the positions of zeros within each context window at test time (keeping the zero rate fixed), and report the performance drop for your model vs. a TCN/LSTM baseline.

(Weakness 2) On M5 (28-step horizon), could you compare against DeepAR and a copula baseline (DeepVAR or GPVAR-Copula) on the calibration of all-zero windows and on coverage of total demand over the lead time?

5 comments

W1&Q1

Nothing in the experiments shows that zz encodes a state rather than just adding flexibility to the output distribution.

Response: We thank the reviewer for raising this point. We added a controlled synthetic experiment with a known ground-truth regime generated by a persistent two-state HMM. The current state, StS_t, governs the complete H=6H=6 forecast window, while demand remains stochastic conditional on StS_t. After training, we freeze the model and fit identical logistic-regression probes to predict the true state:

RepresentationTrue-state AUROC, mean ± SD
Zero rate0.729 ± 0.008
Encoder representation ϕ\phi0.954 ± 0.005
Latent mean mzm_z0.952 ± 0.007

The latent mean mzm_z retains nearly all regime information available in the encoder and substantially more than the zero rate alone. Thus, when the true state is identifiable, ZZ encodes the latent regime rather than merely adding unstructured flexibility to the output distribution.

The latent distribution is conditioned only on information available at inference time, and the same predictive construction is used during training and inference. It is therefore not a target-conditioned variational posterior requiring a VAE-style KL term; the decoding results show that the absence of such a term does not make ZZ state-agnostic.

Does the model actually use the arrangement of zeros rather than just the zero rate? A simple check: shuffle the positions of zeros within each context window,... for a TCN/LSTM baseline.”

We performed a stricter version of the suggested test on the controlled synthetic benchmark with a persistent two-state HMM and H=6H=6. Within each context, we randomly permuted the complete demand observations, preserving the zero count, zero rate, total demand, and complete value multiset while destroying temporal order. Results are averaged over three seeds and ten independent shuffles:

ModelP50 WAPE, original / shuffledAll-Zero Brier, original / shuffledAll-Zero Log, original / shuffledEnergy/H\sqrt{H}, original / shuffled
Shared-Z TCN-NB0.446 / 0.5280.152 / 0.2460.449 / 0.8821.772 / 2.222
No-Z TCN-NB0.452 / 0.5290.167 / 0.2600.531 / 1.2421.776 / 2.265
No-Z LSTM-NB0.452 / 0.5330.172 / 0.2620.561 / 1.3021.776 / 2.262

All three models deteriorate after shuffling, showing that the temporal arrangement of observations, including persistent zero patterns, contains predictive information beyond the zero rate. Because the zero rate, total demand, and complete value multiset are preserved, the decline cannot be attributed to changes in sparsity or the marginal empirical distribution. No-Z TCN-NB and No-Z LSTM-NB show similar degradation, so this result is not specific to the TCN encoder. Shared-Z TCN-NB achieves the best joint-event scores under both original and shuffled contexts. The encoder extracts temporal regime information from the context, while Shared-Z propagates that information into a coherent joint distribution across forecast horizons.

W2&Q2

The dependence argument is not evaluated against the baselines that also model dependence... On real data, the paper does not compare joint calibration against these baselines

Response: We thank the reviewer for identifying this missing comparison. In the new real-data joint evaluations reported in our response to Reviewer s3rp, W1/Q1, Shared-Z achieves the best joint predictive scores on both Car Parts and M5. On M5, relative to DeepAR-NB, it reduces All-Zero Brier, All-Zero Log Score, and Energy Score by 3.5%, 23.8%, and 4.7%, respectively. These results show that the advantage extends beyond per-horizon WAPE to joint-event calibration and complete-trajectory distribution quality.

The evaluation compares matched Shared-Z and Independent-Z models, DeepAR-NB, and DeepVAR-Copula on common forecast origins and under the same evaluation protocol. Car Parts uses H=6H=6, and M5 uses H=28H=28. DeepAR-NB was selected because it performed strongly in our previous experiments, while DeepVAR-Copula provides a representative joint-distribution baseline.

The comparison also covers distinct mechanisms for constructing joint predictive trajectories. Shared-Z directly couples all HH forecast horizons through a common context-conditioned regime; DeepAR-NB propagates temporal dependence sequentially through autoregressive rollout; and DeepVAR-Copula models cross-series dependence through copula covariance while generating trajectories autoregressively. The results suggest that, for sparse series with persistent zero regimes, directly sharing a regime across the forecast window is better aligned with the relevant window-level dependence structure.

“On M5 (28-step horizon), could you compare against DeepAR and a copula baseline ... on coverage of total demand over the lead time?”

Using the same M5 cohort and forecast origins, we evaluated prediction intervals for cumulative demand over the complete 28-day lead time. We report coverage, mean width, and the standard interval score [1]. Coverage alone can be increased by widening the intervals, whereas interval score jointly evaluates calibration and sharpness.

Method80% Coverage80% Width80% Interval Score90% Coverage90% Width90% Interval Score
Shared-Z0.88014.25628.8670.94118.55740.515
Independent-Z0.6978.53831.1950.80310.95147.732
DeepAR-NB0.75810.11837.0090.84713.22255.570
DeepVAR-Copula0.5408.01546.7680.64910.29372.678

This result directly connects the empirical findings to Lemmas 2 and 3. The variance of cumulative lead-time demand satisfies

Var⁡(∑h=1,…,HY(h)∣I)=∑h=1,…,HVar⁡(Y(h)∣I)+2∑h<h′Cov⁡(Y(h),Y(h′)∣I).\operatorname{Var}\left(\underset{h=1,\ldots,H}{\sum}Y(h)\mid I\right)=\underset{h=1,\ldots,H}{\sum}\operatorname{Var}\left(Y(h)\mid I\right)+2\underset{h<h'}{\sum}\operatorname{Cov}\left(Y(h),Y(h')\mid I\right).

Under the Independent-Z construction, the horizon-specific latent variables are mutually independent, so the predictive cross-horizon covariance term is zero conditional on the context. When the true demand trajectory has positive regime-induced covariance, Independent-Z therefore tends to underestimate cumulative-demand uncertainty. This is reflected in its narrower intervals and substantial undercoverage at both nominal levels. Shared-Z allows this covariance term to be represented through the common latent regime, producing wider but substantially better-calibrated cumulative-demand intervals.

At the 80% level, DeepAR-NB is closer to nominal coverage, while Shared-Z is conservative but still reduces the interval score by 22.0% relative to DeepAR-NB. At the 90% level, Shared-Z is closest to the nominal coverage and reduces the interval score by 27.1%. Shared-Z achieves the lowest interval score at both levels, demonstrating the best calibration and sharpness tradeoff rather than merely improving coverage by widening the intervals.

This result is also operationally important because underestimating cumulative-demand uncertainty can lead to insufficient safety stock and stockouts. Together with the joint-event evaluation, it provides direct real-data evidence that Shared-Z more accurately represents predictive dependence across forecast horizons.

Reference

[1] Gneiting, T. and Raftery, A. E. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association, 2007.

I thank the authors for the responses and the additional experiments, which partially address my concerns about the principle of Independent-Z construction. I will maintain my assessment of the paper but positively evaluate the workload of the paper.

Reviewer_W9ZyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a regime-aware framework to model sparse time series. The framework extracts useful information from the sparsity signal in the observed time series and exploit a context-conditioned epinet to jointly predict future observations based on a shared latent regime sampled from a conditioned distribution. The results on four datasets showed improved performance compared to the baselines.

Critical strengths
  • The authors identified a practical problem in time series forecasting and proposed an interesting framework to tackle it based on epinet.
  • The ablation studies in Section 5 helps confirm the design decisions made in Section 3.
Critical weaknesses
  • The problem seems to be highly related to nonstationary time series forecasting. There are quite a few recent developments in this area, including but not limited to [1][2][3], but the authors neither discussed nor compared against this line of work.
  • The datasets used in the experiments seem (unnecessarily) limiting, probably due to how the authors framed the problem from retail and supply chain forecasting. There are other sparse time series, such as electricity usage, which is widely used in previous work. The results would be more convincing and the application scope of the work would be widened, if the paper could expand the experiments by including more datasets across domains.

References

  • [1] Liu, Y., Wu, H., Wang, J. and Long, M. Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting. Advances in Neural Information Processing Systems, 2022.
  • [2] Liu, Y., Li, C., Wang, J. and Long, M. Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors. Advances in Neural Information Processing Systems, 2023.
  • [3] Liu, S. and Lehrmann, A. DynaConF: Dynamic Forecasting of Non-stationary Time Series. Transactions on Machine Learning Research, 2024.
Questions for authors

N/A

5 comments

W1

The problem seems to be highly related to nonstationary time series forecasting... but the authors neither discussed nor compared against this line of work

Response: We thank the reviewer for highlighting this related research direction. Non-stationary forecasting and our work address complementary questions: the former studies how the forecasting mechanism changes over time, whereas our work studies how uncertainty about the persistent regime governing the current forecast window induces dependence across horizons. Although the original manuscript did not explicitly discuss the method from a non-stationarity perspective, our formulation does not impose a global stationarity assumption, and several of its components are designed to adapt predictions to time variation observable from the current context. Specifically, the TCN and sparsity-aware encoder extract local dynamics and temporal zero arrangements from the latest history, while the context-conditioned latent distribution is recomputed at every forecast origin. The zero-arrangement ablation reported in our response to Reviewer z9p2, W1/Q1 further confirms that the model uses this context-dependent temporal structure.

To directly test whether the advantage of Shared-Z is restricted to stationary settings, we conducted a separate controlled synthetic experiment comparing a stationary two-state HMM with fixed transition and emission parameters against a non-stationary two-state HMM whose transition probabilities and state-conditional emission means vary over time:

DGPModelAll-Zero BrierAll-Zero Log ScoreEnergy/H\sqrt{H}
StationaryShared-Z0.168 ± 0.0070.503 ± 0.0321.357 ± 0.014
Independent-Z0.175 ± 0.0020.566 ± 0.0121.361 ± 0.020
Non-stationaryShared-Z0.145 ± 0.0040.420 ± 0.0081.501 ± 0.048
Independent-Z0.151 ± 0.0100.473 ± 0.0231.506 ± 0.049

Within both the stationary and non-stationary DGPs, Shared-Z improves all three joint predictive scores relative to Independent-Z and reduces All-Zero Log Score by approximately 11%11\%. Its advantage therefore persists under controlled non-stationarity rather than depending on stationarity.

We also thank the reviewer for identifying explicit non-stationary modeling as a promising complementary direction. To examine whether it provides additional benefits, we replaced the original TCN encoder with the Non-stationary Transformer mechanism of Liu et al. (2022) [1], while holding Shared-Z, the Negative Binomial head, training objective, and evaluation protocol fixed:

DatasetEncoderP50 WAPEAll-Zero BrierAll-Zero Log ScoreEnergy/H\sqrt{H}
Car Parts, H=6H=6TCN0.49880.19120.56160.4901
Non-stationary Transformer0.50100.19670.57330.4919
M5, H=28H=28TCN0.45750.09090.30370.4516
Non-stationary Transformer0.47390.08820.31390.4682

The Non-stationary Transformer does not consistently improve overall predictive performance relative to the TCN. On M5, it reduces All-Zero Brier from 0.09090.0909 to 0.08820.0882, but increases All-Zero Log Score from 0.30370.3037 to 0.31390.3139 and worsens WAPE and Energy Score. Because Log Score penalizes confident errors more strongly, this combination suggests a small improvement in average event-probability error accompanied by more confident errors on a subset of cases. Thus, explicit non-stationary adaptation can provide complementary information, but does not replace shared-regime uncertainty modeling.

We will expand the related-work discussion to include Non-stationary Transformer [1], Koopa [2], and DynaConF [3], clarify that our formulation does not require stationarity, and identify integration with explicit non-stationary modules as a promising future direction.

W2

“The datasets used in the experiments seem unnecessarily limiting... There are other sparse time series, such as electricity usage... The results would be more convincing and the application scope widened if the paper included more datasets across domains.”

Response: We thank the reviewer for encouraging us to broaden the empirical scope and demonstrate the wider applicability of our framework. As stated in the Abstract and Introduction, this work originates from sparse count forecasting in retail and supply-chain operations; accordingly, the original evaluation aligns the practical motivation, methodological scope, and empirical evidence. Motivated by the reviewer’s suggestion, we further evaluated the framework beyond this original application domain using the Electric Vehicle Charging Station Usage dataset released by the City of Palo Alto [4]. This additional experiment extends our evidence to sparse event counts in the energy and mobility domain.

Using only information available before the first forecast origin, we selected series with historical zero rates of at least 75%75\%, giving 613 user-level weekly series. The resulting cohort is not concentrated in a few locations: it covers 33 of 47 stations (70.2%) and 56 of 79 station-port combinations (70.9%). We evaluate five rolling origins with a 6-week forecast horizon.

All models use only historical weekly session counts; station, port, user, location, and calendar information are not used as model inputs. We compare Shared-Z, Independent-Z, DeepAR-NB, and full-dimensional DeepVAR-Copula using the same cohort, forecast origins, and evaluation protocol.

MethodP50 WAPEAll-Zero BrierAll-Zero Log ScoreEnergy/H\sqrt{H}
Shared-Z0.46920.20790.61620.1809
Independent-Z0.47980.22970.65000.1871
DeepAR-NB0.48330.21800.63010.1869
DeepVAR-Copula0.49200.24431.13540.1962
Zero Forecast*0.5000N/AN/AN/A

Zero Forecast always predicts zero and satisfies P50 WAPE = 0.5 by construction.

All four forecasting models outperform the zero forecast at P50, confirming that the cohort is highly sparse but still predictable. Shared-Z achieves the best result on all four metrics. Relative to matched Independent-Z, it reduces P50 WAPE, All-Zero Brier, All-Zero Log Score, and Energy Score by 2.2%, 9.5%, 5.2%, and 3.3%, respectively. Relative to DeepAR-NB, the strongest external baseline, the corresponding reductions are 2.9%, 4.6%, 2.2%, and 3.2%. The advantage of Shared-Z is therefore not tied to retail demand or supply-chain-specific covariates.

EV charging counts match our target setting in both data structure and forecasting need. Long runs of zeros can correspond to persistent behavioral states, such as travel, switching to home charging, or vehicle changes, which may affect several future weeks simultaneously. The task therefore benefits from a coherent joint distribution over the forecast window rather than only accurate per-horizon marginals. Because the target is count-valued, we retain the Negative Binomial head. For other target types, such as continuous electricity load, the encoder, sparsity-aware representation, and Shared-Z construction can be retained while replacing the observation head. The output-head ablation in Section 5.2 of the submitted manuscript provides indirect evidence for this modularity, and we leave direct evaluation on continuous outcomes for future work. We will add the EV charging experiment to the revision and discuss extensions to healthcare utilization, continuous energy-load forecasting, and other sparse domains.

Additional Reference

[4] City of Palo Alto. Electric Vehicle Charging Station Usage (July 2011–December 2020). City of Palo Alto Open Data, 2021.

Thanks for the explanation and results. Would it be possible for the authors to compare against the non-stationary models directly as baselines instead of as part of the proposed model (i.e. only the encoder)? The authors "propose a regime-aware framework ... combining a sparsity-aware encoder with a context-conditioned latent regime to produce a joint predictive distribution over Negative Binomial outputs." Isn't regime switching one type of non-stationarity?

Reviewer_s3rpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes joint probabilistic forecasting for sparse count series using a context-conditioned Gaussian latent variable shared across forecast horizons, a sparsity-aware encoder, and Negative Binomial outputs. It explains why marginal losses cannot identify cross-horizon dependence and why sharing a latent variable can represent dependence that independent horizon-specific latents cannot. Evaluation covers one proprietary and three public demand datasets, with synthetic dependence ablations and a real-data training-objective ablation.

Critical strengths
  1. The paper connects a concrete forecasting concern to a clear probabilistic construction. Section 4.2 correctly distinguishes accurate marginals from a coherent joint distribution, with explicit conditional-independence assumptions. This provides a useful explanation of why persistent zero-demand events require attention beyond horizon-wise accuracy.

  2. The marginal forecasting results are consistently promising: the method has the lowest reported quantile-WAPE throughout Tables 1 and 2 across four datasets. Table 4 additionally shows that joint training improves both marginal accuracy and all-zero-event scores on Car Parts across three seeds, providing useful evidence that the proposed objective contributes beyond the architecture alone.

Critical weaknesses
  1. The central dependence advantage is not directly established on real data. The SharedZ/Independent-Z/No-Z comparison in Section 5.2 and Figure 3 uses only synthetic data. Table 4 changes the objective within the shared-latent architecture, while Tables 1 and 2 report marginal scores. Thus, the real-data results do not show whether shared regimes outperform independent regimes or existing trajectory-generating forecasters on joint predictions. This leaves the main claimed advantage beyond marginal accuracy insufficiently validated.

  2. The objective ablation is underspecified in ways that affect its interpretation. Appendix D does not define the tail-weighted marginal loss mathematically or report the default gamma and lambda_reg. Averaging probabilities over latent samples before taking the logarithm differs from averaging conditional log-likelihoods; tail weighting can also change the distribution targeted by the loss. Without these details, Table 4 cannot establish precisely which change in training produced its improvement.

  3. The epistemic interpretation goes beyond the demonstrated result. The Introduction distinguishes epistemic regime uncertainty from aleatoric variation, but Section 3.5 learns a context-dependent mixing distribution using predictive loss and a latent-moment penalty. This construction can represent predictive heterogeneity without identifying epistemic uncertainty separately from Negative Binomial dispersion. Section 4 establishes cross-horizon dependence, not this uncertainty decomposition. The distinction matters to the paper's uncertainty contribution, although it does not invalidate the predictive mixture.

Questions for authors
  1. What existing real-data joint-event evidence distinguishes shared regimes from the alternatives? Reporting all-zero-event scores from the already evaluated DeepAR trajectories on the same Car Parts origins would provide a direct comparison. If a matched SharedZ/Independent-Z comparison is available, please report it with the same objective and tuning protocol; this could change my recommendation.

  2. What is the exact marginal-loss formula, including latent-sample averaging and tail weights, and which gamma and lambda_reg values are used in Table 4? Which settings are held fixed between its rows? This would clarify whether the improvement supports the proposed dependence-learning explanation.

  3. Is z intended to quantify epistemic uncertainty or to serve as a predictive mixing variable? What argument or existing evidence distinguishes its variation from aleatoric heterogeneity? If that distinction is not established, can the claims be narrowed to learning joint predictive dependence?

4 comments

W1&Q1

The central dependence advantage is not directly established on real data...

Response: We thank the reviewer for this suggestion. We added real-data joint evaluations comparing Shared-Z with a matched Independent-Z ablation, DeepAR-NB, and DeepVAR-Copula on common forecast origins and under the same evaluation protocol. For the matched ablation, all architecture, objective, data, and tuning settings are fixed; the only difference is whether Z is shared across horizons or sampled independently for each horizon.

DatasetMethodP50 WAPEAll-Zero BrierAll-Zero Log ScoreEnergy/H\sqrt{H}
Car Parts, H=6H=6Shared-Z0.4987680.1911920.5616190.490145
Independent-Z0.5016730.2030290.5956420.493987
DeepAR-NB0.5106810.2028691.3417460.493374
DeepVAR-Copula0.5254260.2014201.4261940.508096
Oracle constant-pp referenceN/A0.2420310.677122N/A
M5 high-sparsity, H=28H=28Shared-Z0.4575000.0909440.3036950.451577
Independent-Z0.4578250.0981560.3766310.457467
DeepAR-NB0.4817670.0942660.3986540.474008
DeepVAR-Copula0.4848060.1525170.6407520.503605
Oracle constant-pp referenceN/A0.1006540.353868N/A

Setup: Car Parts contains 2674 monthly series and uses demand history only, with H=6H=6 and 7 rolling origins. M5 contains 10991 daily series with historical zero rates of at least 75%75\%, evaluated with H=28H=28 and 5 rolling origins using demand history only. DeepVAR-Copula is fitted jointly across the complete M5 cohort. All metrics are pooled over series and origins, and the all-zero event means that the complete HH-step window is zero. Oracle reference: The oracle assigns the empirical evaluation frequency qq to every series-origin pair, where q=0.410728q=0.410728 for Car Parts and q=0.113547q=0.113547 for M5. It is the optimal constant-probability reference for Brier and Log Score, but is not deployable because it uses the realized evaluation frequency.

Shared-Z achieved the best joint predictive scores on both real datasets, even when marginal accuracy changed little. On Car Parts, relative to matched Independent-Z, it reduced All-Zero Brier, All-Zero Log Score, and Energy Score by 5.8%5.8\%, 5.7%5.7\%, and 0.8%0.8\%, respectively; relative to DeepAR-NB, the corresponding reductions were 5.8%5.8\%, 58.1%58.1\%, and 0.7%0.7\%. It also improved Brier and Log Score over the oracle constant-probability reference by 21.0%21.0\% and 17.1%17.1\%. M5 showed the same pattern. Additional cumulative-demand evidence is reported in our response to Reviewer z9p2, W2/Q2: Shared-Z achieves the lowest Interval Score at both the 80% and 90% levels, reducing it relative to DeepAR-NB by 22.0%22.0\% and 27.1%27.1\%, respectively. Its lower joint-event and cumulative-demand scores have direct operational relevance because mischaracterizing forecast-window uncertainty can translate into stockouts or excess inventory.

W2&Q2

Appendix D does not define the tail-weighted marginal loss mathematically...

Response: For KK latent draws, we define the tail weight and conditional NB loss as

w(i,h)=1+βtaillog⁡(1+y(i,h)),w(i,h)=1+\beta^{\mathrm{tail}}\log(1+y(i,h)),

and

ℓ(i,h,k)=−log⁡pNB(y(i,h)∣z(i,k)).\ell(i,h,k)=-\log p^{\mathrm{NB}}\left(y(i,h)\mid z(i,k)\right).

The tail-weighted marginal loss is

LNB=1K∑k=1,…,K∑i,hw(i,h)ℓ(i,h,k)∑i,hw(i,h),\mathcal{L}^{\mathrm{NB}}=\frac{1}{K}\underset{k=1,\ldots,K}{\sum}\frac{\underset{i,h}{\sum}w(i,h)\ell(i,h,k)}{\underset{i,h}{\sum}w(i,h)},

where the sums over (i,h)(i,h) cover the BB training examples and HH forecast horizons.

Averaging probabilities over latent samples before taking the logarithm differs from averaging conditional log-likelihoods...

We agree that the two operations differ, and the different averaging orders are intentional. The marginal term averages conditional NLLs so that every latent draw receives stable horizon-wise supervision. For the joint term, define the conditional probability of the complete trajectory as

P(i,k)=∏h=1,…,HpNB(y(i,h)∣z(i,k)).P(i,k)=\underset{h=1,\ldots,H}{\prod}p^{\mathrm{NB}}\left(y(i,h)\mid z(i,k)\right).

The joint loss is

Ljoint=−1BH∑i=1,…,Blog⁡[1K∑k=1,…,KP(i,k)].\mathcal{L}^{\mathrm{joint}}=-\frac{1}{BH}\underset{i=1,\ldots,B}{\sum}\log\left[\frac{1}{K}\underset{k=1,\ldots,K}{\sum}P(i,k)\right].

Thus, the joint term marginalizes complete-trajectory probabilities before taking the logarithm, allowing different Shared-ZZ draws to represent alternative coherent trajectory-level regimes. Averaging conditional trajectory NLLs instead would decompose the objective into horizon-wise losses and remove the additional cross-horizon mixture signal.

Tail weighting can also change the distribution targeted by the loss...

We agree. Tail weighting defines a weighted predictive risk rather than an unweighted NB likelihood. It prevents the many zero and low-count observations from dominating training and gives greater influence to demand peaks, where forecast errors can have disproportionate operational consequences. It is not an asymmetric underprediction loss.

What is the exact marginal-loss formula... which gamma and lambda_reg values are used in Table 4, and which settings are held fixed?

The combined objective is

L=(1−γ)LNB+γLjoint+λregR(z),\mathcal{L}=(1-\gamma)\mathcal{L}^{\mathrm{NB}}+\gamma\mathcal{L}^{\mathrm{joint}}+\lambda^{\mathrm{reg}}R(z),

with

R(z)=E[m(z)2+s(z)2].R(z)=\mathbb{E}\left[m(z)^2+s(z)^2\right].

Table 4 uses βtail=0.5\beta^{\mathrm{tail}}=0.5, λreg=10−3\lambda^{\mathrm{reg}}=10^{-3}, and K=8K=8. The marginal-only row uses γ=0\gamma=0, whereas the marginal+joint row uses γ=0.75\gamma=0.75. All other settings are fixed, including the Shared-ZZ architecture, tail weighting, latent regularization, number of latent draws, data splits, seeds, and training and evaluation protocols. Table 4 therefore isolates the effect of adding the joint term under a matched protocol.

W3 and Q3

“This construction can represent predictive heterogeneity without identifying epistemic uncertainty separately from Negative Binomial dispersion...”

“Is zz intended to quantify epistemic uncertainty or to serve as a predictive mixing variable?”

Response: We thank the reviewer for identifying the missing empirical link. As stated in Section 1, we define aleatoric uncertainty as irreducible variation given the underlying state and epistemic uncertainty as uncertainty about the process state that is “resolvable in principle with more information.” Under this definition, our intended interpretation is that ZZ represents uncertainty about an unresolved regime, whereas NB dispersion represents conditional count randomness given that regime. We do not claim that predictive loss alone uniquely identifies a general epistemic/aleatoric decomposition on arbitrary data; nor does the formal connection to epistemic neural networks in Section 3.6 by itself validate this interpretation.

We therefore added a controlled synthetic experiment with an identifiable ground-truth regime. Historical states follow a persistent two-state HMM, and the current state governs the complete H=6H=6 forecast window while future counts remain stochastic under a state-conditional Hurdle-NB distribution. Because the data-generating process is fully specified, the true state, its exact filtering posterior, and its entropy are known.

We conduct three tests. First, we freeze the model and use identical logistic-regression probes to decode the true state from the zero rate, encoder representation, and latent mean. Second, we test whether the output variation induced by ZZ, denoted UZU^Z and measured as the average across horizons of the variance over ZZ in log conditional means, increases with exact filtering entropy. Third, we train with an optional state hint, revealed with probability 0.50.5, and evaluate the same model and test cases with the state either hidden or revealed. We then test whether revealing the state reduces UZU^Z while the mean conditional NB variance, denoted UNBU^{NB}, remains unchanged.

TestControlled testMeasure / comparisonResult, mean ± SD
1True-regime encoding (AUROC)Zero rate0.729 ± 0.008
Encoder representation0.954 ± 0.005
Latent mean0.952 ± 0.007
2State-uncertainty alignmentEntropy vs. UZU^Z, Pearson rr0.857 ± 0.060
Entropy vs. UZU^Z, Spearman ρ\rho0.856 ± 0.051
3Information contractionUZU^Z, hidden vs. revealedHidden: 0.1509 ± 0.0103; revealed: 0.1056 ± 0.0415; mean per-seed relative reduction: 29.4%
UNBU^{NB}, hidden vs. revealedHidden: 14.071 ± 2.181; revealed: 14.123 ± 1.783

The latent mean retains nearly all regime information available in the encoder and substantially more than the zero rate alone. Moreover, UZU^Z increases with true-state ambiguity, and revealing the regime reduces UZU^Z for all three seeds, with every per-seed 95% bootstrap confidence interval above zero, while UNBU^{NB} remains essentially unchanged. Thus, under the operational definition in Section 1, ZZ captures reducible epistemic regime uncertainty rather than duplicating aleatoric NB dispersion. We will add this direct validation and clarify the intended interpretation in the revision.

Dear Reviewer s3rp,

Thank you again for your review. We wanted to kindly check whether our responses have addressed your concerns, or if there is anything else we can clarify before the discussion period ends.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their constructive feedback. We have uploaded a revised version, with all changes marked in blue. The main updates are:

  • Theoretical scope (Abstract, Introduction, Conclusion). The claim now concerns models that treat horizons as conditionally independent given the context. Comparison with autoregressive and copula-based models rests on the real-data results. (Reviewer z9p2, W2)
  • Real-data joint evaluation (Section 5.2, Table 3; Appendix E). Matched Shared-Z/Independent-Z comparison with DeepAR-NB and DeepVAR-Copula on Car Parts and M5, together with cumulative lead-time demand intervals. (Reviewer s3rp, W1/Q1; Reviewer z9p2, W2/Q2)
  • Regime interpretation and zero arrangement (Section 5.3; Appendix F). Context-shuffling test with TCN and LSTM controls, and true-state decoding, uncertainty alignment and information contraction. (Reviewer z9p2, W1/Q1; Reviewer s3rp, W3/Q3)
  • Training objective (Appendix D). Exact loss definitions and default hyperparameters. (Reviewer s3rp, W2/Q2)
  • Non-stationary forecasting (Section 2; Appendix H). Stationary and non-stationary controlled DGPs, standalone Non-stationary Transformer baseline, and encoder replacement. (Reviewer W9Zy, W1 and follow-up)
  • Cross-domain evaluation (Appendix H). Palo Alto EV charging counts. (Reviewer W9Zy, W2)

These revisions clarify the scope of our claims and strengthen the supporting evidence, without changing the core assumptions or contributions of the paper. We are grateful to the reviewers for suggestions that substantially improved the experimental evaluation, and for the discussion on non-stationary forecasting and cross-domain applications, which points to promising directions for future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.