- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CPSS, which learns a distribution over 64 weighted trajectories around a frozen TimesFM-2.5 model. Hierarchical, suffix-supported refinements allow related scenarios to share early prefixes, which merge with their probability masses when truncated. Training combines the energy score, marginal CRPS, and entropy regularization. The results show improvements at longer horizons and on path-dependent quantities, but the attribution of these gains and the reliability of the evaluation require further clarification
Critical strengths
**Practically relevant motivation. **The paper clearly distinguishes marginal forecasting from joint trajectory forecasting and evaluates meaningful path-dependent quantities, including maxima, ramps, and exceedance duration.
Clear structural design. Suffix-supported refinements and hierarchical probability allocation provide an interpretable branching representation with exact prefix merging under truncation.
Useful controlled evaluation. CPSS and GB64 share the frozen carrier, scenario budget, and similar parameter counts. The paper also examines path-dependent scores, calibration, and scenario diversity.
Critical weaknesses
The main ablation conflates hierarchy with basis design. CPSS uses local, suffix-supported basis functions, whereas GB64 uses only eight global cosine components per trajectory. These representations differ in their ability to capture local variations, particularly at long horizons. Consequently, the observed gains cannot be attributed solely to hierarchical branching. A flat model using comparable local bases and an ablation removing suffix constraints would strengthen the evidence.
The cross-horizon consistency guarantee has a limited scope. The construction guarantees prefix merging when truncating a single terminal forecast distribution. However, the heads for are trained separately, so consistency between these independently trained forecasts is not established. The paper should distinguish shared-prefix structure from consistency across horizon-specific models and demonstrate the practical value of the former.
Small evaluation panels and unclear data separation limit confidence. The horizon sweep uses only eight forecast origins per dataset, while the 12-domain joint evaluation contains 96 cases. Case identifiers are disjoint, but windows may overlap, and both training and part of the evaluation draw from GiftEvalPretrain. The authors should clarify separation by series and time interval. Case-level bootstrapping may also underestimate uncertainty when windows or channels are correlated; grouped or temporal-block resampling would be more appropriate.
Evidence for improved dependence modeling is mixed. Although CPSS improves energy and path-functional CRPS relative to TimeMCL, its variogram score worsens by approximately 5.78%, with an interval excluding zero. This result deserves discussion. Moreover, the long-horizon conclusions rely primarily on GB64 rather than a broader set of competitive joint forecasting methods.
Sensitivity analyses and implementation details are incomplete. The scenario budget, branching locations, basis size, and entropy threshold are fixed without corresponding ablations. More detail on training-data sampling and exclusion rules, the TimeMCL adaptation, and computational costs is needed to assess reproducibility and generalizability.
Questions for authors
see weakness
4 comments
Response to Reviewer gyNs (2/3): W3–W4
W3
Small evaluation panels and unclear data separation limit confidence. The horizon sweep uses only eight forecast origins per dataset, while the 12-domain joint evaluation contains 96 cases. Case identifiers are disjoint, but windows may overlap, and both training and part of the evaluation draw from GiftEvalPretrain. The authors should clarify separation by series and time interval. Case-level bootstrapping may also underestimate uncertainty when windows or channels are correlated; grouped or temporal-block resampling would be more appropriate.
H128 scenario-head training excludes all evaluation domains, and the 569 new cases exclude recorded window overlap. Grouped resampling keeps related observations together, with the H720 origin-spacing study reported separately.
-
Training separation is verified for the H128 scenario heads. Their 2,368-record, 37-domain bank excludes all 12 H128 evaluation domains. H720 uses a separate 36-domain training bank with frozen schedules, and its run receipts record no benchmark test-target access. These checks concern the scenario heads, with carrier-pretraining overlap and cross-domain aliases remaining unverified.
-
The new H128 cases remove recorded full-window overlap. The 569-case evaluation excludes complete windows intersecting either earlier evaluation and enforces nonoverlap within source items, regardless of channel. Cases were selected before forecasting using fixed source hashes. The original evaluations retain their reported overlapping windows, with these populations distinguished in Appendix D.2.
-
Uncertainty analyses preserve relevant groups. Dataset resampling keeps each dataset's paired comparison together, the five-family sensitivity groups the four ETT datasets, temporal blocks retain all channels at an origin, and H128 item resampling keeps a source series' windows together.
Grouped result CPSS advantage 95% interval Original H720 Energy against GB64, dataset resampling +8.518 points [4.631, 11.744] Matched sharing Energy, ETT treated as one family +0.321 points [0.093, 0.548] New H128 functional composite against GB64, domain-and-item resampling +3.921% [0.975, 21.106] The supplementary H720 origin-spacing study is identified separately because 55 of its 67 origins overlap earlier target intervals. Nonoverlapping windows are not automatically independent observations. Appendix D.2 gives the separation checks and grouped protocols.
W4
Evidence for improved dependence modeling is mixed. Although CPSS improves energy and path-functional CRPS relative to TimeMCL, its variogram score worsens by approximately 5.78%, with an interval excluding zero. This result deserves discussion. Moreover, the long-horizon conclusions rely primarily on GB64 rather than a broader set of competitive joint forecasting methods.
CPSS improves Energy and the functional CRPS composite over TimeMCL in the 96-case evaluation, with the composite gain also present across 569 cases. Four additional H720 interfaces broaden the comparison.
-
CPSS improves complete-path and functional scores in the original H128 comparison. Against TimeMCL, relative loss reductions are +6.69% [1.96, 9.67] for Energy and +11.18% [2.32, 19.14] for four-functional CRPS. The variogram reduction is -5.78% [-14.49, -0.14], favouring TimeMCL. Energy assesses complete trajectories, while variogram examines moments of lagged increments, so the measured trade-off concerns different properties of the forecast law.
-
The expanded evaluation also favours CPSS on the functional composite. Across 569 cases, the reduction over TimeMCL is +2.926% [0.365, 19.352] under domain-and-item resampling. The gain is mainly associated with threshold duration. The constant-history Wind Power case remains included. Appendix D.3 reports both aggregation conventions, with the ratio-of-risks Energy interval against TimeMCL including zero and variogram favouring TimeMCL under mean within-domain skill.
-
Long-horizon comparisons now include four additional joint forecasting interfaces. The following H720 comparisons use 64 paths on the same 1,480 context-512 cases, with positive Energy skill favouring CPSS.
Evaluated H720 interface CPSS Energy skill, points 95% interval TimeMCL carrier adapter -0.116 [-0.903, 0.527] TACTiS-2 carrier adapter +0.137 [-1.007, 1.101] TimePrism-64 adaptation +38.043 [19.608, 58.671] Toto 1.0 native sampler +93.005 [48.546, 136.061] The evaluated TimePrism and Toto interfaces favour CPSS, while the TimeMCL and TACTiS-2 intervals include zero. These system comparisons use differing training and forecasting interfaces, including mixed 1,000/6,000-update TimePrism checkpoints. They broaden the evaluation alongside the matched architectural tests. Section 5.3 and Appendix D.5 give the interfaces and results.
Response to Reviewer gyNs (3/3): W5
W5
Sensitivity analyses and implementation details are incomplete. The scenario budget, branching locations, basis size, and entropy threshold are fixed without corresponding ablations. More detail on training-data sampling and exclusion rules, the TimeMCL adaptation, and computational costs is needed to assess reproducibility and generalizability.
The 48-fit study covers the requested sensitivities, with training sampling and TimeMCL details specified below. Matched implementation tests reduce CPSS head time to 3.18 ms and show memory savings in combined generation and scoring.
-
The requested sensitivities are evaluated in the 48-fit component study. At H720, the following estimates are relative to the 640,059-parameter reference, with positive Energy skill favouring the modified configuration.
Modified configuration Active parameters Energy skill, points 95% interval 32 scenarios 492,315 -1.976 [-2.702, -1.065] 128 scenarios 935,547 +1.743 [0.971, 2.463] Two basis functions per node 509,757 -2.293 [-3.225, -1.222] Eight basis functions per node 900,663 +2.111 [1.099, 3.041] At fixed capacity, the original schedule gains +0.269 [0.105, 0.417] and +0.606 [0.315, 0.898] points over uniform and later refinement. Removing the entropy penalty or raising its floor from 0.35 to 0.70 leaves reference Energy performance stable, and disabling it preserves the relaxed-suffix comparison. Appendix D.1 provides all configurations.
-
Training selection and exclusion rules are explicit. The H128 scenario-head bank has 2,368 records from 37 domains, excluding all 12 evaluation domains. Each 16-window training batch used eight domains and two sampled windows per domain, with a fixed hash-based domain schedule and seeded sampling of eligible records and forecast origins. Contexts cycled through 128, 256 and 512 for H128, and 96, 128, 256 and 512 for the horizon sweep. Fixed source hashes select evaluation cases. The new 569-case evaluation excludes recorded full-window overlap within source items and against both earlier evaluations. Appendix D.2 states the source, channel and temporal rules and the scope of the verified exclusions.
-
The TimeMCL adaptation is specified. It uses the official conditional class at commit 666840a, 160 pooled history-and-carrier features, 64 uniformly weighted hypotheses and relaxed winner-takes-all training. The H128 adapter has 1,759,192 parameters, uses the original 37-domain batches, seeds 11 and 22, and 1,000 updates. AdamW has a 50-update warm-up and cosine decay from 0.0003 to 0.00003. Appendix D.5 gives the implementation.
-
Computational measurements include both generation and prefix evaluation. Basis caching and vectorisation reduce CPSS head time from 67.32 ms to 3.18 ms in a new matched H720 test with frozen checkpoints, float32, an RTX A6000 and batch size 128. Cached GB64 and local-basis heads take 1.97 ms and 3.26 ms. All 1,480 cases and three seeds passed numerical checks.
Streamed H720 head and H336 scoring, batch 64 CPSS GB64 Local control Median time 4.02 ms 3.14 ms 4.00 ms Incremental peak allocation 45.43 MB 49.65 MB 49.65 MB The combined operation includes the conditioner, prefix preparation, Energy and sorted marginal CRPS. It excludes frozen-carrier generation and transfers. Ten warm-ups and 50 synchronised measurements per seed produce the reported medians. These measurements show the memory benefit of prefix aggregation with streamed decoders and the lower-memory evaluator. Appendix D.4 and Table D.8 give every implementation stage and both evaluators.
Thank you for the additional experiments and clarifications. The response addresses most of my concerns. I will maintain my original score.
Response to Reviewer gyNs (1/3): Q1, W1–W2
Q1
Matched tests identify predictive gains from nested sharing, and exact prefix aggregation reduces measured scoring costs. The five responses below give the supporting experiments and implementation details.
The reviewer refers to the five weaknesses. We quote and address each below.
W1
Matched ancestry isolates gains in Energy, variogram and maximum CRPS with local bases and capacity fixed. CPSS also represents H336 forecasts with at most 16 distinct prefixes, versus 64 for the flat local-basis control.
The sharing contribution is isolated with local geometry held fixed. The matched ancestry intervention preserves the dictionary, capacity, initialisation, terminal-logit probability map and training batches, with zero entropy penalty in both arms.
The Energy estimates favour nesting in every dataset and seed. This answers whether ancestry contributes beyond choosing local basis functions.
Both requested additional controls were evaluated. The 638,464-parameter flat local-basis system uses the same suffix dictionary but a different coefficient factorisation and masses. It gains 2.069 [1.136, 2.892] Energy points over CPSS. Relaxed suffix support gains +2.793 [1.520, 3.921] Energy points, while exact-support CPSS gains +12.233 [4.569, 20.393] maximum-ramp CRPS points over that relaxation.
The interventions answer distinct questions. Matched rewiring isolates the sharing effect. The flat-system and suffix comparisons measure predictive flexibility alongside exact prefix structure. At H336 the flat control retains 64 distinct prefixes, while CPSS has at most 16 and supports lossless aggregation. Section 5.2, Table D.2 in Appendix D.1 and Appendix D.4 give these results and their protocols.
W2
Exact prefix aggregation preserves one forecast's restricted distribution and prefix-dependent objectives. At H336, it reduces 64 records to 16 and lowers measured scoring memory under both evaluators, with generation costs reported separately.
The guarantee is exact shared-prefix consistency within one terminal forecast. Related scenarios are identical before their prescribed divergence, and descendant masses sum under restriction. Separately trained heads are separate forecast laws. Section 3.3 states this scope explicitly.
Exact grouping preserves the predictive object and its prefix-dependent uses. At H336, grouping the 64 terminal records into 16 prescribed prefix records leaves the restricted forecast distribution and every prefix-dependent expected-cost objective unchanged.
The compact law reduces measured scoring cost. These are measurements of the same issued CPSS forecasts at batch size 64.
The original scoring operation is also 2.76 times as fast after H540 aggregation. Table D.7 includes H720 without prefix reduction. The score-only measurements exclude generation and merge preparation. Appendix D.4 gives the regrouping identity, equality checks and both evaluators. W5 below reports generation and combined-operation costs separately.