- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes DAShrink, a method for combining horizon forecasts from multiple time-series forecasting models by balancing a uniform combination with a learned router. Given k pre-trained frozen models, DAShrink first combines their predictions using two Mixture-of-Experts (MoE) routers: (1) a uniform router, assigning 1/k to each forecaster prediction (the mean operation), and (2) a learned MoE router, which is trained to predict assignment weights from normalized encodings from each expert and optimized with an MSE loss and a regularization term. The weights obtained by these two routers are balanced via linear interpolation based on two parameters: , which is obtained by fitting on a validation set, and , which is set per time point and per variable based on the disagreement between the two routers. The method is implemented using three experts: DLinear, TimeFilter, and an expert combining decomposition, lag-based extrapolation, and variable-token attention (denoted as a ‘contextual expert’). It is evaluated on the ETT datasets (excluding ETTh1), PEMS datasets, Electricity, Traffic, and Solar-Energy, and compared to time-series baselines as well as to MoE-like methods (e.g., MoLE-DLinear and TimeFuse).
Critical strengths
The method outperforms several time-series baselines and MoE-like methods on standard time-series benchmarks by notable margins in the majority of cases.
Critical weaknesses
-
The method is framed as general, yet it is evaluated using only three specific experts, without showing how the identity of the specific experts affects performance gains or whether the proposed balancing works across various configurations. Crucially, it is not clear whether the performance gain comes from the combination method or from the specific experts chosen. In addition, the paper is missing an apples-to-apples comparison with MoE-like baselines and the baseline of the contextual expert and the uniform router.
-
The conceptual motivation and empirical justification for key design choices is missing or only partially presented. In addition, for some parts, the paper appears rushed (missing notation definitions, missing references).
Questions for authors
Questions
- Have the authors evaluated their method using different configurations (varying the number of experts and expert identities)?
- Could the authors please provide an apples-to-apples comparison using the same set of experts while varying only the combination method? For example, running TimeFuse with the same expert set used by DAShrink, or running DAShrink under the same configuration as MoLE-DLinear (i.e., using only DLinear experts)?
- Could the authors please include the results of the uniform router and of the contextual expert for the main experiments in the paper (Tables 1-2)?
- Could the authors please provide a clear motivation for their main design choices and empirically motivate them?
3 comments
I thank the authors for their time and response, which addressed some of my concerns. However, critical issues remain: the main evaluations are limited to two datasets (Etth2/Ettm1), with ETTh1 still omitted without clarification. Reporting on all four ETT datasets (alongside additional benchmarks) is the minimum standard in recent literature. Furthermore, the matched comparisons with TimeFuse and MoLE show only marginal gains. Taken together, the limited benchmark coverage and modest improvements under matched protocols indicate that more work is needed, and I thus maintain my score.
- Benchmark coverage and ETTh1. The main evaluation tests the complete DAShrink system on 10 datasets and 40 dataset–horizon settings, with four horizons per dataset (Tables 1–2). The suite includes ETTh2, ETTm1, ETTm2, Electricity, Traffic, Solar-Energy, PEMS03, PEMS04, PEMS07, and PEMS08. ETTh2 and ETTm1 are the two datasets used in the additional matched comparison at horizon 96, where the combination heads are independently fitted on identical frozen expert forecasts with three seeds per task (Tables 7 and 10).
ETTh1 was outside the original benchmark grid. In the revision, we retained that grid and used the new experiments to address the attribution question on the two series from the original component study. Holding the expert forecasts fixed lets us measure how changing the combination rule affects the issued forecast on those series. That scope accounts for why ETTh1 was not added in the revision. We agree that extending the matched protocol to all four ETT datasets would strengthen the empirical evidence for the combination rule.
- Matched results and the contribution of DAShrink. The mean MSE differences against matched TimeFuse and MoLE are small. On ETTh2 and ETTm1, DAShrink and matched TimeFuse achieve 0.9703 versus 0.9787 and 0.8946 versus 0.8942, respectively. On the shared DLinear bank, DAShrink and the matched MoLE gate achieve 1.1036 versus 1.1092 and 0.8925 versus 0.8927.
DAShrink uses expert disagreement at each forecast step and channel to set an explicit bound on how far the learned combination can move from the uniform forecast. With the experts held fixed, the 99th-percentile absolute departures on ETTh2 and ETTm1 are 0.017 and 0.102 history standard deviations for DAShrink, 0.081 and 0.538 for constant shrinkage, and 0.306 and 1.397 for plain routing. These values are averaged across seeds on unclipped coordinates (Section 4.3). Proposition 2 explains why observing disagreement matters. Under its conditions, DAShrink retains a larger correction at unit spread than a scalar rule given the aggregate correction and fixed context at the same worst-case sensitivity. In practice, the sensitivity bound lets a forecaster determine at issue time whether a specified perturbation to the proposed weights could flip an alarm decision (Figure 8).
1. Expert configurations and attribution. We independently fit every trainable combiner on eight banks of 2-5 experts on ETTh2/ETTm1 H96 with three seeds each. The banks comprise the contextual/DLinear/TimeFilter (N/D/T) bank, all three two-expert subsets, three independent DLinear models, two DLinear models plus TimeFilter, five experts, and an exact duplicate. Within each task, seed and bank, every rule receives identical frozen forecasts. This fixes expert quality and expert cost across combination rules (Tables 7-10, D.1-D.2).
DAShrink's mean paired MSE reductions from each bank's uniform average are 0.444% for three DLinear models and 2.348% for five experts (six task-seed units each; Table 8). These configurations establish that the rule operates beyond the particular contextual expert. The contextual expert contributes differently by task: adding it to DLinear/TimeFilter changes uniform MSE from .9416 to .9711 on ETTh2 and .9687 to .9285 on ETTm1.
2. Matched TimeFuse and MoLE. The matched TimeFuse uses its upstream linear-softmax fusion head and 22 history features on exactly the shared N/D/T reports. On ETTh2/ETTm1, DAShrink MSE is .9703/.8946 and matched TimeFuse is .9787/.8942. On three shared, frozen DLinear forecasts, DAShrink/MoLE-gate MSE is 1.1036/1.1092 on ETTh2 and .8925/.8927 on ETTm1 (Tables 7, 10). MoLE's adapter uses its four calendar features and zero head dropout; DAShrink uses report/history features. This tests combination heads with expert identity fixed. Original MoLE trains heads/gate jointly; D.2 specifies the frozen-bank adapter.
Each neural combiner is fitted under its own objective and a shared 30-epoch budget with router-stop checkpoint selection. Plain routing minimizes MSE; the two shrinkage heads minimize MSE plus .05 KL. Static stacking solves a simplex-constrained MSE quadratic program. Expert fit, expert stop, router fit, router stop, scale calibration and testing use six disjoint chronological blocks, with standardization estimated from expert-fit data (Tables 6-7, D.1). These separately fitted alternatives complement the original fixed-parameter deletions.
3. Standalone experts and uniform averaging. Table 7 adds the requested controls on ETTh2/ETTm1 H96, with three-seed MSE/MAE means and sample SD. Mean MSE/MAE:
The rows distinguish standalone expert quality, equal pooling and the incremental combination rule on identical test windows per seed. All use D.1's disjoint splits and fit-only target scaling. Tables 1-2 retain the original forty-task system benchmark; the added control coverage is these two H96 tasks. Table 7 supplies same-bank attribution; Section 3 and Tables 15-19 establish the spread-dependent movement constraint and its decision consequence.
4. Design motivation, evidence and notation. Uniform weights are fixed at 1/K. Spread is the sample SD of clipped, history-normalized expert reports at one lead/channel. Original training alternates expert/router updates; the router uses detached reports. Experts stay fixed after checkpoint selection (Section 3, Figure 1).
The post-softmax control enforces a budget for every proposal. Local coefficients shape the correction; the task scale sets its amplitude, with zero recovering exact averaging. Equation 6 derives the quadratic validation-loss path (C.4). The .25 cap gives wbhc(k) in [1/4,1/2] for three experts. With report and spread clips inactive, proposal sensitivity is bounded proportionally to se−s, decreasing above unit spread.
Fixed-parameter removals increase both errors in all eight original ETT settings. Independent refits evaluate accuracy and departure jointly: DAShrink improves on uniform in four core runs and ties in two; plain routing has lower MSE. Seed-mean p99 absolute departures from uniform on unclipped coordinates are .017/.102 history SD for DAShrink, .081/.538 for constant shrinkage and .306/1.397 for plain routing on ETTh2/ETTm1 (Section 4.3). Tables 15-18 compare inverse spread, direct clipping, parameter settings, no-KL and reversed response.
Proposition 2 identifies the value of observing spread: at matched sensitivity, zero-preserving scalar rules given only aggregate correction and fixed history/mean context retain at most one eighth of DAShrink's unit-spread correction, under its parameter, unclipped-report and margin conditions. All 44 references were verified against primary sources. We invite reassessment of novelty and soundness using the enforced movement budget, its information analysis and the matched tests.