- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Matryoshka is a plug-in for multivariate time series forecasting that selects channel interactions using predictive evidence. Training-only calibration constructs a landmark diffusion embedding and a binary hierarchy of channels. Ridge pilots compare neighborhood information against each channel's own lagged history on two chronological evaluation blocks. Accepted fine, coarse, and global scopes receive nested-width, zero-initialized residual mixers, together with a channel-wise self mixer. The paper gives Gaussian-reference risk and independent-evaluation concentration results and evaluates the plug-in across twelve datasets, five backbones in a unified benchmark, and Linear/Transformer backbones in a CI/CD benchmark.
Critical strengths
- The paper identifies an important limitation of existing channel interaction strategies in multivariate time series forecasting. Previous approaches typically determine channel interactions based on predefined correlations, learned structures, or clustering mechanisms, without explicitly verifying whether the selected channels provide additional predictive benefits. In contrast, this paper formulates channel mixing as an evidence-based selection problem and introduces a predictive gain criterion to determine whether incorporating a channel neighborhood improves forecasting beyond each channel's own history.
- The plug-in design is practically appealing, as the proposed module can be integrated with different forecasting backbones without redesigning the entire architecture.
- The experimental evaluation is extensive, covering multiple forecasting benchmarks with different numbers of channels and various backbone architectures. The consistent improvements on high-dimensional datasets demonstrate the effectiveness of adaptive channel interaction, especially when the number of variables is large.
- Although the theoretical results rely on simplified assumptions, they offer useful intuition for why channel interaction should not always be expanded globally.
Critical weaknesses
- The provided implementation includes an additional validation-driven pruning procedure that removes doll modules according to validation performance. While this procedure itself is reasonable, it is not described in the paper and introduces an additional model selection stage beyond the proposed evidence gate. Therefore, the reported improvements are not fully attributable to the evidence-based qualification mechanism alone, and further ablation studies separating the effects of evidence gating and validation-based pruning would be helpful. Moreover, this additional validation-driven selection may introduce extra model selection overhead compared with baselines that do not perform such post-training structural optimization.
- The effectiveness of Matryoshka appears to depend strongly on the underlying backbone. Since channel-dependent architectures already perform channel interaction internally, the additional dolls provide limited gains on models such as iTransformer and DUET.
- Since the evidence gate already evaluates whether different channel neighborhoods provide additional predictive benefits, a natural alternative is to select only the most beneficial interaction range for each channel rather than retaining multiple qualified dolls in a nested manner. However, the paper does not provide sufficient theoretical or empirical analysis explaining why combining multiple qualified channel interaction granularities is preferable to selecting a single optimal one. An ablation comparing nested doll usage with single-doll selection would help clarify the motivation of the Matryoshka design.
Questions for authors
- The evidence gate is used to determine which channel neighborhoods should receive additional interaction capacity. Could the authors clarify why multiple qualified neighborhoods are combined in a nested doll structure rather than selecting only the most beneficial interaction range for each channel? Have the authors compared the proposed nested design with a single-doll selection strategy?
- The implementation appears to include an additional validation-driven pruning procedure after training. Could the authors clarify whether the reported results are obtained before or after this pruning step? If pruning is applied, how much does it contribute to the final performance improvement compared with the evidence gate alone?
5 comments
W3 / Q1. Why nested dolls instead of the single best range per channel?
- The scales are not interchangeable. The fine doll sends pairwise lead-lag messages inside a group of at most 16 channels. The coarse doll adds the mean of a neighborhood of up to 32. The global doll adds an attention summary of all channels. A single range forces each channel to choose between pairwise detail and wide context.
- The theory points the same way. Information never decreases as the neighborhood grows (Lemma 2), while in the linear reference of Theorem 1 the cost of estimating pairwise interactions grows with its size. Nesting takes pairwise capacity where it is cheap, and a fixed-capacity summary for the wide ranges.
- Existing evidence. In Table 6, the fine level alone gives 0.199 on DLinear. Adding the coarse level gives 0.193, and adding the global level gives 0.178.
[Experiment 8]: Nested dolls against a single scale. We ran the comparison you ask for. In "best single scale", each channel keeps only the accepted scale with the largest held-out pilot gain. The innermost doll is kept in every row.
| MSE | DLinear (7 datasets with hundreds of channels) | iTransformer (Solar, Electricity, PEMS08) |
|---|---|---|
| Backbone | 0.333 | 0.195 |
| Fine only | 0.199 | 0.175 |
| Coarse only | 0.227 | 0.180 |
| Global only | 0.192 | 0.183 |
| Best single scale per channel | 0.212 | 0.179 |
| Nested (ours) | 0.178 | 0.165 |
On DLinear, the nested model is better than the best single scale on all seven datasets, and better than every fixed single scale on average. The global doll alone, the strongest single scale, matches the nested model on Electricity and Traffic (0.181 and 0.489 against 0.182 and 0.487). On Solar and the four PEMS datasets the nested model is 3–24% below it. On iTransformer, the nested model is again better than every single scale on average.
Summary. Nesting is better than any single scale overall. Against the alternative you propose, the best single scale per channel, the nested model has a 16% lower MSE with DLinear (0.178 against 0.212) and an 8% lower MSE with iTransformer (0.165 against 0.179). Against the strongest fixed scale it is 7% and 6% lower (global only with DLinear, fine only with iTransformer). The gain is largest where cross-channel structure is strong: 15–24% on the PEMS datasets.
Overall.
- The results do not depend on pruning. 237 of the 240 runs are identical without it, and no backbone average changes by more than 0.001. The step never read the test split. The revised code does not contain it, and the revision reports every result without it.
- Backbones that mix channels still gain where the structure is strong. On Solar and the four PEMS datasets, Matryoshka lowers the MSE of iTransformer and DUET by 13–30%. The clear losses are on Traffic.
- Nesting is better than a single scale. The nested model has a 16% lower MSE than the best single scale per channel with DLinear and an 8% lower MSE with iTransformer. It is also better than every fixed single scale on average.
We hope that the full account of the pruning step and the two new experiments address your concerns, and we would be grateful if you considered them in your final assessment.
New experiments referenced: [Experiment 7] Validation pruning: effect, cost, and interaction with the gate · [Experiment 8] Nested dolls against a single scale
I appreciate the additional experiments, but I remain unconvinced by the rebuttal. The authors have partially addressed my questions, but several concerns remain unresolved:
-
The proposed method still lacks consistent effectiveness on models that already incorporate channel interactions
-
The experimental evidence remains insufficient to establish the distinct contribution of the proposed core mechanisms, much of the performance improvement may come from introducing additional channel interactions into the original backbone, while the proposed evidence gating and hierarchical structure provide relatively limited additional gains.
The current evidence is still insufficient to justify acceptance, and I will maintain my original rating.
Response to Reviewer N5Eq
We thank the reviewer for the valuable and constructive feedback.
W1 / Q2. Validation-driven pruning.
[Experiment 7]: Validation pruning: effect, cost, and interaction with the gate. We checked the stored pruning mask and the log of every reported run.
(i) Effect on the reported results (12-dataset MSE):
Pruning switched off a unit in 3 of the 240 runs: DLinear on Traffic at H = 192 and 336, and DUET on PEMS07 at H = 48. In the other 237 runs the pruned and unpruned models are the same model. The three affected cells, without -> with pruning, are 0.465 -> 0.451, 0.483 -> 0.468, and 0.091 -> 0.090. So the step matters in one place, DLinear on Traffic: 0.487 without it and 0.480 with it, against 0.625 for the backbone.
(ii) Gate against pruning, on the four ETT datasets and Weather (the setting of Table 7):
Pruning removed nothing in any gated run on these datasets, so the gate alone gives the reported numbers. Pruning cannot replace the gate.
(iii) Cost. The step needs at most 13 forward passes over the validation split and no retraining. It took 7% of the training time over 60 runs. The revised code no longer has this stage.
Summary. The pruning step was left in the code by mistake, and the reported results include it. Its effect is small: it changed 3 of the 240 runs and one four-horizon average, and it cannot replace the gate. The revision removes the step from the code and reports the results without it.
W2. Dependence on the backbone.
We agree, and Section 5 names it as the main limitation. The 4% average on iTransformer and DUET mixes three different cases (Tables 13–14):
Summary. The dependence is real, and it is expected: a plug-in that mixes channels has less to add to a backbone that already mixes them. On the seven datasets with hundreds of channels, Matryoshka lowers the MSE by 47% and 28% with the channel-independent DLinear and PatchTST, and by 20%, 9%, and 9% with TSMixer, iTransformer, and DUET. But where the cross-channel structure is strong, these backbones still gain. On PEMS08, iTransformer improves by 30% [Experiment 6]. The revision states this in Section 4 and Section 5.