- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
TRACE adds a carried structural state to the window-normalization adapter slot (RevIN, SAN, FAN, DDN). A fixed Kalman filter with a level and K undamped periodogram harmonics runs over the whole series, keeping one state per channel. A zero-initialized gate of 226–610 parameters mixes the filter's closed-form rollout with the window-normalized backbone forecast. On 4 backbones x 7 datasets x 4 horizons at L = 96, TRACE lowers MSE by 10.66% against RevIN and 6.10% against DDN. Controlled experiments on 12 cells attribute the gain to history outside the window
Critical strengths
- Simple, cheap, and consistent under the standard protocol. At L = 96, TRACE beats RevIN in 105/112 cells and DDN in 97/112, and the resampled intervals exclude zero. It equals the backbone exactly at initialization, and the filter has no trainable parameters. Thus, TRACE is an easy add-on that reliably helps short-window models
- Informative ablations. When the model is retrained with the state limited to in-window history, the gain disappears (Tables 3, 19: all 12.2% on DLinear/Weather-96), so it comes from out-of-window history. Table 9 combines window-only forecasts with the backbone in the same way, including a ridge fit given the filter's frequencies, and they do not reproduce the gain. Negative results that sometimes the gate does not perform better than a scalar weight are reported openly. This makes the paper informative even where results are mixed
Critical weaknesses
-
The headline gain mostly reflects access to more history.
- TRACE sees the history outside the window, while the baselines see only 96 steps. The gains concentrate where cycles exceed the window: against RevIN, 22.1% on Traffic, 15.6% on ECL, and 12.9% on Weather, versus 3.1–4.1% on ETTh1, ETTh2 and ETTm2
- Gains concentrate where the dominant cycle is longer than 96 steps. Pooled over ECL, Traffic, and Weather, TRACE is 18.6% below RevIN and 8.6% below DDN, while on the four ETT sets the figures are 5.3% and 4.5% (computations are from Tables 25–28)
- At L = 336 (Table 4), TRACE is lowest in only 3 out of 16 cells, while DDN beats it in 13/16. Its mean MSE is 3.8% above DDN and 2.4% above RevIN. On the same 16 cells at L = 96, it is 4.2% and 5.1% below them
- On ETTh1-720 at L = 336, TRACE is 10.8% (DLinear) and 21.1% (iTransformer) worse than RevIN
- TRACE at L = 96 has higher peak training memory than RevIN at L = 336 in 7/8 configurations of Table 17 (for example, 475 vs 150 MB in DLinear on ECL)
-
The paper's own ablations undercut two design choices.
- The learned gate seems to be unnecessary. A single weight fitted on validation is as good as the learned gate: it has lower MSE in 14 of 24 cells, and 1.4% lower on average (Table 8)
- Components longer than the window hurt. Removing them and using one fixed weight raises the median gain on the factor cells from 1.36% to 8.48%, winning 10 of 12 cells (Tables 3, 9, 10). On ETTh1 at H = 720, this simpler variant reaches 0.431–0.436 MSE, against 0.484–0.528 for TRACE in the same setting (Table 22). The deployed ETTh1 filter still keeps these components, which raise its forecast error from 0.446 to 0.502 (Table 11)
Questions for authors
- Lengthening the window from 96 to 336 lowers RevIN's and DDN's mean MSE by 7.2% and 7.6%, but leaves TRACE's unchanged (mean is 0.0%, median cell is −2.7%). RevIN and DDN at L = 336 therefore already match or beat TRACE at L = 96, except on Weather (Table 4). This suggests the L = 96 advantage is mostly the value of history beyond 96 steps, which a longer window also provides, rather than an advantage of carrying a state. Please report every method at its validation-selected on at least ECL, Traffic, and Weather, with all four backbones
- CycleNet's RCF [1] is a plug-in that supplies periodic structure the window cannot see, which looks like a close prior work that should be discussed and added to comparison
- TRACE's learning rate and K are selected on validation from 12 settings per backbone–dataset pair (4 learning rates x , Appendix G, Table 21), while the baselines keep their published settings. Appendix N shows that tuning the learning rate alone improves RevIN by up to 6.25% on the six pairs checked, none of them ECL or Traffic. Please give every adapter the same learning rate search
- The paper mentions that iTransformer is run without its built-in normalization (Appendix G). However, this makes it weaker than the published model, which normalizes its input. Without normalization, iTransformer is 40% worse than its published L = 96 results (ETTh2 +119%, ETTm2 +98%, Traffic +33%). Against the published iTransformer (with normalization), TRACE's gain is 4.3% (about 0 on ETTh1/ETTh2), not the ~32% that Table 1 implies. Please report Table 1 with each backbone's own normalization
[1] Lin, S., et al. "CycleNet: Enhancing time series forecasting through modeling periodic patterns." Advances in Neural Information Processing Systems 37 (2024), 106315-106345
6 comments
W2. Long components and the gate. The submitted factor comparison favors short components plus a validation-fitted scalar in 10/12 cells (median gain 1.36% → 8.48%). ETTh1's retained periods 650/524 also raise standalone error from 0.446 to 0.502. We therefore tested full/short × gate/scalar under main-table training, retaining the learning rates and using three seeds.
Here short removes every period and reselects noise ratios on the same fitting split. The jointly trained scalar tests an end-to-end alternative, not the validation-fitted scalar in the factor study. Both variants retain exact backbone initialization and one-stage training. Three-seed mean MSE:
| Backbone / data / H | Full+gate | Full+scalar | Short+gate | Short+scalar |
|---|---|---|---|---|
| DLinear / ETTh1 / 720 | 0.534 | 0.485 | 0.497 | 0.499 |
| TSMixer / ETTh1 / 720 | 0.585 | 0.622 | 0.540 | 0.585 |
| MICN / ETTh1 / 720 | 0.504 | 0.480 | 0.513 | 0.495 |
| iTransformer / ETTh1 / 720 | 0.506 | 0.512 | 0.530 | 0.532 |
| DLinear / ECL / 96 | 0.155 | 0.161 | 0.166 | 0.171 |
| DLinear / ECL / 720 | 0.220 | 0.223 | 0.229 | 0.232 |
| DLinear / Traffic / 96 | 0.516 | 0.520 | 0.508 | 0.540 |
| DLinear / Traffic / 720 | 0.544 | 0.551 | 0.544 | 0.569 |
| TSMixer / ECL / 96 | 0.152 | 0.151 | 0.152 | 0.151 |
| TSMixer / ECL / 720 | 0.213 | 0.215 | 0.213 | 0.215 |
| TSMixer / Traffic / 96 | 0.452 | 0.454 | 0.455 | 0.458 |
| TSMixer / Traffic / 720 | 0.511 | 0.519 | 0.521 | 0.520 |
Short equals full in the two TSMixer/ECL cells. Short+scalar wins 3/12 cells, with a 2.1% mean relative MSE increase. On ETTh1 H720 the deployed full-state gate is best for only iTransformer: full+scalar is 9.2% lower for DLinear (the only main-benchmark cell where submitted TRACE is worse than the plain backbone, 0.534 vs 0.514) and 4.8% for MICN, and short+gate 7.7% lower for TSMixer. With the gate, removing long components raises MSE in 5 of the 6 ECL/Traffic cells that contain them, at full precision (one increase is below 0.05%); the exception is DLinear/Traffic H96 (1.6% lower). These tests do not establish a rule for selecting harmful components in advance.
With full state, the trained scalar wins 3/12, with a 0.2% mean relative MSE increase. This differs from post-hoc fitting (Table 8: scalar wins 14/24, mean 1.4% lower MSE; the expanded checkpoint comparison includes a 24.0% gate disadvantage on TSMixer/ETTh1 H720). The gate's distinct role is state-conditioned weighting. We will report these protocols separately and emphasize the persistent branch as the contribution.
Q2. CycleNet's RCF. We combined RecurrentCycle with RevIN at (112 cells × 3 seeds, public RevIN lr). Cycle lengths in observation steps are 24 for ETTh, 96 for ETTm, 144 for Weather and 168 for ECL/Traffic. RCF improves RevIN in 100/112 cells (mean 1.7%); TRACE improves on RevIN+RCF in 98/112 (8.2%). RCF supplies a learned periodic pattern; TRACE updates its state from observations. Representation/mixing differ and this comparison uses public-lr RCF, so this is a complete-method comparison. We will add RCF to Secs. 2 and 4.1.
Q3. Equal learning-rate search. We retuned RevIN, DDN, GAS-Norm, SAN and FAN on 112 cells at . We used TRACE's four lr candidates and selected by mean validation rank across horizons. SAN/FAN also retained their original lr as a candidate; SAN's adapter lr and GAS-Norm's selected were fixed. Each selected setting has three seeds. Public → retuned values are given for each baseline; the GAS-Norm port has no public-lr entry.
| Baseline | TRACE lower / 112 | Mean cellwise reduction | Overall-mean reduction | Baseline's own mean cellwise gain from retuning |
|---|---|---|---|---|
| RevIN | 105 → 101 | 9.7% → 6.3% | 10.66% → 6.55% | 3.7% (82/112) |
| DDN | 97 → 92 | 5.8% → 4.9% | 6.10% → 5.71% | 0.9% (76/112) |
| SAN | 101 → 91 | 9.7% → 4.9% | 16.11% → 5.38% | 5.3% (75/112) |
| FAN | 108 → 104 | 11.6% → 9.7% | 13.78% → 11.94% | 2.1% (88/112) |
| GAS-Norm (our port) | 98 | 11.1% | 15.39% | — |
The GAS-Norm row includes ETTh2, where the port trails public-lr RevIN; excluding ETTh2, TRACE is lower in 82/96 cells by 8.1%. The shrink is largest for SAN, whose selected lr differs from the published one in 22/28 backbone–dataset pairs; its overall-mean shrink comes mainly from iTransformer/Traffic, where the published lr gives MSE 1.36–1.41 against 0.46–0.53 after retuning. ETTh1 favors retuned SAN (TRACE's mean relative increase: 1.4%). Separately, retuning RCF gives TRACE 78/112 wins (2.5% mean reduction).
Thank you for the detailed response.
Resolved:
- Q3, equal learning-rate search. RevIN, DDN, SAN and FAN were retuned on all 112 cells with TRACE's four learning rates. TRACE is still lower in 91–104 cells
- Q4, iTransformer's own normalization. The 4.5% reduction (25/28 cells) agrees with my estimate of 4.3%
- Q2, CycleNet's RCF. After retuning, RevIN + RCF is the closest baseline at L = 96. TRACE is lower in 78/112 cells
Partly resolved (W2):
- Long components. On ECL and Traffic, the new table is convincing: removing them raises MSE in 5 of the 6 cells that contain them. For DLinear and TSMixer, the best of the four trained variants (0.485, 0.540) is still above the best public-setting adapter (SAN 0.470, DDN 0.507 from Tables 25 and 27)
- Gate. A jointly trained scalar is within 0.2% of the gate. The validation-fitted scalar is still lower in 14/24 cells. So the gate does not harm, but nothing shows that state-conditioned weighting improves accuracy. I agree with presenting the persistent branch as the contribution
Not resolved (W1/Q1):
The experiment was run as requested, and it confirms the concern:
- With the input length selected on validation, TRACE has 2.1% higher MSE than SAN (lower in 19/40 cells) and 1.4% higher than DDN (15/40). It is 0.9% lower than RevIN (28/40) and 2.4% lower than FAN (22/40), but RevIN and FAN were not retuned in this experiment
- TRACE needs 2.2x the peak training memory of RevIN (L=96) and 1.3x memory of DDN (L=720)
At a fixed L = 96, TRACE is 5–7% better than retuned RevIN, DDN and SAN, and 2.5% better than RevIN + RCF. When the input length is tuned, it has no average advantage.
I am keeping my current score. The numbers are now reliable, but every fair-comparison control shrinks the effect, and with a tuned input length it is about zero.
Thank you for raising the input-length comparison.
W1/Q1. Validation-selected inputs. Across five datasets (ECL, Traffic, Weather, ETTh1, ETTm2), four backbones and H96/H720, selected SAN improved over its own L=96 result in 36/40 cells (mean reduction 11.0%). When TRACE selected from 96, 336, 512 and 720 and SAN from 96, 336 and 720, TRACE had 2.1% higher MSE on average (TRACE lower in 19/40); with three lengths for both methods, it was 2.5% higher across 40 cells (TRACE lower in 17/40). Both comparisons favor SAN on average. In the four-length comparison, TRACE was lower in 8/8 Traffic and 5/8 Weather cells but only 2/8 ECL, 3/8 ETTh1 and 1/8 ETTm2 cells, with 6.5% and 5.7% higher mean MSE on ETTh1 and ETTm2.
For each backbone–dataset pair, we selected L by mean validation rank across both horizons at seed 2021. Both comparisons exclude 512 for SAN because its segment length (24 or 12) must divide L. For SAN and FAN, we then selected the learning rate at the chosen L and evaluated three seeds.
Four-length TRACE uses three-seed results at its selected configuration: the reselected lr at the chosen long input in 32 cells and the main-table lr at L=96 in eight. Three-length TRACE uses three seeds in 32 cells (3.4% higher MSE than SAN, 12/32 wins) and one in eight. Substituting three-seed results from a different L or K in those eight gives 2.4% higher MSE (20/40). Against FAN, which also supports 512, TRACE selected from the same four lengths won 22/40 cells with 2.4% lower MSE on average.
Earlier three-seed controls also depend on protocol. Selecting among four lengths at original learning rates gives TRACE 28/40 wins over RevIN (0.9% mean reduction) and 15/40 over DDN (1.4% higher MSE). Selecting among three long inputs (336, 512, 720) before retuning gives 25/40 and 19/40 wins, with mean reductions of 1.3% and 0.4%.
For excess MSE over SAN, we compute 100 × (TRACE − SAN)/SAN from seed-averaged MSEs in each cell, then average equally over the 40 cells (32 for the three-seed subset). The abstract and introduction will scope the headline to fixed L=96: across 112 cells, reductions in overall mean MSE are 10.66% vs RevIN and 6.10% vs DDN, or 6.55% and 5.71% after baseline lr retuning. TRACE carries state without extending the backbone input. Across 16 H720 settings, its median peak training memory at L=96 is 2.2× RevIN's and 1.3× DDN's at L=720; its median latency ratio vs DDN is 0.52×.
Q4. iTransformer's own normalization. With native normalization enabled, TRACE retains a 4.5% mean MSE reduction over iTransformer and is lower in 25/28 cells (3 seeds). We will include this comparator explicitly in Table 1 and label the unnormalized plain model as an ablation. The native result differs from the submitted RevIN comparison (cellwise 4.3%, 24/28): native normalization has no affine parameters, while RevIN does. The large gap against the unnormalized model will not be presented as TRACE's gain over the published backbone.