- Rating
- 1
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
I appreciate the authors' effort on this work. I have read the paper carefully, and below is my summary of the paper and its contributions: The paper introduces Reconstruction Advance Prediction (ReAP), a plug-in-and-play training paradigm for LTSF, inspired by Dynamic Mode Decomposition (DMD). Rather than training a model solely to map a historical window to a disjoint future window , ReAP modifies the model's output during training to jointly reconstruct an overlapping tail of the historical window alongside the future forecast. To encourage the neural network predictor to align with the underlying dynamical transition operator while avoiding the unstable computation of the Moore-Penrose pseudoinverse , the authors replace the theoretical operator error with a transpose surrogate consistency loss. For the experimental part, they evaluate the mechanism ReAP on a single linear layer to form (ReLinear), and integrate into 5 different backbones, reporting consistent error reductions, improved performance and also report ReLinear's competitiveness with SOTA Models. Lastly, the authors conduct experiments on the last section for scaling-law where they claim that baseline degrades as the history length goes from 96 to 720 while ReLinear+ (trained with ReAP) improves monotonically.
Critical strengths
Here I listed some bullet points that I believe are the strengths of the paper:
- (S1) Promising Direction/Importance of the paper: LTSF is an impactful research field. The core direction of this paper of moving beyond simple disjoint past-to-future mappings by explicitly leveraging the overlap window to capture underlying system dynamics through Dynamic Mode Decomposition (DMD) is well-motivated and conceptually appealing.
- (S2) Flexibility: Formulating ReAP as a training-time paradigm makes it flexible to integrate into existing deep learning backbones.
- (S3) Substantial Experimental Effort and Reproducibility: I appreciate the authors' extensive effort in conducting experiments across seven standard real-world benchmarks, multiple prediction horizons, and different baseline families. Furthermore, providing detailed sets of hyperparameter tables ( and ) configurations across datasets and horizons in Appendix C demonstrates transparency for reproducibility.
Critical weaknesses
I really appreciate the authors' effort on the paper. Here are some bullet points on the weaknesses of the paper:
Theoretically:
-
(W1): I am not convinced the gain comes from reconstructing the overlap. In the released code, ReLinear is a single
nn.Linear(seq_len, pred_len + label_len)applied along time, with no normalization layer and no parameters shared across output steps. So Eqs. 27 and 28 are sums over output steps, and the overlap columns share no weights with the future columns. They cannot change the evaluated horizon, and with Adam, loss normalization only rescales . What is left, up to normalization, is . This is a Gram-weighted squared error, not a DMD operator. In the experimental part, the tables point the same way: in Tables 9-12, ReLinear without the consistency loss is about on par with DLinear, and the pattern of the gain from the consistency loss seems to match what a regularizer would do. -
(W2) I could not find a theorem that supports the central claim. The abstract says reconstructing the overlap "can enhance prediction performance", but nothing compares training with and without the overlap term, and there is no risk or generalization statement. Prop. 1 is the standard consistency of least squares, and it assumes stationarity, while Sec. 5 itself points to non-stationary dynamics. Lemma 1 is an exact SVD identity, and its interpretations are informal remarks. Prop. 2 bounds the surrogate against , but Eq. 17 is , which is a different quantity, and the paper never connects the two. The bound's constant is also , so it is weak for ill-conditioned windows.
Empirically:
- (W3): The paper reports no seeds, standard deviations, or repeated runs, and many gains are small so they could be noise. ReAP is grid-searched over and for each model, dataset and horizon, with ranging from to across Tables 3, 4 and 7 (about 13 orders of magnitude and about 7 for ReLinear alone), while the baselines use their official settings. The baselines also miss recent strong models, such as OLinear and TimeMixer++, and several works the paper cites are not compared, including Koopa, PatchTST, FITS(Citations are included below).
References
- Yue, W., Liu, Y., Wang, H., Li, H., Ying, X., Guo, R., Xing, B., Shi, J. OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain. NeurIPS 2025. arXiv:2505.08550.
- Wang, S., Li, J., Shi, X., Ye, Z., Mo, B., Lin, W., Ju, S., Chu, Z., Jin, M. TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis. ICLR 2025. arXiv:2410.16032.
- Liu, Y., Li, C., Wang, J., Long, M. Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors. NeurIPS 2023.
- Nie, Y., Nguyen, N. H., Sinthong, P., Kalagnanam, J. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. ICLR 2023.
- Xu, Z., Zeng, A., Xu, Q. FITS: Modeling Time Series with 10k Parameters. ICLR 2024.
Questions for authors
Questions for Authors
- (Q1) In ReLinear, the overlap columns share no weights with the future columns. If you train with only the future outputs plus (no overlap reconstruction), do you get the same gain?
- (Q2) Does ReAP beat ordinary regularization of the same linear layer (weight decay, ridge) when each is tuned with the same budget as ? Can DLinear get the same search over learning rate and weight decay?
- (Q3) Can you report mean and standard deviation over several seeds for ReLinear and ReLinear+, since many gains are small?
- (Q4) Is there any result, even for a linear model, showing that adding the overlap term lowers the forecasting risk? Prop. 2 bounds , but Eq. 17 is . How do you connect the two?
- (Q5) Were and selected on validation or on test? Can you also compare against OLinear, TimeMixer++, Koopa and PatchTST under the same tuning?
3 comments
###Part 2 (W3–Q2)
W3. Multi-Seed Evaluation, TimeMixer++, & Tuning Protocol
- TimeMixer++ Clarification: Our experiments use
Time-Series-Library/models/TimeMixer.py, which implements the TimeMixer++ architecture (identical to the official TimeMixer codebase); thus, the results reported as "TimeMixer" in our paper are actually TimeMixer++. We will update the model name and clarify this explicitly in the revised manuscript. - 3-Seed Runs Across 5 Backbones: Across 3 random seeds and all 7 datasets (35 pairs; full results in Appendix H), ReAP achieves mean reductions of 2.99% (MSE) and 2.37% (MAE), substantially exceeding run-to-run standard deviations (0.27 pp and 0.16 pp):
| Metric | DLinear | PatchTST | Koopa | OLinear | FITS |
|---|---|---|---|---|---|
| MSE Improve (%) | |||||
| MAE Improve (%) |
Q1. Future-Only () vs. Overlap Reconstruction ()
No—setting (Linear+) does not reproduce ReAP's gains and degrades long-horizon scaling. Averaged across all 7 datasets and prediction horizons (Appendix I), enforcing without overlap (Linear+) worsens MSE by -2.50% () and -5.51% () vs. vanilla Linear, whereas ReLinear+ () outperforms Linear+ by 8.54% MSE (11.70% at ):
| Comparison | Metric | Across Input Lengths | ||||
|---|---|---|---|---|---|---|
| ReLinear+ vs. Linear+ | MSE / MAE | 6.45 / 5.95 | 5.38 / 4.78 | 10.62 / 9.61 | 11.70 / 9.61 | 8.54 / 7.49 |
| ReLinear+ vs. Linear | MSE / MAE | 6.49 / 5.05 | 6.29 / 5.15 | 8.44 / 6.97 | 7.32 / 5.85 | 7.14 / 5.76 |
| Linear+ vs. Linear | MSE / MAE | 0.05 / -0.98 | 1.07 / 0.38 | -2.50 / -3.11 | -5.51 / -4.32 | -1.72 / -2.01 |
Consistent with Lemma 1, maximizes the shift lag ; at large , temporal decorrelation causes to become ill-conditioned, making the unanchored consistency penalty noisy. Reconstructing the overlap () shortens and anchors the trajectory to the historical boundary.
Q2. Matched-Budget Ridge, Weight Decay, & Learning Rate Sweeps (ETTh1, )
Under a fixed random seed, batch size , and official (or for LR sweeps), ReLinear+ (MSE = 0.384, MAE = 0.398) and DLinear+ (MSE = 0.382, MAE = 0.394) outperform all 17 baseline configurations:
1. Explicit Ridge Sweep (, ):
| Model | Metric | ||||||
|---|---|---|---|---|---|---|---|
| Linear+Ridge | MSE / MAE | .413 / .424 | .412 / .423 | .411 / .423 | .410 / .423 | .412 / .424 | .446 / .445 |
2. Adam Weight Decay Sweep ():
| Model | Metric | ||||||
|---|---|---|---|---|---|---|---|
| Linear | MSE / MAE | .400 / .415 | .400 / .415 | .400 / .415 | .400 / .415 | .400 / .415 | .410 / .422 |
| DLinear | MSE / MAE | .392 / .408 | .392 / .408 | .392 / .408 | .392 / .408 | .392 / .408 | .397 / .415 |
3. Learning Rate Sweep ():
| Model | Metric | |||||
|---|---|---|---|---|---|---|
| Linear | MSE / MAE | .603 / .517 | .433 / .437 | .400 / .415 | .396 / .401 | .395 / .398 |
| DLinear | MSE / MAE | .411 / .422 | .397 / .400 | .392 / .408 | .387 / .399 | .389 / .402 |
Unlike isotropic penalties () that shrink weights uniformly toward zero (degrading at or ), anchors transitions to empirical cross-covariances along dominant singular modes while preserving forecast amplitude.
Part 3 (Q3–Q5)
Q3. Multi-Seed Evaluation of ReLinear+ vs. ReLinear (, 3 Seeds)
Averaged across (Appendix J), ReLinear+ reduces dataset-averaged MSE by 4.32% () and MAE by 3.32% () with lower variance (mean std 0.0029 vs. 0.0049 for MSE):
| Model | Metric | ETTh1 | ETTh2 | ETTm1 | ETTm2 | Weather | ECL | Traffic |
|---|---|---|---|---|---|---|---|---|
| ReLinear+ | MSE | |||||||
| ReLinear+ | MAE | |||||||
| ReLinear | MSE | |||||||
| ReLinear | MAE |
Q4. Lowering Linear Risk & Connecting Eq. 17 to Proposition 2
- Lowering Linear Forecasting Risk: Across all 7 datasets and horizons , ReLinear+ () reduces error over vanilla Linear by 7.14% MSE (5.76% MAE) and over Linear+ () by 8.54% MSE. Theoretically, Lemma 1 proves shortens , keeping well-conditioned and lowering operator estimation variance.
- Connecting Eq. 17 () and Proposition 2 (): Contracting across shared variates in the transpose space resolves the temporal dimension mismatch when . On the rank- SVD subspace (), both losses bound the identical projected residual via and with shared zero-error optimum (), while weights modes by to damp noisy tail modes.
Q5. Validation-Only Selection & Backbone Comparisons
All hyperparameters are selected strictly on validation set MSE, with the test set held out for a single final evaluation under identical chronological splits and history–future boundaries. Under the exact same validation-only tuning protocol, ReAP improves OLinear, Koopa, PatchTST (Part 1, W3), and TimeMixer++ (Table 7).
We thank Reviewer 3 for your insightful comments and careful code inspection. Due to the 5,000-character limit per comment, we split our response into Part 1 (W1–W2), Part 2 (W3–Q2), and Part 3 (Q3–Q5).
W1. Overlap Coupling, Surrogate Role, & Ablated ReLinear
ReLinearasnn.Linear(seq_len, pred_len + label_len)forces a single linear map fromseq_lento jointly predict thelabel_lenoverlap window (which reconstructs the tail ofseq_len) together withpred_len. Because L^sur is computed over the concatenated trajectorylabel_len + pred_len, thepred_lenoutput weights are directly coupled to thelabel_lenreconstruction weights rather than being learned independently (Lines 287–288; Lemma 1).ReLinearMatchesDLinear(Tables 9–12):ReLinearin Tables 9–12 is specifically an ablated baseline trained without L^sur (α=0) to isolate the contribution of the consistency loss. Because α=0 reducesReLinearto a standard linear projection, matchingDLinearis expected, while ReLinear+ (α>0) outperforming both directly confirms the effectiveness of dynamical consistency.W2. Theoretical Chain, Eq. 17 vs. Prop. 2, & Condition Ratio (λ1/λk)2