- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work compares SCALEIN, which computes the loss against scaled targets, with SCALECON, which first restores predictions to their original scale and then computes the loss. It shows that SCALECON implicitly prioritizes the optimization of high-scale series by introducing scale-dependent weights into the gradients. The work shows, both theoretically and empirically, that SCALEIN removes this scale-driven training bias under the stated scaling assumptions and generally improves forecasting performance as measured by MASE and WQL, which account for differences in scale.
Critical strengths
- The paper is clearly written, and its figures, together with the explanations, make the scale-driven training bias and the effect of SCALEIN easy to understand and follow.
- The controlled synthetic and real-data scale-swap experiments are well designed and provide convincing empirical support for the claims.
- Comparisons across architectures and benchmarks, covering both time series foundation model pretraining and supervised forecasting baselines, generally show improvements in MASE and WQL, which account for differences in scale. These results show that the proposed loss-space choice is not only theoretically motivated but also empirically effective on real-world forecasting benchmarks.
Critical weaknesses
- The experiments on time series foundation models use small model configurations and a fixed training budget, leaving the benefits with larger models and longer pretraining unverified. Thereby, the comparisons do not fully distinguish faster convergence from better forecasting performance after sufficient optimization. It remains unclear whether SCALECON could narrow the gap with longer training.
- The normalized-target training strategy and its scale-balancing motivation have already been explored in prior work, as acknowledged in the paper. The main contribution therefore lies in a systematic analysis of the mechanism and broader empirical validation. While these contributions are important, the novelty of the training method itself is limited.
- The empirical validation in multivariate settings is limited. Although the theory allows cross-channel dependencies under channel-wise scaling, the practical benefits in joint multivariate forecasting are not separately demonstrated.
Questions for authors
Overall, the clear explanations and systematic analysis make me lean toward acceptance. The weaknesses above are my main concerns, and addressing them would have the greatest impact on my assessment. I also have the following additional questions for clarification:
- The evaluation primarily relies on scale-normalized metrics. Could the authors also report original-scale MAE and RMSE to clarify the benefits and limitations of SCALEIN under these metrics?
- I agree that SCALEIN addresses an important issue, which led me to look at how tabular foundation models are trained for regression. Based on the open-source training code I have seen, most models already compute regression losses on normalized targets. Do the authors expect scale-invariant training to be similarly important for tabular regression, and could the paper’s analysis be extended to that setting?
4 comments
Response (2/2)
W3
We run an offline analysis of 8 multivariate datasets that we used in Section 4.2 by computing per-channel normalization statistics and comparing their scale across channels. For every training window, we compute the per-channel standard normalization scale (std. dev.) and measure each channel's share of the total ScaleCON loss weight (MSE) across all windows. Under ScaleIn each channel gets the same weight, but under ScaleCON per-channel loss weights can widely differ. For example, the top 10% of channels account for the following percentages of the scaling weight (offline analysis): Electricity (321 ch.) 100%, FRED-MD (107 ch.) 100%, Wind Farms (308 ch.) 62%, Taxi (1214 ch.) 44%, KDD Cup 2018 (269 ch.) 40%, Pedestrian (26 ch.) 37%, Solar (137 ch.) 35%, Traffic (862 ch.) 32%. In other words, in a jointly forecast multivariate series, ScaleCON concentrates a large part of its loss weight on a small subset of channels.
For a brief empirical evaluation, we trained PatchTST (channel-independent) and iTransformer (channel-mixing) on Electricity (321 channels) with both losses (context 64, horizon 16, 3 seeds). We report the relative median MASE change of ScaleIN with respect to ScaleCON ( MASE (%), lower is better), and the share of the gradient signal that goes to the 10% of channels with the largest gradients. For ScaleCON, these channels receive almost the entire gradient signal (99.6%), starving the remaining 90%. ScaleIN weights all channels equally a priori. However, in practice, harder-to-forecast channels receive more gradient (in this example 28%), so the gradient is allocated by forecast error rather than by scale.
| Model | MASE (%) | ScaleIN grad share | ScaleCON grad share |
|---|---|---|---|
| PatchTST | -12.6 | 28% | 99.6% |
| iTransformer | -15.5 | 28% | 99.6% |
Q1
We thank the reviewer for this suggestion. We computed the original-scale MAE and RMSE for all runs in Table 1, aggregated per benchmark in the same way as MASE and WQL. ScaleIN achieves lower MAE in 23/24 architecture-benchmark comparisons and lower RMSE in 18/24. We added the full table to the appendix.
We note that metrics in raw scale should be interpreted with care. Averaging errors across series of heterogeneous scales weights each series by its magnitude, so a few high-scale series may dominate the result (e.g., series concerning electricity demand forecasting in Wh units will have much higher magnitude than temperature forecasting series). Even though these metrics apply scale-dependent weighting, ScaleIN still wins most comparisons under them, which suggests that its benefits are not an artifact of scale-free evaluation.
Q2
Our focus is TSFMs, but Lemma 3.1 holds for any affine target normalization with a homogeneous loss, including per-dataset target standardization in tabular regression. The bias arises only when the training targets span heterogeneous scales. Forecasting PFNs (e.g., ForecastPFN, TabPFN-TS, TempoPFN, ApolloPFN, Nori) train solely on synthetic data, where the scale range is a design choice of the prior and can be bounded during generation, perhaps making the loss-space choice less consequential. In contrast, all TSFMs in Table 3 train on real-world data, alone or mixed with synthetic data, so scale heterogeneity is inherited from the corpus and cannot be controlled at generation time. We have expanded Appendix A with this discussion on forecasting PFNs.
We thank the reviewer for suggestions, which have helped us strengthen the empirical evaluation. We will share the remaining results concerning longer PatchTST and TimesFM as they become available in about two days. We would be happy to discuss any further questions.
Full 500k-step TSFM results
Dear Reviewer mSgo,
We're following up with full 500k steps results for the TSFM experiment. We revised the paper to report the completed 500k-step runs as the main TSFM results, replacing the 80k runs (changes in blue).
ScaleIN lowers MASE in 23/24 comparisons (was 24/24) and WQL in 21/24 (was the same). Average reductions are 10.4% on GIFT-Eval and 12.7% on the M-competitions (was 18.8% and 21.9% at 80k).
Below is the relative MASE percentage change over the six benchmarks (ScaleIN relative to ScaleCon, lower is better):
| 80k | 125k | 250k | 375k | 500k | |
|---|---|---|---|---|---|
| Chronos-2 | −16.2 | −11.7 | −5.1 | −4.5 | −5.5 |
| Moirai 2.0 | −7.1 | −6.7 | −6.9 | −5.6 | −6.0 |
| TimesFM-2.5 | −14.0 | −12.5 | −11.5 | −9.3 | −7.4 |
| PatchTST | −35.8 | −32.6 | −28.1 | −23.8 | −23.0 |
As evidenced, ScaleIn converges faster (Figure 5b) and retains a consistent advantage at a six times longer budget and ScaleCon does not catch up.
Thanks for running the longer experiments. The training budget was my biggest concern, and the new results address it. It would be great to note in the main paper that the gap narrows with more training. Even with a smaller gap, I think reaching good forecasting accuracy sooner is valuable in its own right, given the cost of pretraining foundation models.
I’m raising my score. I like papers that take a familiar design choice and show why and how it matters. This one made me think more carefully about how the choice of loss space affects which series drive training.
For the area chair, my confidence rating of 2 reflects my limited direct experience with time-series foundation models. I’ve worked on time-series models and on foundation models in other domains.
Response (1/2)
We thank the reviewer for their feedback and are glad that they found our controlled synthetic and real-data scale-swap experiments are well designed and provide convincing empirical support for the claims. We have addressed the comments below and paper edits are reflected in red in the updated paper.
W1
Our models range from 1.5 to 49.6M parameters and are pretrained with an 80k optimization-step budget. We did not test larger model configurations, but the bp weighting (Lemma 3.1) still applies to them. Beyond the empirical results, our analysis identifies an implicit weighting scheme that should be explicitly considered in TSFM design.
To address the reviewer's concern about faster convergence, we continued pretraining for another 420k steps, raising the total budget ~sixfold to 500k steps. Reported below is the relative percentage reduction in MASE for ScaleIN over ScaleCon, averaged over the six benchmarks (lower is better). ScaleIN continues to outperform ScaleCON on every benchmark and at every budget. The gap narrows for Chronos-2 (from 16.2% to 5.5%) and stays stable for Moirai 2.0 (from 7.1% to 6.0%), and in both cases it plateaus after about 250k steps. TimesFM-2.5 and PatchTST runs are still in progress (est. ~1.5 day). We will update Section 4.3 of the paper with the full 500k results and notify the reviewer afterwards.
W2
We fully agree that computing the loss on normalized targets is not a new method. We aimed to be transparent about this by providing Tables 3 and 4 in Appendix A that survey the loss space choice (ScaleIn vs ScaleCon) across 24 TSFMs and popular forecasting codebases, which we also summarize in the introduction. To make this more explicit, we have moved Tables 3 and 4 (previously in Appendix A) to the main text. We would like to clarify that the aim of this work is to provide theoretical and empirical evidence and bring awareness to the fact that the choice of loss space has critical consequences for training and generalization in TSFMs and should be reported as a design decision to enable reproducibility. As represented by Table 3, the choice of loss space is only explicitly documented for 9 of 24 TSFMs, cannot be determined from the paper or code for 3 TSFMs, and no surveyed forecasting codebase makes it configurable.
Beyond prior empirical observations (Berthelier et al., 2026, Ahmed et al., 2026), we contribute the following conclusions, which do not follow from prior empirical comparisons. We have revised the related work to state these distinctions explicitly: