- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes FlowTSFM, a 38.8M-parameter encoder-based time series foundation model that applies a single weight-shared Transformer block K times as an Euler-style residual update and decodes a 99-quantile forecast at every intermediate state. It is trained with a quantile-flow objective combining pinball losses at final and interior exits, a soft anchor to a Gaussian source, and a straight-line path term toward the detached final forecast, alongside a randomized-depth variant sampling K from 3 to 15. On GIFT-Eval and TIME, the model trails Chronos-2 by 3% to 5% and TiRex by 1% to 2%, while a 5x5 train-by-inference depth grid shows fixed-depth models specialize to their training budget whereas randomized depth improves up to 9 steps.
Critical strengths
The paper provides a complete and transparent depth study across 100 grid cells and a randomized-depth checkpoint, showing clearly that fixed-depth models degrade by 10% to 89% in CRPS off their training budget while randomized depth reduces error by 26% to 29% at 6 steps compared to a shortened 12-step model, and candidly reporting degradation beyond the training range at 18 steps.
The design is clean and honestly scoped, reaching within 3% to 5% of Chronos-2 using a single shared block plus one pre-recurrent layer, while clearly stating the theoretical boundaries of Proposition 1 and experimental limitations.
Critical weaknesses
The efficiency claim confuses parameter count with compute, as the 38.8M model applies a width-1024 block 12 times over 512 patches, costing roughly as much compute as a 12-layer untied encoder with 150M non-embedding parameters which exceeds Chronos-2, while reporting no FLOPs, latency, or memory and losing on all four metrics to the similarly sized 35.3M TiRex.
The test-time compute knob has a small dynamic range and incomplete evaluation, since increasing compute by 3x from 3 to 9 steps improves error by only 3% to 5% before plateauing and the 4-step specialist is only 2.3% to 3.0% behind the 12-step one; moreover, the randomized checkpoint is compared with specialists at only 2 of 5 budgets, namely 6 and 12 steps while omitting 4, 8, and 10, performs 0.4% to 2.0% worse on all four scores at the base 12-step budget, and relies on a straw-man shortened 12-step baseline that feeds unseen step sizes instead of evaluating true early exit.
Questions for authors
Can you report the accuracy of intermediate exits across steps k alongside a true early-exit baseline, add geometry ablations comparing non-straight paths or simple consistency losses, and test whether divergence without the path term persists under lower peak learning rate, longer warm-up, QK-norm, or gating?
Can you report FLOPs, inference latency, and memory for K from 3 to 15 against Chronos-2, TiRex, and Moirai-2 to substantiate the efficiency claims under actual compute?
Why was the randomized-depth model evaluated at steps 3, 9, 15, and 18 instead of matching the specialists at 4, 8, and 10, and can you report its performance at 4, 8, and 10 steps as well as K=1 and K=2?
2 comments
Thank you for the detailed new compute and depth experiments, which address my main concerns. I am updating my rating from 2 to 3.
Thank you for a very concrete review. Each question pointed to a clear experiment, and we ran each of them. On compute you were right, and we have corrected the claim.
W1 and Q2 · Parameters versus compute
One H200, batch 1. Appendix J.2 lists every K from 1 to 15.
* TiRex needs its own CUDA kernel, so it was timed in a separate environment, with FlowTSFM and Chronos-2 as controls (42.5 vs 13.1 ms). The Chronos-2 ratio is the same in both sessions.
Each step costs about 0.55 ms and 14 GFLOPs. TiRex has the fewest FLOPs but is the slowest, because it recurs over time, token by token; it also reads only the last 2,048 points, where we use all 8,192.
W2 and Q3 · The randomized model at every budget
We had picked 3, 9 and 15 as the ends and middle of the training range, which left out the budgets where specialists exist. Table 3 now covers K = 1 to 18. GIFT-Eval / TIME MASE:
At K = 4, 8 and 10 the single model beats the specialist on 9 of 12 scores (MASE and CRPS) and is never more than 1% worse. At K = 12 it is 0.4% behind on MASE, within seed noise, and 2.0% on GIFT-Eval CRPS: the price of one model for all budgets.
We agree the useful range starts near K = 4. Our claim is one checkpoint for every budget, not that depth buys much beyond six steps.
W2 and Q1 · True early exit
We now stop one 12-step run after k steps, with the training step size (Appendix J.4, Figure 1). The raw read-out is poor at small k. Extrapolating along the path, q^=q0+(qk−q0)K/k, where q0 is the forecast decoded before the first step, needs one extra decoder read and no extra block. GIFT-Eval MASE:
Extrapolation removes 26–30% of the error at k = 3–4, and 41–42% for the path-only model. For a small budget, though, running the randomized model at K = k remains the better choice, and we say so.
Q1 · Divergence under lower LR, longer warm-up, QK-norm or gating
Lower LR with a longer warm-up, a latent stabilizer and QK-norm all make the no-path run stable, but each costs accuracy and loses the trajectory:
The gate is per-channel on every update (g=2σ(a), starting at 1). Among the options we tested, the path term is the only one that is both stable and accurate.
Q1 · Geometry ablations
Endpoint distillation keeps the final GIFT-Eval accuracy but loses the trajectory (4-step exit 2.85). A consistency loss between exits is 0.9 / 1.9% worse, again with no usable early exit (2.82). A curved path matches the final accuracy and, extrapolated along its own curve, exits better than the straight one (0.85 vs 0.96). So the experiments do not show that the path must be straight, only that prescribing one helps; the paper now discusses the choice of path.
If these results address your concerns, we would be grateful if you would consider raising your score.