acceptodds
Under review as a conference paper at ICLR 2027

One Patch, Three Roles: What Is Actually Coupled in Autoregressive Time-Series Forecasting?

Abstract

Patch-based autoregressive time-series forecasting often ties input representation, learned transition span, and per-call output length to one patch length. We investigate whether these roles can be adjusted separately. For encoding, we explore building patches from short temporal atoms. We find that grouping preferences vary across data sets and model widths, but performance is more sensitive to width than to the number of atoms per patch on the evaluated grid. For decoding, we ask whether models can predict more subsequent patches per call. Lightweight parallel exits fit a parent’s recursive trajectory substantially more easily than the observed future, motivating Autoregressive Trajectory Distillation (ATD). ATD compiles the frozen parent’s rollout into selectable multi-patch execution. ATD-8 achieves a 5.54× end-to-end speedup with stable quality across widths in a paired four-data-set comparison. However, imitation can preserve the parent’s accumulated forecast error. We therefore ask whether this error can be mitigated independently of execution width. We find a correctable residual projection along a train-selected periodic history direction, motivating Spectrum Tangent. This correction adds no neural parameters or Transformer calls. At horizon 720, it reduces mean squared error (MSE) and mean absolute error (MAE) by 2.54% and 2.33% across seven data sets and two execution widths, retaining a 3.24× speedup over recursive inference. Experiments on three public autoregressive parents and four data sets demonstrate both methods’ transferability, with Tangent providing further long-horizon improvements.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series forecasting, autoregressive forecasting, long-horizon forecasting, trajectory distillation, inference acceleration, patch-based forecasting

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_WCWKOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper investigates three roles of patches in autoregressive forecasting: input representation, transition length, and execution width. It introduces ATD to generate multiple future patches per model call and Spectrum Tangent to correct accumulated forecasting errors using periodic history. Experiments report up to 5.54x faster inference with similar accuracy, alongside modest long horizon improvements.

Critical strengths

The separation of transition learning from execution width is useful and practically relevant. ATD provides substantial measured speedups while preserving forecasting quality. The matched comparisons distinguish trajectory imitation from actual prediction accuracy, and experiments with AutoTimes, Timer,and TimesFM provide additional evidence of transferability.

Critical weaknesses

main concern is limited methodological novelty. Multioutput decoding and trajectory distillation build on established approaches, while Spectrum Tangent is essentially a fitted correction toward a periodic history template. Its additional contribution over simpler seasonal correction methods is not clearly established. empirical claims also require qualification. The reported 5.54x speedup is measured on local models, without corresponding end to end timing on the three external forecasting backbones. Spectrum Tangent provides relatively modest average improvements and deteriorates on Exchange Rate, indicating sensitivity to data characteristics. The evaluation also relies on only three training seeds.

Questions for authors

Can the authors clarify how Spectrum Tangent compares with simpler seasonal or residual correction baselines under matched conditions? Do the reported inference speedups persist on the external pretrained models? Can the authors identify conditions under which the periodic correction is likely to degrade performance?

3 comments
AuthorsOpenReview ↗

Residual-correction baselines and public-parent speedups

Thank you for recognizing the value of separating transition learning from execution width. Under patch-role coupling (p0p_0/PP/KK), we conduct three targeted studies—two separate decouplings (local input scale and commit width) plus a correction motivated by ATD's inherited parent residuals—using minimal instruments to test whether local encoding can be fixed, frozen trajectories compressed, and inherited residuals corrected.

1. Contribution and correction baselines

Spectrum Tangent tests whether a single train-calibrated history direction reduces inherited residuals in closed loop, hence the uncorrected-forecast baseline. R2 adds your requested seasonal and residual baselines; Tangent achieves the lowest MSE on 4/5 datasets in both modes.

R2. MSE change (%) vs. uncorrected frozen serial Timer (W=672W=672, H=720H=720); first 96-point block protected. Closed loop writes predictions back; open loop acts on fixed proposals.

MethodModeETTh1ETTh2ETTm1WeatherExchange
History meanClosed+9.63+0.48+3.97−3.89+1.84
Last cycleClosed+5.11+0.54−13.54−0.69−13.04
SteerCastClosed−0.70+1.01−1.360.00−0.62
Spectrum TangentClosed−4.44−2.85−21.07−5.11+0.97
UEC-STDOpen+23.29+2.66−5.58−4.36+13.59
BOOSTOpen+2.33+45.72−3.45−3.04−10.39
Spectrum TangentOpen−5.40−4.00−20.34−6.29+0.29

Per series, history mean uses the lookback mean, last cycle tiles the final lookback cycle, and Tangent averages matching phases across cycles. All templates share the train-selected period, ramp, 384 training origins, and one fitted nonnegative scalar. SteerCast uses the matched mask, UEC-STD validation adaptation, and BOOST five Timer lags. UEC-STD/BOOST are evaluated open-loop as post-hoc baselines lacking a Timer patch-writeback rule; Tangent is shown in both modes.

2. Public-parent speedups

Yes. Native-path ATD-8 speedups hold fixed parent and cache policy: 2.87–3.54× (Timer), 2.09–2.11× (TimesFM), and 7.83–7.96× (AutoTimes).

R1. ATD-8 P50 speedup over serial parent (batch 128, W=672W=672, H=720H=720, RTX 5880 Ada): native KV reuse / cross-call KV disabled (AutoTimes uses native re-encoding).

ParentETTh1ETTh2ETTm1Weather
AutoTimes-GPT2— / 7.84×— / 7.85×— / 7.83×— / 7.96×
Timer-base-84M3.54× / 10.91×3.53× / 10.91×3.52× / 10.85×2.87× / 11.81×
TimesFM-2.5-200M2.09× / 8.84×2.10× / 8.83×2.10× / 8.87×2.11× / 8.79×

3. Gain size, consistency, and failure cases

We agree average gains are modest; Tangent's contribution is showing that long-horizon AR residuals retain exploitable structure along a simple periodic history direction. At H720, it lowers MSE by 2.54% over local ATD across all 42/42 runs (14 dataset–width configs × 3 seeds) with zero added parameters/calls and 3.24× pipeline speedup (Table 2b). R2 confirms this on frozen serial Timer (4/5 datasets, −2.85% to −21.07% MSE), showing a consistent yet bounded effect.

R4(a). Frozen serial Timer (H=720H=720) across chronological test blocks; ΔMSE (%) vs. uncorrected Timer.

DatasetBlockCosineFitted/break-evenFixed proposalsClosed loop
ETTh21+0.12820.69−1.40+0.23
ETTh22+0.06041.33+0.65+2.53
ETTh23+0.06471.49+1.21+5.16
ETTh24+0.36670.29−11.09−12.54
ExchangeAll (1–4)−0.4197—+0.29+0.97

On frozen serial Timer, the correction direction is negatively aligned with Exchange test residuals in all four blocks (pooled cosine −0.4197), raising closed-loop MSE by 0.97%. ETTh2 blocks 2–3 show positive alignment but step overshoot on fixed proposals; block 1 shows writeback reversing a fixed-proposal gain. In the submitted local ATD-4/8 Exchange runs, the training-selected correction (τ∗=12\tau^{\ast}=12, α∗≈1.50\alpha^{\ast}\approx 1.50 vs. 0.0850.085–0.4820.482 on the seven periodic sets) passed all six held-forward training checks but increased H720 test MSE at both widths. The revised PDF adds R1, R2, and R4(a) in Appendix C (Tables 23, 24, 26) and highlights Secs. 4.5, 4.6, and 5 in blue.

On seeds: the selector fits on training blocks 1–3 and confirms on block 4; leave-seed-out and no-pooling controls pass 42/42 held-forward checks with matching periods, and named-parent Tangent improves 36/36. Mean four-horizon MSE seed std is 0.0016/0.0017 (ATD-4/8) vs. 0.0069/0.0124 (Joint Direct); serial Timer uses one frozen checkpoint evaluated by blocks.

Dear Reviewer WCWK,

With the discussion period closing shortly, thank you again for your constructive review. We would like to gently check whether our added baselines and timing results address your questions:

  1. Matched correction baselines (Appendix C, Table 24): Added closed-loop and open-loop comparisons against History Mean, Last Cycle, SteerCast, UEC-STD, and BOOST on frozen serial Timer (Tangent achieves the lowest MSE on 4/5 datasets in each mode).
  2. Public-parent batch-128 speedups (Appendix C, Table 23): Confirmed end-to-end ATD-8 speedups under native cache settings (2.872.87–3.54×3.54\times on Timer, 2.092.09–2.11×2.11\times on TimesFM, 7.837.83–7.96×7.96\times on AutoTimes).
  3. Boundaries & seeds (Secs. 4.5–4.6, 5 & Appendix C, Table 26): Added chronological block diagnostics and explicitly qualified Exchange/ETTh2 failure cases and seed checks.

Please let us know if any questions remain. Thank you for your time.

The added correction baselines and failure case diagnostics substantially address my empirical concerns and clarify the scope of the claims. While methodological novelty remains modest and seed coverage limited, the practical contribution supports retaining my weak-acceptance recommendation.

Reviewer_3euYOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper examines whether input representation, learned transition span, and transitions emitted per inference call must share the same patch structure. Its main method, Autoregressive Trajectory Distillation (ATD), trains lightweight exits to imitate a frozen parent's recursive forecasts and emit several patches per call. Spectrum Tangent adds a periodic-history correction selected using training data. Experiments separate rollout fidelity, forecast accuracy, and inference cost, with additional tests on three public autoregressive parents.

Critical strengths
  1. The paper establishes a useful distinction between imitating a parent and improving its forecasts. Equation 7 explains the difference, and Table 3 changes only later-patch supervision: ATD improves rollout fidelity in all 21 matched seed runs, while worsening forecast accuracy on ETTm2 and Traffic relative to the clean-target control. This is a concrete insight into the limitations of trajectory distillation.

  2. The deployment evaluation is careful. ATD-1 exactly recovers the frozen parent, while Table 2 shows nearly unchanged forecast MSE across selectable widths and a 5.54x ATD-8 speedup on the matched subset. The correction uses training-only selection, includes its latency overhead, and retains a negative Exchange result. Cross-parent tests in Table 4 further support compatibility beyond the local model.

Critical weaknesses
  1. The acceleration results do not establish an advantage over a matched alternative. Table 12 correctly identifies the STRIDE operating points as incomparable, so its relative speedups cannot rank the methods. Moreover, the named-parent study reports compatibility and forecast accuracy rather than latency, while the main speedup comes from small local parents at batch one. This leaves the practical advantage of compiling a larger deployed parent uncertain, despite the convincing local reduction in calls.

  2. Spectrum Tangent's incremental value over simpler corrections is insufficiently isolated. Its main comparison is uncorrected ATD. Tables 17-20 provide useful direction and sensitivity diagnostics, but do not compare separately fitted history-mean or seasonal-template corrections in complete recursive forecasts. The exact coefficient minimizes fixed-proposal risk; corrected writeback changes that trajectory. Consequently, the diagnostics alone cannot establish whether periodic structure or the chosen horizon ramp explains the closed-loop gains beyond generic shrinkage.

Questions for authors
  1. Do you have an end-to-end latency measurement for ATD on one named parent, including its native cache handling, and a matched alternative at the same parent, context, horizon, and forecast-error tolerance? Even one operating point would clarify the practical acceleration claim; otherwise, please delimit which speed conclusions transfer beyond the local parents.

  2. Does Spectrum Tangent outperform a history-mean or simple seasonal-template correction in closed-loop forecasting with the same training origins and one fitted coefficient? A small comparison would distinguish the periodic correction's contribution from generic shrinkage.

  3. Do the flat/linear ramp and coefficient perturbations in Table 20 retain their benefits in complete closed-loop forecasts, rather than only on fixed proposal trajectories? An existing result or a focused check on one representative dataset would clarify how sensitive the deployed correction is to the calibration approximation.

2 comments
AuthorsOpenReview ↗

Native-cache timing and closed-loop controls

Thank you for the careful review and for recognizing the distinction between trajectory fidelity and forecast accuracy. Because ATD and Tangent are deliberately minimal probes of the PP/KK coupling (transition span vs. commit width), the submitted comparisons were matched controls rather than method rankings; your questions identify the matched external checks this leaves open. We have added native-cache timing, matched template controls, and ramp and coefficient sensitivity tests under complete recursive writeback (R1–R3).

Q1. Public-parent latency

With native KV reuse, Timer on ETTh1 takes 190.89 ms serially and 53.89 ms with ATD-8 (3.54×), with full-test H720 MSE of 0.44552 and 0.42995, respectively. At batch 128 with native KV reuse, the gain is 2.87–3.54× for Timer and 2.09–2.11× for TimesFM. AutoTimes gains 7.83–7.96× on its native re-encoding path.

R1 reports end-to-end P50 speedup of ATD-8 over each frozen parent's serial execution on an RTX 5880 Ada at batch 128, W=672W=672, and H=720H=720. Timing includes the exits and rollout updates. Entries are native KV reuse / cross-call KV reuse disabled. AutoTimes uses native re-encoding.

ParentETTh1ETTh2ETTm1Weather
AutoTimes-GPT2— / 7.84×— / 7.85×— / 7.83×— / 7.96×
Timer-base-84M3.54× / 10.91×3.53× / 10.91×3.52× / 10.85×2.87× / 11.81×
TimesFM-2.5-200M2.09× / 8.84×2.10× / 8.83×2.10× / 8.87×2.11× / 8.79×

No matched STRIDE point is available: its reported operating points differ in parent, draft, context, batch, hardware, and error tolerance (Table 12), so we do not claim a ranking. Within the same parent, context, and horizon, ATD-1 provides the exact zero-error-tolerance serial baseline and Frozen Direct shares ATD-8's exit runtime. R1 instead measures the quantity our claim needs: speedup over the same parent's native serial path at matched cache settings, batch 128, W=672W=672, H=720H=720. The transfer claim is limited to these parents and settings. In the revised PDF (highlighted in blue), we have added R1–R3 to Appendix C (Tables 23–25) and updated Sec. 4.6 and the throughput scope in Sec. 5.

Q2. Matched closed-loop template controls

The submitted study measures the effect of adding one history-derived direction to a frozen parent’s rollout. To measure the contribution of that direction, R2 varies the template under a common closed-loop procedure. Tangent gives lower MSE than both simple templates on four data sets. Exchange favors last-cycle repetition.

R2. MSE change (%) on frozen serial Timer (W=672W=672, H=720H=720) versus uncorrected Timer. Per series, history mean uses the initial lookback mean as a constant template; last-cycle repetition tiles the last observed cycle at the forecast origin; Tangent averages observations at matching phases across cycles. The two periodic templates share the training-selected period. All three use the same template-minus-proposal rule and staged ramp, protect the first 96-point block, write back corrected later blocks, and fit one nonnegative scalar each on the same 384 training origins.

TemplateETTh1ETTh2ETTm1WeatherExchange
History mean+9.63+0.48+3.97−3.89+1.84
Last cycle+5.11+0.54−13.54−0.69−13.04
Spectrum Tangent−4.44−2.85−21.07−5.11+0.97

Q3. Ramp and coefficient sensitivity after writeback

R3. Three-seed mean closed-loop MSE change (%) versus uncorrected local ATD-4 at H=720H=720. We hold the submitted period fixed and refit one training coefficient per ramp. The coefficient rows rescale the fitted staged coefficient α⋆\alpha^\star. Each call protects its first slot.

SettingETTh1ETTm2WeatherExchange
Flat−2.48−2.46−0.87+1.14
Linear−2.12−3.33−0.99+2.12
Staged (1.00×)−2.06−3.19−1.01+2.61
Staged (0.50×)−2.62−3.13−0.66+1.18
Staged (0.75×)−2.60−3.34−0.87+1.88
Staged (1.25×)−1.16−2.79−1.08+3.35
Staged (1.50×)+0.01−2.15−1.09+4.07

All three ramps and all tested staged multipliers from 0.50× to 1.25× retain gains on the three non-Exchange datasets—notably, even the Flat ramp (constant weighting without horizon scaling) retains −2.48%, −2.46%, and −0.87% MSE reductions, showing that on the tested data sets, closed-loop gains can be retained under constant weighting. At 1.50×, the ETTh1 gain disappears. Exchange degrades for every setting. The fitted coefficient minimizes fixed-proposal risk, while this experiment evaluates sensitivity after recursive writeback.

Dear Reviewer 3euY,

With the discussion period closing shortly, thank you again for your supportive and insightful review. We would like to gently check whether our new experiments in Appendix C (Tables 23–25) and Secs. 4.6 & 5 (highlighted in blue) fully address your questions on:

  • Batch-128 public-parent timing with native KV/cache reuse (2.092.09–7.96×7.96\times across Timer, TimesFM, and AutoTimes; Table 23);
  • Matched closed-loop template controls on frozen serial Timer (Table 24); and
  • Closed-loop ramp and coefficient sensitivity across Flat/Linear/Staged ramps and 0.50×0.50\times–1.50×1.50\times multipliers (Table 25).

Please let us know if any further clarification would be helpful. Thank you for your time.

Reviewer_VihEOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors examine whether the three roles commonly assigned to a patch in autoregressive time-series forecasting-input representation, learned transition span, and per-call output length-need to remain coupled. They study atomic patch encoding and propose Autoregressive Trajectory Distillation (ATD), which trains lightweight exits to reproduce a frozen parent model's recursive trajectory in fewer calls. Since this can also preserve the parent's accumulated errors, they introduce Spectrum Tangent, a history-based correction for long-horizon forecasts. The reported results show substantial inference speedups with largely stable forecast quality, as well as further long-horizon gains from the correction. Nevertheless, the evidence for the three parts of the paper is uneven, and the correction's applicability appears to depend on the data's periodic structure.

Critical strengths

1)Separating patch representation, learned transition, and execution width is a clear and useful way to frame the problem. 2)ATD offers substantial measured speedups while largely preserving the parent model’s forecasting accuracy. 3)The matched comparison between parent-trajectory and clean-future supervision helps explain why trajectory distillation is useful for efficient execution.

Critical weaknesses

1)The notation for Spectrum Tangent should be clarified. In particular, Eq. (8) defines a correction direction for the proposal, but the stated rule that the first slot of each call receives no correction is not reflected in the equation. The authors should explicitly include the slot mask and clarify how the direction is applied during recursive writeback. 2)The atomic-encoding study produces small and mixed accuracy changes. Its role in the overall contribution is less compelling than the execution and correction results. 3)ATD mainly compiles the parent's behavior rather than improving forecasting accuracy; it can retain the parent's existing errors. 4)Spectrum Tangent relies on a periodic history-derived direction and degrades performance on the weak-seasonality Exchange dataset. The paper should clarify when the correction should be applied or disabled. 5)Although the average long-horizon correction gains are positive, some chronological blocks worsen. An analysis of these failures would better establish the correction's reliability.

Questions for authors

My questions for the authors are detailed in the weaknesses above.

2 comments
AuthorsOpenReview ↗

Correction mask, writeback, and failure analysis

Thank you for recognizing the value of the patch-role framing and the matched supervision study, and for the detailed comments on the correction. As our reply to p4LS details, these are three targeted studies under patch-role coupling (p0p_0, PP, KK)—two separate decouplings and a correction motivated by ATD's inherited residuals—using minimal instruments so effects remain cleanly attributable. We clarify the mask and recursive writeback, explain how Atomic Encoding supports the representation study, and add chronological diagnostics to examine the failures you identified.

1. Eq. (8) and recursive writeback

For call cc and slot jj, let Gc,jG_{c,j} be the proposal, Tc,jT_{c,j} the selected template segment, and b(ℓc,j)b(\ell_{c,j}) the ramp at the slot's absolute forecast endpoint. The masked direction and committed forecast are dc,j=1[j>1]b(ℓc,j)(Tc,j−Gc,j)d_{c,j}=1[j\gt 1] b(\ell_{c,j})(T_{c,j}-G_{c,j}) and Fc,j=Gc,j+α∗dc,jF_{c,j}=G_{c,j}+\alpha^{\ast}d_{c,j}.

The first slot of each call retains the native proposal. All committed patches Fc,jF_{c,j} update the history or native KV cache before the next call. In the K=1K=1 serial Timer control (Appendix C), the mask instead protects only the first 96-point block of the full forecast (1[c>1]\mathbf{1}[c\gt 1]). In the revised PDF (highlighted in blue), we have updated Eq. (8) and Sec. 3.5 to include 1[j>1]\mathbf{1}[j\gt 1] and the writeback step explicitly, and documented the serial control mask in Appendix C.

2–3. Atomic Encoding and parent residuals

The small, mixed accuracy changes in Table 15 are relevant to Atomic Encoding's role as an experimental control. Its purpose is to test whether the local span can be fixed, not to improve accuracy. Our starting point is that data-set-specific patch spans complicate a common encoder. We therefore ask whether the projection shared across variates can operate at a fixed local span. Comparable accuracy at matched spans and backbone supports using this fixed granularity for the next experiment. With the local sharing scale fixed, we can ask whether width tuning suffices with a common grouping. Both preferences vary by data set, with a larger MSE range along width on the tested grid (Table 16). Grouping still changes the transition span, and width changes capacity.

We agree with your observation that ATD can retain the parent's errors. The matched supervision control demonstrates this distinction between fidelity and accuracy (Table 3). Spectrum Tangent is designed to address this directly: it applies a history-derived periodic direction to reduce those inherited residuals after writeback.

4–5. Exchange and chronological failures

On fixed proposals, let AA be the mean squared direction norm and BB the mean inner product with the true residual. A positive step α\alpha changes MSE by α2A−2αB\alpha^2 A-2\alpha B. When B>0B\gt 0, the break-even step is 2B/A2B/A.

R4(a). Frozen serial Timer at H=720H=720. Blocks partition test origins chronologically. MSE changes (%) are relative to uncorrected Timer.

DatasetBlockCosineFitted / break-even stepFixed proposalsClosed loop
ETTh21+0.12820.69−1.40+0.23
ETTh22+0.06041.33+0.65+2.53
ETTh23+0.06471.49+1.21+5.16
ETTh24+0.36670.29−11.09−12.54
ExchangeAll (1–4)−0.4197—+0.29+0.97

On frozen serial Timer, the correction direction is negatively aligned with the Exchange test residuals in all four blocks, so a positive step worsens fixed-proposal MSE. ETTh2 blocks 2 and 3 have positive alignment, but the fitted step exceeds the break-even value. Block 1 improves on fixed proposals and worsens after writeback, showing that closed-loop outcomes also depend on writeback. These diagnostics use realized test residuals.

In the submitted local ATD-4/8 Exchange runs, the training-selected correction passed all six held-forward training checks but increased H720 test MSE at both widths. The selected period was τ∗=12\tau^{\ast}=12, with α∗≈1.50\alpha^{\ast}\approx 1.50 versus 0.0850.085–0.4820.482 across the seven periodic datasets. This illustrates how periodic alignment observed during training can fail to persist at test time. The revised PDF highlights this boundary in Secs. 3.5, 4.5, and 5, and adds R4(a) in Appendix C (Table 26).

Dear Reviewer VihE,

With the discussion period closing shortly, we would like to gently follow up on our rebuttal and revised PDF (revisions highlighted in blue) to ensure our updates address your remaining concerns:

  1. Eq. (8) & writeback (Sec. 3.5): Explicitly included the slot mask 1[j>1]\mathbf{1}[j\gt 1] and recursive writeback update (and documented the serial control mask in Appendix C).
  2. Atomic Encoding scope (Sec. 4.2): Clarified its role as a matched representation control rather than an accuracy-maximizing module.
  3. Failure diagnostics & boundary (Secs. 4.5, 5 & Appendix C, Table 26): Added chronological block diagnostics separating frozen serial Timer Exchange residuals (cosine −0.4197-0.4197) from the submitted local ATD-4/8 Exchange failures, and bounded Tangent's empirical claims accordingly.

Please let us know if any points remain unclear—we would be happy to provide further clarification before the deadline. Thank you again for your time and helpful feedback.

Reviewer_p4LSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies ways to speed up patch-based autoregressive time-series forecasting and reduce prediction errors. It tests an input encoding built from short time segments. Autoregressive Trajectory Distillation (ATD) trains small parallel prediction heads to copy a frozen parent model’s recursive predictions, reducing the number of model calls. Spectrum Tangent then uses a periodic template from past observations to adjust the predictions.

Critical strengths

ATD speeds up inference while keeping the parent model’s parameters unchanged. The local paired tests report end-to-end speedups with similar average forecast errors at different output widths. Tests with public parent models also show that the method can work with several architectures.

Critical weaknesses
  1. The research problem is not clearly motivated. The paper does not clearly explain what problems arise from tying the three patch roles together. ATD does not need atomic encoding, and Spectrum Tangent is a separate correction step, so the reason for studying these three parts together is unclear. The encoding sensitivity tests also change the prediction span or model capacity, so they do not isolate the effect of input granularity.

  2. The methods’ added value and error-correction claims are unclear. ATD resembles standard multi-step distillation, while Spectrum Tangent applies a linear correction using a periodic template. The current comparisons do not show a clear advantage over existing acceleration methods and simple periodic calibration, respectively. The experiments also do not separate errors from the parent model, errors introduced by ATD, and errors propagated through recursion. Larger gains at longer horizons therefore do not show that the method specifically corrects recursive error accumulation.

Questions for authors
  1. What specific problem caused by patch-role coupling does the paper aim to solve? After atomic features are joined into a token covering P observations, what aspect of the representation changes independently? Which experiment separates this effect from changes in prediction span and model capacity?

  2. What does ATD add beyond training extra prediction heads to copy a teacher’s outputs over multiple future patches? Is the main contribution the measured speed–accuracy result, or does ATD introduce another technical idea that the paper has not clearly explained?

  3. How does Spectrum Tangent differ from a simple periodic-template correction? What evidence shows which errors it reduces? In particular, would the same correction also improve the original parent model without ATD? The answer would help determine whether the benefit is specific to ATD or comes from general calibration.

3 comments
AuthorsOpenReview ↗

Patch-length coupling and what each experiment tests

Thank you. In standard patch AR, one patch length ties together local input encoding granularity (p0p_0), token and one-step transition span (PP), and points committed per call (KPKP, set by commit width KK). Enlarging the patch changes both input encoding and training target, so prior quality does not carry over. We conduct three targeted studies under this coupling: Atomic Encoding tests fixing a 12-point local encoding basis (p0=12p_0=12) across datasets at fixed PP; ATD tests committing KK patches (KPKP points) per call at fixed parent transition PP; and Spectrum Tangent—motivated directly by the parent residuals ATD inherits—tests reducing long-horizon residuals via a history-derived periodic direction outside the backbone. The two decouplings can be examined separately, while Tangent targets inherited residuals.

Each study uses the minimal machinery needed so effects are cleanly attributable to the question rather than added capacity. Our contribution is the empirical answer, compared via single-factor matched controls (Frozen Direct, Table 3; template controls, R2).

Q1. What coupling matters, and what does the encoding control isolate?

Dataset-specific patch spans hinder a unified foundation-model encoder because raw projections tie variate-shared input weights to each dataset's patch length PP. Atomic Encoding fixes p0=12p_0=12: instead of projecting all 96 points of a patch at once, it applies one shared projection per 12-point atom and concatenates the eight outputs before projecting to width DD. Table 15 varies only input parameterization at matched token/transition span and backbone; a full-column-rank lift preserves the raw linear class (Prop. 2) with comparable accuracy (9/15 paired wins, mean H720 MSE −0.51%-0.51\%), supporting a fixed local basis. Fixing p0=12p_0=12 lets Table 16 isolate atom grouping m=P/12m=P/12 vs. model width DD: patch preference varies across datasets (2.85%2.85\% mean normalized MSE range across mm), yet width DD shows a larger range (5.08%5.08\%). This audit is descriptive, not causal (mm co-varies with PP; DD changes capacity); decoupling full token-input span from PP remains open.

Q2. What does ATD contribute beyond extra heads?

Exits build on multi-token prediction Gloeckle et al.. ATD contributes the compiled target and evidence: preserving the parent's first patch and training later exits on its deployed rollout yields a trajectory far easier to fit than observed futures even under updating history/instance normalization (unexplained fraction 0.109 vs. 0.845; Sec. 4.3). Selectable width KK (KPKP points) at fixed PP and exact ATD-1 recovery follow. In Table 3, changing only the target improves fidelity in 21/21 runs and forecast accuracy in 15/21.

Q3. What does Tangent correct, and does it help the serial parent?

Tangent uses one history-derived periodic direction per origin and one train-fitted nonnegative scalar to test whether AR residuals retain an aligned periodic component without a learned correction network. R2 tests this on frozen serial Timer (no ATD) under matched rule, ramp, mask, and writeback; Tangent outperforms both templates on 4/5 datasets (Exchange favors last cycle).

R2. Closed-loop Δ\DeltaMSE (%) on frozen serial Timer without ATD (W=672W=672, H=720H=720); matched rule, 1 scalar each.

TemplateETTh1ETTh2ETTm1WeatherExchange
History mean+9.63+0.48+3.97−3.89+1.84
Last cycle+5.11+0.54−13.54−0.69−13.04
Spectrum Tangent−4.44−2.85−21.07−5.11+0.97

With serial rollout QQ, observation-writeback rollout QobsQ^{\rm obs}, and ATD rollout GG, y−G=(y−Qobs)+(Qobs−Q)+(Q−G)y-G=(y-Q^{\rm obs})+(Q^{\rm obs}-Q)+(Q-G) separates observation-writeback parent error, recursion error, and ATD error.

R4(b). H720 mean squares and Tangent inner products on local ATD-4 fixed proposals (3 seeds), full-forecast first-block mask 1[(c,j)≠(1,1)]\mathbf{1}[(c,j)\neq(1,1)] (cross terms omitted in components, included in totals).

DatasetParent (obs.)Recursion errorATD errorTotal MSETan·ParentTan·Recursion
ETTh10.28050.13330.00760.4054+0.002+0.025
ETTm20.10370.30180.03090.3884+0.006+0.029
Weather0.11820.20330.00780.3278−0.001+0.017
Exchange0.10500.79740.02591.0146−0.007+0.057

In every row, the recursion inner product exceeds the parent (obs.) inner product, showing stronger directional alignment with accumulated recursive drift than with observation-writeback error (R2 measures net closed-loop gain after writeback).

In the revised PDF (blue highlights), we made the "three targeted studies" framing explicit in Sec. 1, qualified Table 16 as descriptive in Sec. 4.2, and added R2 and R4(b) in Appendix C (Tables 24 and 27; Tables 1–22 keep original numbering).

Dear Reviewer p4LS,

With the discussion period closing shortly, we would like to gently check whether our rebuttal and revised PDF (highlighted in blue) resolve your main concerns:

  1. Unified motivation (Sec. 1): Reframed the paper around three targeted studies under patch-length coupling—two separate decouplings (input scale p0p_0, commit width KK) plus Tangent targeting the parent residuals inherited by ATD.
  2. Encoding audit qualification (Sec. 4.2): Explicitly qualified Table 16 as a descriptive sensitivity audit rather than causal separation.
  3. Serial-parent controls & error attribution (Appendix C, Tables 24 & 27): Added matched closed-loop template controls on frozen serial Timer (Tangent wins 4/5 datasets) and counterfactual inner-product checks on local ATD-4.

Please let us know if anything further would be helpful before the deadline. Thank you again for your time and thoughtful comments.

Thank you for your response and for addressing my concerns. The novelty seems limited to me, so I will keep my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.