acceptodds
Under review as a conference paper at ICLR 2027

Time-series Instruction Following: An Endogenous Control Language

Abstract

Time-series forecasters face changing phase, trend, and shared channel activity even when the prediction task stays fixed. We study whether this state can be expressed as instructions that remain useful when the predictor changes. Our *endogenous instruction language* separates message generation from forecasting response. A compiler maps the observed window and horizon to shared and optional channel-aligned numerical messages; architecture-specific readers learn to use them through joint forecast training. Across seven backbones, the value of instructions depends on the information accessible to the base prediction path. Capacity-matched comparisons establish the predictive value of multichannel content beyond local adaptation. Fixed-weight interventions show that trained predictors use the supplied content, and held-out probes recover phase, trend, and volatility from FITS messages. A frozen learned compiler generates messages for new windows while a new backbone and reader learn their forecasting response. Reusing a FITS compiler with DLinear reduces mean squared error by 12.66% relative to a frozen-random compiler on a new sensor dataset, with all five seeds improving. A horizon-aware DLinear reader also uses the ordered compiler message tokens generated by PatchTST, reducing MSE by 7.56% over a frozen-random compiler across five seeds. Native forecaster configurations also achieve lower mean squared error than official-code iTransformer reproductions across 12 datasets. These results identify learned state expression as a reusable part of forecasting knowledge.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series, Instructions Following, Conditioned Generation, Temporal Feature Modulation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4okDOpenReview ↗
Rating
3(+2)
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors tackle the problem of multi-channel time-series prediction and introduce a latent representation model which uses both cross-channel and channel-local features to create a conditioning vector that enables superior performance.

They show that the learned features transfer to multiple prediction back-ends.

Critical strengths
  1. The idea is sound: extracting latent features from windows with different views is intuitive and domain-agnostic.

  2. The results are convincing: a 41% MSE reduction and transfer across back-ends suggest that relevant features are truly being learned during training.

Critical weaknesses
  1. The idea is not novel, and seems a simple application of representation learning:

The framing of this method with instruction following and conditioning seems primarily cosmetic; there is no natural language aspect to this work. The "language" that the authors refer to is essentially a hidden representation that is learned. This work then draws from a standard technique in deep learning, i.e. representation learning.

To my knowledge, none of the inductive biases implied by the chosen message passing structure is novel.

For example, SOFTS explores pools all channels into one shared "core" vector and feeds it back to each channel. Similar to the channel-averaged shared message in this work. Following SOFTS related work reveals other methods which have distributed structures across channels, which is also explored in this work.

The result showing transfer to back-ends is also not surprising, since when trained with general enough data, representations often learn useful features which can boost the performance of unseen back-ends.

Questions for authors
  1. Specifically what are the unique contributions of this work from an architectural point of view, and how do they interact with prior approaches which connect features in similar ways?
Reviewer_rGTgOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a compiler–reader method for time-series forecasting. Compiler analyzes past observations and creates numerical instructions that represent relationships over both time and channels. A forecasting model or reader then uses the instructions alongside the input data to output predictions.

The authors evaluate the method with several univariate forecasting models and test several ways for the models to use the instructions, including attention, FiLM, and changes to the model’s output. They also run experiments to separate the effects of adding cross-channel mixing information from other parts of the method.

The reported results differ across models and datasets. Improvements are more evident for some channel-independent models, while other settings show smaller gains or no clear benefit. The paper also tests whether a compiler trained for one forecasting model can be frozen and reused with another; the results suggest this can help in some cases, but the size of the benefit varies.

Critical strengths
  • The separation of instruction generation and consumption is an interesting approach, allowing learned representations to be reused across different forecasting architectures.
  • The paper provides extensive experimental evaluations across multiple datasets and architectures, along with several controlled ablations.
  • The reported improvements for channel-independent models are substantial, and the frozen-compiler transfer experiments show some potential for reusing learned representations.
Critical weaknesses
  • The novelty of the proposed approach is unclear. The compiler essentially acts as a learned feature encoder, while the readers use existing conditioning mechanisms such as attention, FiLM, and residual adapters.
  • Section 5.1 shows that the improvements are primarily observed in channel-independent models, while models already capturing cross-channel dependencies, such as iTransformer, show little to no improvement. This raises questions about the broader utility of the framework.
  • The paper lacks direct comparisons with simpler cross-channel conditioning approaches, making it difficult to establish whether the compiler-reader framework itself provides meaningful benefits.
  • Table 2 evaluates cross-dataset transfer only between PEMS04 and PEMS08, two closely related traffic datasets, limiting the evidence for generalizability across domains.
Questions for authors
  1. What is the advantage of using a learned compiler over directly providing cross-channel statistics or learned features to the forecasting backbone?
  2. Since the improvements are primarily observed in channel-independent models, what additional benefit does the framework provide to architectures that already model cross-channel dependencies? Or is the purpose of this work to just close the gap?
  3. Can the authors demonstrate compiler transfer across substantially different datasets or domains, such as to electricity or weather?
  4. Can the authors provide comparisons against stronger multivariate forecasting baselines and simpler conditioning approaches under matched parameter counts and computational budgets?
Reviewer_3UCPOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether window-level and channel-level state in multivariate time-series forecasting (MTSF) can be represented through a modular "compiler–reader" interface. A lightweight compiler maps the observed lookback window into numerical vectors ("messages"), while architecture-specific readers consume these vectors via prepended tokens, attention keys and values with FiLM modulation, or additive output residuals. The method is evaluated across 7 forecasting backbones. The authors perform fixed-weight interventions and linear readouts, test frozen compiler transfer across models, and compare a complete "Native DPT" iTransformer variant against official-code iTransformer reproductions across 12 datasets.

Critical strengths
  • Extensive mechanistic controls: The authors run 5 random seeds across their experimental matrix and include thoughtful diagnostic experiments (e.g., capacity-matched local vs. cross-channel comparisons, Hungarian-matched component substitutions, and frozen-random vs. frozen-learned compiler controls).
  • Clear contrast between channel-independent (CI) and mixing backbones: The paired experiments on PEMS04 clearly illustrate that CI backbones, which lack cross-channel access and channel-specific parameters, benefit substantially more from external conditioning packages than unconstrained channel-mixing backbones, i.e. iTransformer, FACT.
Critical weaknesses
  1. Unfair comparison in Table 3 and contradiction with Appendix results: While Table 3 claims that Native DPT beats official-code iTransformer across all 12 datasets (-0.6% to -37.1% MSE), Tables A.12 and A.13 reveal that the baseline iTransformer is restricted to FFN = 512 and 10 epochs (patience = 3, lr = 10−310^{-3}) on PEMS07/08 and Traffic, whereas Native DPT uses FFN = 2048 (4×4\times wider), adds +26.5% prompt/FiLM parameters (Appendix F), trains for 50 epochs (patience = 5) with tuned learning rates (10−410^{-4} / 3×10−43\times 10^{-4}), disables dynamic compilation in favor of Static constants on all 4 ETT datasets, and relies on a single diverged baseline seed (0.4452 vs. 0.0718–0.0731, Line 1115) for the -55.7% claim on PEMS03 (h=12h=12). Crucially, when iTransformer is evaluated under a strictly matched recipe in the appendix, adding the instruction package yields zero or negative gains: +0.2% on PEMS04, +2.4% worse on ECL, and +0.9% worse on Solar (Table A.15, where the matched Base iTransformer already beats the Table 3 official-code reproduction), and -0.3% / +0.2% on ETTm1/m2 (Appendix G).
  2. Self-Contradictions Between Main Claims and Appendix Controls, Plus Incomplete Reuse Evaluation:
  • First, while Section 5.2 attributes CI backbone gains to cross-channel shared observations (highlighting PatchTST), Table A.22 reveals that for nearly all other models, 93%–98% of the MSE reduction comes from Base to Local (simply adding 46,516 channel-specific adapter parameters to channel-shared backbones: 54.34% Local vs. 3.92% Cross-channel on FITS; 44.26% vs. 0.75% on PaiFilter; 35.78% vs. 0.99% on C-LoRA RMLP; and -2.25% degradation on TexFilter).
  • Second, shuffling all 8 hand-crafted window statistics in Native DPT (Section 3.3) changes MSE by -0.010% (Table A.4, showing they are ignored), and beyond PEMS04, paired packages degrade PatchTST on Solar by +3.4% across all 5 seeds (Table A.17).
  • Finally, in compiler reuse (Section 5.5), Table A.18 reveals that standard PatchTST-to-DLinear transfer with the pooled residual reader completely fails (0.08% MSE gain, worse MAE, and far behind its 0.3271 Joint scratch baseline), yet this failed row is omitted from main Table 2 and replaced by a custom 10-run DCT-basis reader (PatchTST-to-DLinear†, Eq. 2, increasing trainable target parameters from 58K to 96K) for which the Joint scratch baseline (--) was never run. (Minor: framing a 6,913-parameter linear projection of the channel mean + adapters as an "Endogenous Instruction-Following Control Language" also overstates the connection to instruction following.)

References

  • Yong Liu et al., iTransformer: Inverted Transformers Are Effective for Time Series Forecasting, ICLR 2024.
  • Yuqi Nie et al., A Time Series is Worth 64 Words: Long-Term Forecasting with Transformers (PatchTST), ICLR 2023.
  • Ailing Zeng et al., Are Transformers Effective for Time Series Forecasting? (DLinear), AAAI 2023.
  • Zhijian Xu et al., FITS: Modeling Time Series with 10k Parameters, ICLR 2024.
  • Kun Yi et al., FilterNet: Harnessing Frequency Filters for Time Series Forecasting (PaiFilter / TexFilter), NeurIPS 2024.
  • Tong Nie et al., Channel-Aware Low-Rank Adaptation in Time Series Forecasting (C-LoRA), CIKM 2024.
Questions for authors
  • How does Official-code iTransformer perform in Table 3 when trained with the exact same FFN width (1024/2048), training budget (50 epochs, patience 5), and learning rate as Native DPT (Table A.12)? And how to interpret results in Table 3 with the zero/negative matched-recipe results in Table A.15 and Appendix G?
  • Given that Base to Local accounts for 93%–98% of the gain on FITS, PaiFilter, and C-LoRA RMLP (Table A.22), isn't the primary benefit of the residual package simply adding channel-specific adapter capacity rather than cross-channel compilation? Also, what is the missing Joint scratch MSE for PatchTST-to-DLinear† in Table 2?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.