acceptodds
Under review as a conference paper at ICLR 2027

ROAR: Random-Order Autoregressive Time Series Forecasting from a Visual Perspective

Abstract

Autoregressive (AR) models have been increasingly applied to time series forecasting, owing to their favorable scalability and ability to accommodate prediction tasks of different horizons. However, most existing AR time series models are constrained by inflexible AR modeling paradigms and struggle with error accumulation. More general and flexible AR models and training methods for time series remain underexplored. To bridge this gap, we propose ROAR, a novel AR framework for time series forecasting. For AR modeling, motivated by the additional spatial structure introduced by the visual representation of time series, we reformulate forecasting as a random-order AR generation task on the time series image. For training, we devise a coupled training strategy that jointly supervises the entire backbone with signals from both teacher-forcing and free-running. These compact designs not only enable ROAR to learn richer contextual dependencies, but also narrow the training–inference mismatch, thereby alleviating the error accumulation incurred by the exposure bias. Comprehensive experiments conducted on 10 real-world benchmarks demonstrate that ROAR consistently achieves state-of-the-art performance while exhibiting superior robustness and generalization across multiple domains. Our code is available at: https://anonymous.4open.science/r/ROAR-6702

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_fmZkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ROAR, a visual autoregressive framework for time series forecasting. It transforms 1D time series data into 2D visual representations and performs autoregressive prediction over randomly ordered visual tokens. ROAR has a dual-branch training strategy combining teacher forcing and free running to mitigate exposure bias and error accumulation. Experiments demonstrate competitive forecasting performance, particularly for long prediction horizons.

Critical strengths

S1. An alternative approach to autoregressive modeling.

  • ROAR explores random-order autoregressive generation over visual tokens while preserving their positional identities, providing an alternative way to conventional autoregressive forecasting.

S2. Comprehensive empirical evaluation.

  • The paper provides a comprehensive evaluation across multiple SOTA benchmarks, including extended-horizon experiments, component ablations, and robustness analyses. The results support the effectiveness of the proposed components under the evaluated settings.
Critical weaknesses

W1. Limited distinction from prior visual time series modeling approaches.

  • Prior works, including TimesNet, VisionTS, and AURORA (ICLR 2026), have explored 2D representations or visual modeling for time series forecasting. ROAR's main contribution is therefore its random-order autoregressive modeling rather than the 1D-to-2D transformation itself. Although the ablation results show that random-order training improves performance over fixed-order training, the paper does not establish whether this benefit specifically depends on the visual representation.

W2. Insufficient analysis of training and inference consistency.

  • Random-order teacher forcing may use temporally later ground-truth tokens to predict earlier ones, creating a potential mismatch with fixed-order inference. Moreover, free-running training covers only within-window generation, leaving its effectiveness in mitigating error accumulation across recursive forecasting windows unclear.

W3. Limited robustness analysis of period-based visual representations.

  • ROAR constructs 2D representations using dataset-specific periodicities, but does not systematically evaluate sensitivity to period selection or robustness under dynamic patterns. This leaves the generalizability of the proposed representation unclear, particularly when a single fixed period can't adequately capture the temporal structure.
Questions for authors

Q1. About Random-order autoregression.

  • Would applying the same random-order autoregressive mechanism directly to 1D temporal tokens yield comparable improvements? What specific advantage does the 2D visual representation provide?

Q2. Periodicity and generalization

  • How sensitive is ROAR to the selected periodicity, particularly under multiple, changing, or weak periodic patterns? What about datasets with non-stationary properties?
6 comments
AuthorsOpenReview ↗

Response to Reviewer fmZk (Part 1)

Many thanks to Reviewer fmZk. Regarding the weaknesses and questions, we respectfully provide point-by-point clarifications below:


A1. TimesNet transforms a time series into 2D tensors to capture its periodic characteristics. VisionTS introduces a visual representation of time series and couples it with a heavy pretrained vision backbone to enhance representation learning. AURORA distills informative features from the endogenous images of a series to improve forecasting accuracy. ROAR differs from these methods: through the additional spatial property introduced by the visual representation, it extends standard autoregression from the original time axis to a more flexible random-order autoregressive modeling paradigm, which effectively improves forecasting performance and robustness.

To evaluate the effectiveness of random-order autoregression more comprehensively, we added an ablation. Specifically, we added a pair of variants, standard autoregression on 1D temporal tokens and random-order autoregression on 1D temporal tokens, verifying whether the performance gain of ROAR depends only on the visual representation transformation in a more isolated manner. As shown in Table R1, random-order autoregression on 1D temporal tokens outperforms standard autoregression on 1D temporal tokens. This indicates that the gain brought by ROAR does not rely solely on the visual representation transformation. In addition, regarding the advantage provided by the 2D visual tokens, random-order autoregression on 2D visual tokens achieves the best performance among all variants. This is consistent with the reflection and analysis in our introduction: compared with 1D temporal tokens, the additional 2D spatial property introduced by the visual representation provides the basis for more flexible autoregressive modeling and gives ROAR a more comprehensive ability to learn from context, which improves the downstream forecasting performance.

Table R1. Average MSE over the four horizons with standard and random-order autoregression on 1D temporal tokens and on 2D visual tokens (three seeds).

TokensAutoregressionETTh1ETTm1Exchange
1D temporalstandard0.4102 ± 0.00270.3651 ± 0.01340.6415 ± 0.1975
1D temporalrandom-order0.4086 ± 0.00150.3560 ± 0.00180.4691 ± 0.1867
2D visualstandard0.3982 ± 0.00270.3512 ± 0.00300.4584 ± 0.0644
2D visual (ROAR)random-order0.3917 ± 0.00450.3476 ± 0.00160.3275 ± 0.0260

In summary, the ablation is incorporated into Appendix J (Table 10, Lines 1266–1282) of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer fmZk (Part 2)

A2. Temporally later tokens during training provide richer contextual supervision than standard left-to-right autoregression, thereby stabilizing optimization and enabling ROAR to learn more comprehensive temporal dependencies. At inference, no future tokens are used as conditions for autoregressive generation. Instead, inference strictly follows the standard order, which ensures coherent and stable predictions. To verify this empirically, we evaluated autoregressive inference under random orders (Table R2). Averaged over three seeds, random-order inference consistently underperforms standard-order inference.

Table R2. MSE under standard-order and random-order inference at the longest horizon and averaged over the four horizons (three seeds).

HorizonETTh2 standardETTh2 randomExchange standardExchange random
7200.4130.4190.7020.781
10800.4650.4731.0861.184
14400.5140.5242.1632.216
18000.5490.555——
21600.5800.583——
Avg. (96–720)0.3620.3670.3280.346

Regarding free running, it acts within one forecasting window and reduces exposure bias, so every forecast that is fed back to later windows is more reliable. We further examined the ablation at the longest horizon H=720, which requires eight recursive windows (Table R3). Removing free running clearly degrades the forecast, which shows that its gain is retained across recursive windows.

Table R3. MSE / MAE at H=720 with and without free running (mean ± standard deviation over three seeds).

MetricETTm1ETTm2Wind
ROARMSE0.414 ± 0.0030.375 ± 0.0031.340 ± 0.006
ROARMAE0.424 ± 0.0010.394 ± 0.0030.862 ± 0.003
Without free runningMSE0.549 ± 0.0790.424 ± 0.0051.420 ± 0.004
Without free runningMAE0.491 ± 0.0330.429 ± 0.0040.898 ± 0.002

A3. Regarding the sensitivity to period selection, we added a period-sensitivity experiment, as shown in Figure R1(https://anonymous.4open.science/r/ROAR-6702/rebuttal/fig1.jpg). On benchmarks with a clear and stable temporal structure, deviating from the dominant period changes the phase alignment of the visual representation, which leads to a performance degradation. A smaller period splits one complete cycle across several columns, which weakens the ability of each column to express the full intra-period pattern. A larger period packs several cycles into one column, which introduces redundant information and leaves fewer repeated periods across columns.

Regarding datasets without a clear temporal structure and with non-stationary properties, ROAR remains effective and generalizes well. Exchange is a strongly non-stationary benchmark: its level drifts over time and it shows no stable periodicity, as shown in Figure R2(https://anonymous.4open.science/r/ROAR-6702/rebuttal/fig2.jpg). On Exchange, ROAR achieves the best MSE and MAE among all ten methods (Table R4). This indicates that ROAR is not limited to data with a stable periodic structure and generalizes to non-stationary data.

Table R4. Results on Exchange, averaged over the four horizons.

MethodMSEMAE
ROAR0.3280.392
TimeTK0.3610.410
VisionTS0.3770.420
PatchTST0.4110.434
iTransformer0.4220.442
AutoHFormer0.4340.447
TimeLLM0.4410.459
TimeMixer0.4920.486
Timer-XL0.8510.611
TimesNet0.9930.659

In summary, the period-sensitivity results are incorporated into Appendix K (Figure 14 and Table 11, Lines 1285–1328) of the revised manuscript.

Thank you for the clarifications and additional experiments. Table R1 and Table R3 address my earlier concerns. But several points still to be clarified.

  • The Consistency of reported results.

    • The standard-order ETTh2 results in Table R2 differ a lot from those in the original Table 7. Were the same checkpoints, and evaluation settings used?
    • Please also add the std in Table R2.
  • Scope of the periodicity analysis.

    • My concern is not only about sensitivity to the chosen folding period, but also whether a single fixed period remains suitable when the temporal structure changes across different segments of the same series. Varying fixed periods and reporting aggregate results on Exchange data as a whole do not directly address this issue. Is there evidence that ROAR remains effective across such changes under a fixed configuration?
Reviewer_QaMGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies time series forecasting through a visual representation. ROAR folds each channel into an image and trains an autoregressive predictor over visual tokens in random orders, using both teacher-forcing and free-running losses. Experiments showed that the proposed model achieves compelling performance over then datasetes.

Critical strengths

The design has a clear rationale, which is that the random ordering exposes the model to different token dependencies, while teacher forcing and free running combine stable supervision with learning from its own predictions. The multi-seed ablations support these design choices. The experiments also provide useful evidence for the practical value of using one checkpoint across prediction horizons.

Critical weaknesses

I'm less convising by the claim about the visual representation itself. The ablation retains the visual tokenizer, while comparisons with temporal models also change the model and training setup. This leaves unclear how much the gains come from the 2D representation rather than the training strategies. Random-order visual autoregression has also been explored in a CVPR '25 paper (arXiv 2412.01827), so I'd suggest the authors to provide a clearer account of what the forecasting adaptation adds.

Questions for authors

I have the following question: how does the 1D token model perform with the same random-order training and teacher-forcing/free-running objectives, under a comparable model and training budget?

3 comments
AuthorsOpenReview ↗

Response to Reviewer QaMG

We thank Reviewer QaMG for the helpful comments and respond to each point below.


A1. To evaluate the contribution of the visual representation more comprehensively, we added an ablation. Specifically, we added a pair of variants, standard autoregression on 1D temporal tokens and random-order autoregression on 1D temporal tokens, which use the same predictor, teacher-forcing/free-running objectives, and training budget as ROAR and differ only in the token representation. As shown in Table Q1, under the same training strategy, 2D visual tokens outperform 1D temporal tokens with both standard and random-order autoregression, and random-order autoregression on 2D visual tokens achieves the best performance among all variants. This indicates that the gain of ROAR comes from the visual representation in addition to the training strategies: compared with 1D temporal tokens, the additional 2D spatial property introduced by the visual representation provides the basis for more flexible autoregressive modeling and gives ROAR a more comprehensive ability to learn from context, which improves the downstream forecasting performance.

Table Q1. Average MSE over the four horizons with standard and random-order autoregression on 1D temporal tokens and on 2D visual tokens (mean ± standard deviation over three seeds).

TokensAutoregressionETTh1ETTm1Exchange
1D temporalstandard0.4102 ± 0.00270.3651 ± 0.01340.6415 ± 0.1975
1D temporalrandom-order0.4086 ± 0.00150.3560 ± 0.00180.4691 ± 0.1867
2D visualstandard0.3982 ± 0.00270.3512 ± 0.00300.4584 ± 0.0644
2D visual (ROAR)random-order0.3917 ± 0.00450.3476 ± 0.00160.3275 ± 0.0260

In summary, the ablation is incorporated into Appendix J (Table 10, Lines 1266–1283) of the revised manuscript.


A2. We carefully examined RandAR (Pang et al., CVPR 2025), a random-order autoregressive model designed for image generation. Unlike RandAR, ROAR introduces a visual representation transformation for time series. This transformation provides an additional 2D spatial structure, which ROAR uses to develop a more flexible random-order autoregressive modeling paradigm for time-series forecasting.

ROAR further introduces dual-branch training with teacher forcing and free running. Teacher forcing learns from ground-truth tokens, whereas free running performs autoregressive generation from the model's own predictions. Since ground-truth future tokens are unavailable during inference, free running reduces exposure bias and the resulting error accumulation. We cited and discussed RandAR in Related Work (Lines 120--123) to clarify its distinction from ROAR.

AuthorsOpenReview ↗

Waiting for response

Dear Reviewer QaMG,

As the rebuttal period is approaching its end, we would like to kindly ask whether our responses have adequately addressed your primary concerns. If so, we would sincerely appreciate it if you could reconsider your overall evaluation.

If you have any remaining concerns or further suggestions, we would be more than happy to provide additional clarification and continue the discussion. We sincerely appreciate the time and effort you have devoted to reviewing our work and helping us improve the manuscript.

Kind regards,

The Authors

AuthorsOpenReview ↗

Kindly following up for a response

Dear Reviewer QaMG,

As the rebuttal discussion period is approaching its end, we sincerely apologize for reaching out to you again. We would like to kindly ask whether our previous responses have adequately addressed your concerns. If so, we would sincerely appreciate it if you could reconsider your evaluation of our work.

Once again, we sincerely apologize for reaching out to you again, and we would like to express our heartfelt appreciation for the valuable time and effort you have devoted throughout both the review and rebuttal process.

Reviewer_NTmCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, they proposes ROAR, a time-series forecasting method that generates "visual tokens" via an autoregressive mechanism with a randomized order. The approach combines "teacher-forcing" (using ground-truth data) and "free-running" (relying on previous predictions) to minimize the discrepancy between the training and inference phases. Experiments across ten datasets demonstrate competitive performance, while ablation studies confirm the effectiveness of the proposed components.

Critical strengths
  1. The paper is well-written and generally easy to follow.
  2. ROAR achieves the best average Mean Squared Error (MSE) under the reported rolling forecasting protocol and performs well over long forecast horizons, highlighting its utility for recursive forecasting.
  3. Controlled ablation studies demonstrate the benefits of using a randomized order and a "free-running" supervision mechanism, thereby strengthening the paper's empirical contributions.
Critical weaknesses
  1. It is unclear how the forecasting model determines the position of the next target when a randomized order is applied. Please clarify this conditioning mechanism, as such information is essential to verify whether the actual implementation aligns with the stated formulation.

  2. Comparisons with language-model-based forecasters are limited. Although TimeLLM is included, Chronos [1] and Scaler [2], has not been evaluated. Comparing results under equivalent forecasting and adaptation settings will help clarify the benefits of "visual tokens."

  3. The "free-running" training mechanism is described in the context of a 96-step forecast; however, longer forecast horizons require feeding predicted sequences back into the model as historical data. Does the training process incorporate this feedback mechanism across the forecasting steps? This distinction is crucial for understanding how the proposed method mitigates error accumulation in long-term forecasts.

[1] Ansari, A. F., et al. Chronos: Learning the Language of Time Series. Transactions on Machine Learning Research, 2024.

[2] Le, X.-M., et al. Test-Time Scaling for LLM-based Time Series Forecasting (SCALER). KDD, 2026.

Questions for authors

See the weaknesses above.

4 comments
AuthorsOpenReview ↗

Response to reviewer NTmC (Part 1)

We would like to sincerely thank Reviewer NTmC for the helpful comments and respond to each point below.


A1. During training, ROAR samples a history permutation πh\pi_h and a future permutation πf\pi_f. The generation step follows the formulation in our paper:

(Uh,z^πf(i)f)=Gθ(Zhπh,BOS,Zf,πf(<i)).\big(\mathbf{U}_h,\hat{z}^f_{\pi_f(i)}\big)=G_\theta\big(\mathbf{Z}_h^{\pi_h},\mathrm{BOS},\mathbf{Z}_{f,\pi_f(\lt i)}\big).

Here, πh\pi_h determines the serialization of the history tokens, while πf\pi_f specifies which future token is generated at each step. At step ii, the model predicts zπf(i)fz^f_{\pi_f(i)} from all history tokens and the preceding future tokens in the sampled order. Its physical-position embedding identifies the target location; therefore, the permutation changes the generation order without changing where each token belongs. The future inputs are shifted accordingly: BOS is used at the first step, while later steps use the preceding ground-truth token under teacher forcing or the preceding prediction under free running (Section 3.2, Lines 207--227; Appendices B.3--B.4, Lines 844--859).

At inference, we use a fixed πh\pi_h and a left-to-right raster πf\pi_f rather than random perturbations, ensuring compact and stable predictions (Section 3.2, Lines 214--215; Appendix B.3, Lines 851--853). A detailed analysis is provided in our response to Reviewer fmZk (A2). We clarified both mechanisms in the revised manuscript.


A2. We added SCALER and Chronos to the related work and cited both studies (Lines 104--107). We also conducted the following additional comparisons.

SCALER. We included SCALER as an additional baseline and followed the default hyperparameters in its official repository; the learning rate was selected on the validation split. As shown in Table NT1, ROAR achieves lower mean MSE and MAE on Exchange, Environment, and Wind.

Table NT1. MSE and MAE at H=96, mean ± sample standard deviation over three seeds.

DatasetROAR MSEROAR MAESCALER MSESCALER MAE
Exchange0.0900 ± 0.00270.2148 ± 0.00230.1414 ± 0.01330.2895 ± 0.0143
Environment0.2578 ± 0.00170.3651 ± 0.00190.2687 ± 0.00630.3870 ± 0.0118
Wind0.9017 ± 0.00560.6573 ± 0.00370.9153 ± 0.00810.6620 ± 0.0048

Chronos. We also compared ROAR with the Chronos family, which provides strong zero-shot forecasting performance, including Chronos-T5-base, Chronos-Bolt-base, and Chronos-2. We set the context length of Chronos to be the same as that of ROAR, and the specific results are shown in Table NT2.

Table NT2. MSE / MAE. ROAR values are three-seed means. Average is over the four horizons.

DatasetHorizonROARChronos-T5-baseChronos-Bolt-baseChronos-2
ETTm1960.2886 / 0.34760.4476 / 0.40030.3272 / 0.33310.2969 / 0.3171
ETTm11920.3286 / 0.37280.5532 / 0.46330.3822 / 0.36540.3584 / 0.3540
ETTm13360.3593 / 0.39350.6312 / 0.50470.4275 / 0.39270.3991 / 0.3812
ETTm17200.4139 / 0.42420.8362 / 0.56470.5167 / 0.43790.4799 / 0.4288
ETTm1Average0.3476 / 0.38450.6171 / 0.48320.4134 / 0.38220.3836 / 0.3703
ETTm2960.1921 / 0.28050.1955 / 0.27220.1796 / 0.24600.1707 / 0.2363
ETTm21920.2435 / 0.31290.2623 / 0.31790.2520 / 0.29420.2456 / 0.2880
ETTm23360.2922 / 0.34360.3248 / 0.35920.3166 / 0.33640.3111 / 0.3328
ETTm27200.3747 / 0.39410.4242 / 0.42020.4129 / 0.39560.3974 / 0.3893
ETTm2Average0.2756 / 0.33280.3017 / 0.34240.2903 / 0.31810.2812 / 0.3116

As shown in Table NT2, ROAR achieves lower four-horizon average MSE than all three Chronos models on both datasets, while Chronos-2 achieves the lowest average MAE. These results demonstrate ROAR's competitive forecasting performance through random-order autoregressive modeling of visual tokens and motivate extending it through large-scale pretraining. The comparisons were incorporated into Appendix L (Lines 1331--1384).

AuthorsOpenReview ↗

Response to reviewer NTmC (Part 2)

A3. Free running is applied within each 96-step training window. ROAR first predicts a future token from the observed history and the BOS token. The prediction, rather than the corresponding ground-truth token, is then fed back to predict the next future token. This process continues through all four future latent tokens representing the 96-step window, and the decoded sequence is supervised by series-space MSE. In contrast, teacher forcing uses the ground-truth future-token prefix at each step.

For longer horizons, each decoded 96-step prediction is appended to the history, the most recent context is retained, and the same predictor is called again. H=192, 336, and 720 require two, four, and eight calls, respectively; the final output is truncated to the requested length.

Table NT3. MSE, mean ± sample standard deviation over three seeds. The final column reports the increase after removing free running.

DatasetHorizonFull modelw/o FRMSE increase
ETTh1960.3660 ± 0.00550.3786 ± 0.0088+3.4%
ETTh11920.3881 ± 0.00380.4006 ± 0.0110+3.2%
ETTh13360.3997 ± 0.00380.4093 ± 0.0131+2.4%
ETTh17200.4130 ± 0.00670.4190 ± 0.0157+1.5%
ETTm2960.1921 ± 0.00120.3215 ± 0.0037+67.4%
ETTm21920.2435 ± 0.00280.3415 ± 0.0044+40.2%
ETTm23360.2922 ± 0.00290.3672 ± 0.0051+25.7%
ETTm27200.3747 ± 0.00250.4243 ± 0.0050+13.2%
Weather960.1717 ± 0.00160.2144 ± 0.0017+24.9%
Weather1920.2121 ± 0.00200.2517 ± 0.0016+18.7%
Weather3360.2588 ± 0.00280.2933 ± 0.0023+13.3%
Weather7200.3256 ± 0.00400.3558 ± 0.0020+9.3%

As shown in Table NT3, removing free running increases the mean MSE in all 12 settings. The improvement is retained at H=336 and H=720 on all three datasets, showing that free running remains effective in long recursive forecasts. A teacher-forcing-only model is trained with ground-truth prefixes that are unavailable during recursive inference. Free running instead exposes the model to self-generated prefixes during training, reducing the training–inference mismatch and mitigating error accumulation. The full model also has a lower standard deviation in 9 of the 12 settings, indicating greater robustness across random seeds. We clarified the mechanism in Section 3.3 (Lines 248--252).

AuthorsOpenReview ↗

Waiting for response

Dear Reviewer NTmC,

As the rebuttal period is approaching its end, we would like to kindly ask whether our responses have adequately addressed your primary concerns. If so, we would greatly appreciate it if you could reconsider your evaluation.

If you have any remaining concerns or further suggestions, we would be more than happy to provide additional clarification and engage in further discussion. We sincerely appreciate the time and effort you have devoted to reviewing our work and helping us improve the paper.

Kind regards, The Authors

AuthorsOpenReview ↗

Kindly following up for a response

Dear Reviewer NTmC,

As the rebuttal discussion period is approaching its end, we sincerely apologize for reaching out to you again. We would like to kindly ask whether our previous responses have adequately addressed your concerns. If so, we would sincerely appreciate it if you could reconsider your evaluation of our work.

Once again, we sincerely apologize for reaching out to you again, and we would like to express our heartfelt appreciation for the valuable time and effort you have devoted throughout both the review and rebuttal process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.