acceptodds
Under review as a conference paper at ICLR 2027

TinyCycle: Time-Series Forecasting in 30 Parameters with Closed-Form Fitting

Abstract

In long-term time-series forecasting, lightweight models can compare to those of complex models, but aggregate metrics reveal little about the information these models recover or the sources of their performance differences. Based on phase recurrence across cycles, we decompose both the true future and model predictions into Structure and Dynamics. We then use orthogonality to decompose MSE. Diagnostics across multiple models show that models recover Structure substantially better than Dynamics, and that modeling Structure can account for most of their predictions. Motivated by this observation, we propose TinyCycle, which directly models the Structure of the data without modeling complex temporal mappings. Across experiments, TinyCycle recovers most of the forecasting performance of mainstream time-series forecasting models with no more than 31 parameters using closed-form fitting.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Analysis

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_xZq1OpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
2(+1)
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper proposes TinyCycle, a highly parameter-efficient time-series forecasting method that uses closed-form fitting to exploit recurring periodic patterns without gradient-based training. Experiments across 32 forecasting tasks show competitive performance with substantially fewer parameters and low computational cost.

Critical strengths

s1. The paper conducts extensive experiments across multiple datasets. The experimental setup and implementation details are well documented, and the proposed method is compared against a broad range of baselines. The related work section also provides a clear discussion of how the method differs from existing approaches.

s2. The paper provides some theoretical analysis to support the effectiveness of the proposed method.

Critical weaknesses

w1: It is difficult to understand the precise problem the paper aims to solve and the core architecture or methodological contribution being proposed. Overall, the paper provides limited insights beyond the empirical results.

w2: TinyCycle appears to be an improvement or an extension of CycleNet. The authors should provide a detailed experimental comparison with CycleNet or clearly explain why such a comparison is omitted.

w3: Although the experiments demonstrate that TinyCycle is lightweight, its predictive performance does not appear particularly impressive. Furthermore, it remains unclear whether TinyCycle can serve as a plug-and-play module to improve the performance of existing models.

Questions for authors

Please refer to the weaknesses.

7 comments
AuthorsOpenReview ↗

Response to Reviewer xZq1 [1/3]

We sincerely thank you for your review comments. But we also need to clarify some points:

w1: It is difficult to understand ... Overall, the paper provides limited insights beyond the empirical results.

Response: Thank you for your feedback. We reaffirm the paper’s central research question and approach below and have refined some parts of the paper.

(1) Research question | Which component contributes most to forecasting performance? We want to explain why models with very few parameters can achieve competitive long-term forecasting accuracy. Specifically, we examine how much of the error gap between models comes from recovering the window mean and recurring phase patterns, and how much comes from predicting cross-cycle variations that deviate from these patterns. Overall MSE or MAE measures the size of the error gap, but identifying its sources requires separating these two error components. We now clarify this question and its motivation earlier in the revised introduction (lines 43–53).

(2) Core method | Diagnose errors through Structure–Dynamics decomposition, then use TinyCycle to test the forecasting accuracy that Structure can support. We apply the same phase-based decomposition separately to the ground-truth future and model predictions. Structure consists of the window mean and the phase pattern that repeats across cycles. Dynamics comprises the remaining variation after subtracting Structure, including cross-cycle changes in level, amplitude, and phase. Under the same period and evaluation window, the prediction errors of these two components are orthogonal, so total MSE decomposes exactly into Structure MSE and Dynamics MSE. When comparing two models, the differences in these component errors quantify their respective contributions to the total MSE gap (lines 54–63; lines 193–215; lines 232–244). Based on this diagnostic, TinyCycle tests what forecasting accuracy can be achieved with very few parameters when Structure serves as the main predictive signal. It partitions recent history into aligned cycle blocks and combines same-phase observations using coefficients shared across phases and channels to generate the next block. It then inserts the predicted block into the rolling window, removes the oldest block, and reuses the same coefficients to continue forecasting. The cycle coefficients and boundary-correction coefficient are fitted separately in closed form. The revised Figure 4 caption explains this forecasting process step by step (Figure 4, lines 226–229; lines 259–277; lines 288–307).

(3) Key findings | Dynamics accounts for more remaining error, while Structure more often explains accuracy gaps at distant horizons. In the aggregate results for 720-step forecasts across seven datasets and eleven models, Dynamics accounts for 58.11% of total MSE. However, in the most distant forecast interval in the updated Figure 8 (steps 481–720), the model with lower overall error achieves a larger MSE reduction in Structure than in Dynamics in 96.70% of model pairs (88/91 pairs across fourteen models). These results address two distinct questions: where most forecasting error remains, and where accuracy gaps between models primarily come from. They also explain why TinyCycle can achieve competitive accuracy by focusing on Structure, even when Dynamics remains difficult to predict (Table 8, lines 1400–1403 and 1415–1421; Figure 8, lines 432–450 and 468–477).

AuthorsOpenReview ↗

Response to Reviewer xZq1 [2/3]

w2: TinyCycle appears to be an improvement ... why such a comparison is omitted.

Response: Thank you for your mention CycleNet. We have included it in the baseline of our paper. However, we would like to emphasize that TinyCycle and CycleNet differ fundamentally in their underlying assumptions, model architectures, and design objectives.

(1)TinyCycle and CycleNet characterize different aspects of time series. CycleNet repeats a globally learned periodic template and trains a model to fit the residuals, whereas TinyCycle directly uses the context to model the dominant structure of the data. Moreover, CycleNet’s learned global periodicity is distinct from Structure in TinyCycle. The former captures the periodic properties of the dataset itself, whereas the latter is derived from the context and reflects the dominant stable state that the model can capture within that context.

(2) TinyCycle and CycleNet are different types of models. TinyCycle is a lightweight model, whereas CycleNet focuses on improving forecasting accuracy without an explicit lightweight design. The table below compares their efficiency and accuracy. These results will also be included in the revised manuscript.

ModelMSE ↓MAE ↓Params ↓Train (s) ↓Test (s) ↓
TinyCycle0.3210.34525.250.5260.063
CycleNet (Linear)0.3300.349246,102666.1773.313
CycleNet (MLP)0.3350.353545,366903.3734.328

(3)The experiments further demonstrate the differences between TinyCycle and CycleNet. Across eight datasets and four forecast horizons (32 tasks; 128 test windows per task), we compare the models through two interventions: shifting the absolute phase index by ±P/4\pm P/4 with observations fixed, and perturbing the 1st (most recent) and 7th historical cycles using paired inputs with matched mean, variance, and cycle-end values. TinyCycle is invariant to absolute phase labels and propagates historical perturbations within the same phase. CycleNet responds to phase shifts and can propagate perturbations across phases, even when its periodic template is set to zero with all other weights unchanged, demonstrating cross-phase propagation through its residual predictor. The table reports medians across tasks: RMS measures normalized forecast changes, while cross-phase energy is the fraction of response energy outside the perturbed phases. Full details are provided in Appendix E3.

ModelAbsolute-phase response RMSCycle 1 response RMSCycle 1 cross-phase energyCycle 7 response RMSCycle 7 cross-phase energyZero-template cycle 1 cross-phase energy
TinyCycle0.0000000.0034810.00%0.0006510.00%—
CycleNet Linear0.0626210.00205785.61%0.00128773.63%85.61%
CycleNet MLP0.0232800.00173887.53%0.00095886.90%87.53%
AuthorsOpenReview ↗

Response to Reviewer xZq1 [3/3]

w3: Although the experiments demonstrate ... serve as a plug-and-play module to improve the performance of existing models.

Response: We have reiterated that TinyCycle was not designed with the primary goal of achieving high predictive accuracy. Nevertheless, TinyCycle outperforms many more complex models. Furthermore, unlike CycleNet, TinyCycle cannot be simply integrated into other models in a plug-and-play manner, as the two approaches fundamentally differ in what they aim to model. However, we acknowledge that plug-and-play integration is indeed effective for certain models.

(1) Performance. We reiterate that TinyCycle is not primarily designed to maximize forecasting accuracy. Its main strength is its lightweight design, while its broader contribution lies in revealing the primary sources of performance in time series forecasting models. While remaining lightweight, TinyCycle achieves lower MSE on the LTSF datasets than models with more parameters, including iTransformer (ICLR 2024), PhaseFormer (ICLR 2026), Toto-2 (arXiv 2026), and MixLinear (ICLR 2026)(Figure 5). On GIFT-Eval, its MASE is second only to xLSTM-Mixer (NeurIPS 2025) among the supervised models compared, and it outperforms first-generation TSFMs such as Chronos (TMLR 2024) and Moirai (ICML 2024)(Figure 16).

(2) Plug-and-play Capability. The core modeling principle of TinyCycle is to map phases to structures. In this paper, we adopt a naive Linear layer to maintain an extremely lightweight architecture, just as a demo. However, this Linear layer can be readily replaced with other architectures, enabling plug-and-play integration. Notably, this modeling paradigm reduces the mapping complexity from O(H×L)O(H \times L) to O(H/P)O(H/P), where HH, LL, and PP denote the lookback window, prediction horizon, and period length, respectively. This reduction improves modeling performance while substantially decreasing the number of parameters. A detailed theoretical proof is provided in Appendix I of the revised manuscript. The following table presents the results obtained by replacing the Linear layer with different backbone models.

DatasetBackbonebackbone MSE / MAETinyCycle-plug MSE / MAEAvg. Parameters (Backbone → plug in)
ETTh1PatchTST0.458 / 0.4540.423 / 0.425805,344 → 74,490
ETTh1TimeMixer0.446 / 0.4560.416 / 0.42689,977 → 61,611
ETTh1DUET0.428 / 0.4430.435 / 0.428696,141 → 179,032
ETTh1iTransformer0.457 / 0.4660.427 / 0.43444,080 → 13,834
ElectricityPatchTST0.167 / 0.2620.163 / 0.2522,783,504 → 217,666
ElectricityTimeMixer0.179 / 0.2780.164 / 0.253347,977 → 179,048
ElectricityDUET0.180 / 0.2790.166 / 0.254502,549 → 68,344
ElectricityiTransformer0.169 / 0.2680.165 / 0.258482,736 → 358,250
ExchangePatchTST0.388 / 0.4180.373 / 0.410630,920 → 14,482
ExchangeTimeMixer0.475 / 0.4480.406 / 0.422906,057 → 60,991
ExchangeDUET0.464 / 0.4750.341 / 0.405282,961 → 44,157
ExchangeiTransformer0.405 / 0.4380.412 / 0.447110,656 → 57,902

For a fair comparison of parameter counts, identical backbone configurations are maintained between the original backbone models and their TinyCycle-plug counterparts. The learning rate is independently tuned for each model to achieve optimal performance on the validation set. Additional experimental results are provided in Appendix I.2 of the revised manuscript.


We hope these responses address your concerns, and we would be happy to provide further clarification during the discussion period.

Reviewer_2DJVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose TinyCycle, which predicts future cycles through a linear combination of historical cycles and recursively reuses the same coefficients. The reported results demonstrate an attractive tradeoff between model size, fitting cost, and forecasting accuracy, using at most 31 learned coefficients, with an average of 0.321 MSE across datasets.

Critical strengths
  1. The diagnostic decomposition is mathematically precise and internally consistent. The distinction between phase repetition and smoothness is also useful.

  2. The model provides a strong performance of parameter-efficient forecasting. And the reported accuracy is competitive given the model’s simplicity.

  3. The experiments are solid and comprehensive, especially in Appendix H.

Critical weaknesses
  1. I have noticed some mismatch in experiments: TinyCycle outperforms patchtst, itransformer in the Tab 12 in Exchange dataset, but is outperformed by them in Tab 4 in 720 setting. Is there a mistake or if I miss something.

  2. I think the CycleNet is a particularly relevant missing experimental comparator. It explicitly models periodic patterns through a learned cycle component and predicts residual variation, making it closely related to this paper’s motivation. Can the authors discuss the related work and add some experimental results?

  3. Tab 19 reports an average MSE increase of 1.56% when soft sharing is removed and only 0.46% when age weighting is replaced by uniform weighting. Among the 20 tasks affected by the age-weighting change, uniform weights improve MSE on 13 tasks and worsen it on seven. Can this means that age weighting is not that useful?

Questions for authors

See weakness.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 2DJV

We sincerely thank the reviewer for their careful review, particularly for highlighting the importance of including relevant baselines and identifying ambiguities arising from insufficiently specified experimental settings in our original manuscript. We respond to each weakness below.


w1: I have noticed some mismatch ... Is there a mistake or if I miss something.

Response: There are indeed some inconsistencies in the data presented in the tables. However, these are not errors but rather the result of using different configurations selected according to specific experimental requirements. We have added clarifications to the paper to explain these differences.

(1)Table 4 reports forecasting accuracy. To avoid underestimating baseline performance, we adopt the results reported in TimeBase and PhaseFormer rather than relying solely on our reproduced results.

(2)Table 12 is for analysis. This table serves a different purpose: decomposing prediction errors requires access to forecasts at individual time points. We therefore reproduced the baseline models for this analysis. Our hyperparameter search was conducted within a practical, predefined search space, which we explicitly report below for reproducibility. Learning rate: {1e-4, 5e-4},Dropout: {0.10, 0.25},Hidden dimension: {16,128}, FFN dimension: {128,256}, Encoder layers: {1,2}, Attention heads: {4,8}:

w2: I think the CycleNet is a particularly ... and add some experimental results?

Response: Thank you for your mention CycleNet. We have included it in the baseline of our paper. However, we would like to emphasize that TinyCycle and CycleNet differ fundamentally in their underlying assumptions, model architectures, and design objectives.

(1)TinyCycle and CycleNet characterize different aspects of time series. CycleNet repeats a globally learned periodic template and trains a model to fit the residuals, whereas TinyCycle directly uses the context to model the dominant structure of the data.

(2)CycleNet’s learned global periodicity is distinct from Structure in TinyCycle. The former captures the periodic properties of the dataset itself, whereas the latter is derived from the context and reflects the dominant stable state that the model can capture within that context.

(3) TinyCycle and CycleNet are different types of models. TinyCycle is a lightweight model, whereas CycleNet focuses on improving forecasting accuracy without an explicit lightweight design. The table below compares their efficiency and accuracy. These results will also be included in the revised manuscript.

ModelMSE ↓MAE ↓Params ↓Train (s) ↓Test (s) ↓
TinyCycle0.3210.34525.250.5260.063
CycleNet (Linear)0.3300.349246,102666.1773.313
CycleNet (MLP)0.3350.353545,366903.3734.328

w3: Tab 19 reports an average MSE ... Can this means that age weighting is not that useful?

Response: Thank you for pointing out the lack of rigor in our reporting. Age weighting is essential for certain datasets, while it is optional for others.

The table bellow is case by case result of Table 19(We omit results that are identical because λ=0\lambda=0.). The results indicate that age weighting is necessary to ensure model stability on unstable time series, such as ETTh1 and Exchange. For example, it reduces MSE on ETTh1 from 0.412 to 0.393, although the benefit is dataset-dependent.

DatasetQuadratic MSEUniform MSEQuadratic MAEUniform MAE
ETTh10.3930.4120.4270.441
ETTh20.3460.3390.3840.384
ETTm20.2580.2560.3150.317
Traffic0.4160.4150.2800.280
Exchange0.3840.3860.4130.413

We hope these responses address your concerns, and we would be happy to provide further clarification during the discussion period.

Reviewer_2DJVOpenReview ↗

Response to Authors

I thank the authors for their responses. I have no further questions on this paper.

Reviewer_wouJOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper shows that TSF models are generally better at capturing recurring patterns than changes over time. Based on this finding, the authors introduce TinyCycle, a very simple model that focuses on recurring patterns. Despite using at most 31 parameters, TinyCycle achieves forecasting performance comparable to much more complex models.

Critical strengths
  • TinyCycle, a very compute-friendly approach that rivals some more complicated models at a fraction of the cost. In an era in which increasing data collection and computation are a must, I am in favor of simple, compute-efficient approaches.

  • The Structure-Dynamics decomposition presented in Section 4 is interesting and, to the best of my knowledge, novel.

  • The paper is well-organized and written. It also has several plots and interesting analyses.

Critical weaknesses
  • The paper is missing references and experimental comparisons to two other very simple yet effective baselines, such as AutoARIMA [1] and AutoAR [2]

  • The paper follows the standard benchmark suite of ETT/Electricity/Traffic, etc., but does not evaluate TinyCycle and the baselines on datasets with more correlated channels. Current benchmarks mostly have intra-variate dependencies [3].

[1] Nhits: Neural hierarchical interpolation for time series forecasting, AAAI 2023

[2] Specialized Foundation Models Struggle to Beat Supervised Baselines, ICLR 2025

[3] A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle, ICML 2025

Questions for authors

I like the main idea of the paper and the insights coming from the analyses. Please see the weaknesses.

UPDATED SCORE AFTER REBUTTAL.

6 comments
AuthorsOpenReview ↗

Response to Reviewer wouj

Thank you for recognizing the value of TinyCycle and for your constructive suggestions. Your feedback has helped us improve the manuscript. We provide detailed responses below.

w1: The paper is missing ... such as AutoARIMA and AutoAR

Response: We sincerely thank you for highlighting these two important baselines. We agree that they are highly relevant to our work and have included them in the revised manuscript.

(1)Forecasting accuracy and computational efficiency The table reports overall averages across the eight datasets, with MSE and MAE averaged over all 32 dataset–horizon tasks (H∈{96,192,336,720}H \in \{96,192,336,720\}). Accuracy results are taken from the official papers, while efficiency measurements were obtained on an Intel Core i5-13490F platform. Train and Test are measured in seconds and exclude hyperparameter search. TinyCycle has the lowest average error, parameter count, training time, and test time in this comparison. The best value in each column is shown in bold.

ModelMSE ↓MAE ↓Params ↓Train (s) ↓Test (s) ↓
TinyCycle0.3210.34525.250.5260.063
AutoAR0.3780.3591901.0651.871
AutoARIMA0.8320.533974.12515.1440.385

AutoAR and AutoARIMA combine AR/ARIMA models with automated parameter selection to model the full time series. TinyCycle focuses on Structure, the component that our analysis identifies as central to much of the predictive performance of more complex models. This design avoids modeling the full signal directly and accounts for its favorable average efficiency and accuracy in these comparisons.

(2)Structure–Dynamics decomposition We added the decomposition analysis for AutoAR. Its results show a pattern similar to the other baselines, as detailed in the revised manuscript. We did not extend this analysis to AutoARIMA because its overall forecasting errors were substantially higher than those of the other baselines.

DatasetMSEMAEStructure MSEDynamics MSERemaining waveform
ETTh10.4240.4490.1050.3190.108
ETTh20.3950.4380.1250.2700.065
ETTm10.4190.4140.1950.2240.085
ETTm20.3560.3750.1860.1700.056
Weather0.3320.3410.1750.1570.053
Electricity0.2120.3010.0850.1270.049
Traffic0.4640.3190.0930.3710.216
Exchange2.3101.0181.6740.636—

w2: The paper follows the standard benchmark ... Current benchmarks mostly have intra-variate dependencies

Response: Thank you for you mention up. We added evaluations on the PEMS benchmarks and computed Avg Var Corr following A Closer Look... to characterize their cross-channel correlations.

TinyCycle achieves lower errors than PatchTST by modeling stable Structure, despite lacking an explicit mechanism for cross-channel dependencies. It nevertheless trails methods such as iTransformer, reflecting the limitations of its channel-independent design. We report these results in the revised manuscript. Each forecasting cell reports MSE / MAE. Lower values are better.

DatasetChannelsAvg Var Corr ↑TinyCyclePatchTSTiTransformerSCINet
PEMS033580.84000.146 / 0.2540.180 / 0.2910.113 / 0.2210.114 / 0.224
PEMS043070.76990.154 / 0.2600.195 / 0.3070.111 / 0.2210.092 / 0.202
PEMS078830.80020.143 / 0.2480.211 / 0.3030.101 / 0.2040.119 / 0.234
PEMS081700.78400.220 / 0.2740.280 / 0.3210.150 / 0.2260.158 / 0.244

We hope these responses address your concerns, and we would be happy to provide further clarification during the discussion period.

I would like to thank the authors for their answers, which have addressed a subset of my requests. However, I still have concerns regarding the rebuttal that prevent me from sharing my final judgment of the paper:

  • AutoAR was not originally evaluated on Exchange, so I assume the authors ran the evaluation (including HP search) on this dataset. If you remove Exchange from the evaluation pool (the 7 datasets originally evaluated in the AutoAR paper), AutoAR and TinyCicle get almost identical mean MSE and MAE (actually, AutoAR wins in mean MAE from my quick computation). This result's proximity would have a strong impact on the main paper results (e.g., Figures 1 and 5). The very poor performance of AutoAR on Exchange (e.g., at horizon 720, MSE=2.31) compared to the almost identical results on the 7 remaining datasets doesn't fully convince me, and I suspect there may be issues with the authors' Exchange evaluation.

  • In TinyCycle's revised Appendix Table 5, the AutoAR implementation chooses lags from {64, 96, 128, 192}, while in the original paper, AutoAR uses lags up to 512. The authors of AutoAR explicitly said that longer lags are an important factor for competitive performance. How come the authors stopped at 192, and was this search performed identically for Exchange too?

  • From the requested analysis of inter-channel correlations, TinyCycle is indeed much worse than architectures with channel-dependent design. This important limitation is currently not explicitly addressed in the revised main manuscript but is only mentioned in the Appendix.

AuthorsOpenReview ↗

Re-evaluation of AutoAR and Solutions for Channel Modeling

AutoAR does not achieve satisfactory results on the Exchange dataset.

Response: We sincerely thank the reviewer for the kind and constructive feedback, and we apologize for overlooking this issue. We have extended the lag length of AutoAR to 719 to align it with TinyCycle. We then re-evaluated AutoAR and AutoARIMA and updated the corresponding table and figures related to AutoAR.

However, we must acknowledge that AutoAR still performs relatively poorly on the Exchange dataset. We attribute this to the removal of the moving average (MA) component (Following the original setup of AutoAR). Since Exchange is a dataset with relatively low stability, preserving its mean level is particularly important, which explains why AutoARIMA achieves better performance.

We acknowledge that AutoAR is a very strong baseline, even outperforming some more recent models (e.g., MixLinear, ICLR 2026; TimeBase, ICML 2026 Spotlight). TinyCycle does not claim to outperform AutoAR in predictive accuracy. Rather, our central argument is that modeling the dominant structural patterns alone can recover most of the predictive performance. Nevertheless, TinyCycle consistently maintains advantages over AutoAR in terms of parameter efficiency and computational efficiency.

The following table presents the AutoAR results before and after the revision. Notably, increasing the lag length to 719 does not lead to consistent improvements across all datasets.

DatasetAvg MSEAvg MAE
ETTh10.3948 → 0.4013 ↑0.4173 → 0.4216 ↑
ETTh20.3385 → 0.3391 ↑0.3873 → 0.3895 ↑
ETTm10.3573 → 0.3545 ↓0.3773 → 0.3769 ↓
ETTm20.2548 → 0.2501 ↓0.3118 → 0.3103 ↓
Weather0.2455 → 0.2399 ↓0.2795 → 0.2782 ↓
Electricity0.1683 → 0.1638 ↓0.2615 → 0.2567 ↓
Traffic0.4253 → 0.4149 ↓0.2948 → 0.2883 ↓
Exchange0.8416 → 0.4536 ↓0.5461 → 0.4503 ↓
Average0.3782 → 0.3271 ↓0.3594 → 0.3464 ↓

TinyCycle has limited cross-channel modeling capability.

Response: The central idea of this paper is to model Structure, while the linear model used in TinyCycle serves only as a simple demonstration of this modeling paradigm. The first version following works like PatchTST and AutoAR, is channel independence. However, the core framework of TinyCycle is not tied to any specific model architecture. Therefore, with appropriate modifications, TinyCycle can incorporate cross-channel modeling capabilities.

For a historical lookback window of length HH, conventional approaches typically learn an H→HH \rightarrow H mapping, whereas TinyCycle reformulates the task as an H/P→1H/P \rightarrow 1 mapping. Importantly, this formulation is not restricted to any specific model architecture. Furthermore, this reformulation substantially reduces modeling complexity, potentially improving predictive performance (theoretical proofs are provided in Appendix I).

We explored two extensions of TinyCycle from the perspectives of feature representation and model architecture. At the feature level, we employ Principal Component Analysis (PCA) to extract shared variation directions across multiple channels from the training data. At the architectural level, we aggregate multiple cycles into a single token and introduce a cross-channel attention mechanism to enhance the model's ability to capture inter-channel dependencies. The following table presents the results of these two approaches.

The table is organized with MSE and MAE as rows and models as columns. Only the Overall Avg. results are retained, with the best results highlighted in bold.

Table: Performance Comparison on the PEMS Datasets

MetricTinyCycle-attentionTinyCycle-PCAPatchTSTiTransformerTimesNetTinyCycle
MSE0.0990.1420.2170.1190.1480.166
MAE0.2030.2390.3060.2180.2460.259

Incorporating PCA into TinyCycle leads to a clear improvement in forecasting performance, while introducing cross-channel attention yields even more substantial gains. Although cross-channel modeling is not an inherent capability of the linear model used in the original TinyCycle implementation, the proposed modeling paradigm readily supports the integration of cross-channel modeling mechanisms.

The corresponding code repository has also been updated.


Thank you once again for your valuable feedback and kindness. we would be happy to provide further clarification

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.