acceptodds
Withdrawn from ICLR 2027

Time DeCode: A Denoising Codebook-Based Model for Time Series Analysis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Time-series models have been widely explored for learning temporal representations from sequential data. However, real-world time series often contain complex noise from measurement errors, stochastic disturbances, and environmental perturbations, which can affect modeling at multiple stages. We identify three interrelated noise dimensions: data-level noise in token representations, objective-level noise in pretraining targets, and inference-level noise in model predictions. To address these issues, we propose Time DeCode, a denoising codebook-based time-series model that connects representation learning and forecasting inference through a learned code space. Specifically, a noise-aware discrete tokenizer separates sparse irregular components from observations and encodes temporal structure into discrete codes. A next multi-code prediction objective learns temporal dependencies by predicting the code indices of future patches rather than their continuous values, reducing reliance on noisy numerical targets. For prediction, code-space prediction calibration aggregates overlapping predictions for the same future position before decoding to improve inference stability. Extensive experiments on forecasting and classification benchmarks demonstrate the effectiveness of Time DeCode, while noise injection evaluations further support its performance and robustness under noisy observations. The code is available at https://anonymous.4open.science/r/Time_Decode/.

Primary area:
learning on time series and dynamical systems
Keywords:
time series analysis, self-supervised learning, denoising, discrete tokenization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hpKDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Time DeCode addresses noise through a discrete tokenizer that separates sparse irregular components and applies frequency-ordered residual quantization. A Transformer predicts future code indices using next multi-code prediction, and code-space prediction calibration aggregates overlapping soft predictions before decoding. Experiments cover seven forecasting benchmarks, ten classification datasets and controlled corruption settings. Forecasting training uses only each target dataset's training split; classification adapts the objective and attention architecture.

Critical strengths
  1. The design is motivated by the characteristics of time-series data, specifically observational noise. The targeted methodological choices and detailed analytical experiments strengthen the empirical credibility of the work.
  2. The codebook-based approach is relatively simple, with moderately coupled components that may offer good modular extensibility.
Critical weaknesses
  1. The methodological design is insufficiently explored. The Noise-Aware Discrete Tokenizer uses “a lightweight network” without specifying its architecture, and the overall structure may introduce representational bottlenecks. The frequency-related ablations do not independently establish the advantage of additive fusion.
  2. The main forecasting experiments cover relatively few datasets and domains, providing insufficient evidence of broad effectiveness. Recent baselines such as TimesFM-3 and Toto 2.0 are missing. Moreover, the short input context of 96 steps may substantially affect performance and raises concerns about the adequacy and fairness of the comparison.
Questions for authors
  1. The evaluated datasets are relatively small. Experience from other modalities suggests that their scale, complexity and pattern diversity may be insufficient to train comprehensive codebooks. Have the authors considered joint pretraining on substantially larger datasets?
  2. Could the method directly support time-series foundation models, or serve as a general-purpose embedding approach that improves their generalization and scalability? Larger-scale experiments would help assess these possibilities.
Reviewer_hFCqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Time DeCode distinguishes noise at the levels of data representations, pretraining targets, and inference, and introduces a component for each. A sparse branch first estimates irregular components, while a residual vector quantizer encodes the remaining signal under sparsity, orthogonality, and frequency-ordering constraints. The tokenizer is then frozen, and a causal Transformer is trained to predict discrete code indices for multiple future patches. Finally, overlapping predicted code distributions are converted into soft embeddings, aggregated with learnable weights, and decoded through code-space prediction calibration (CPC). The authors evaluate seven forecasting benchmarks at four horizons and ten UEA classification datasets, and conduct input-corruption and inference-time code-replacement experiments on ETTh1. Classification uses an adapted bidirectional architecture with masked-code pretraining, rather than directly reusing the same forecasting model. The paper reports average MSE/MAE improvements of approximately 3.8%/3.5% over TimeFilter and an average classification accuracy of 74.18%, exceeding TimesNet by 0.44 percentage points.

Critical strengths
  1. A coherent combination of components with targeted ablations. The discrete tokenizer, future-code prediction, and inference aggregation share a common code space, giving the components a clear relationship. Figure 5 (p. 8) ablates pretraining, the sparse branch, orthogonality loss, frequency ordering, and CPC, and tests replacement with a standard VQ-VAE. These comparisons are more informative about component utility than a comparison of complete models alone.
  2. Accuracy results span multiple datasets and corruption types. Table 1 and the horizon-specific results in Appendix C indicate that the gains are not confined to a single example. Appendix D distinguishes test-input corruption, pretraining-input corruption, input corruption throughout training, and feedback-code replacement. It specifies severity levels and perturbation seeds, and explicitly reports that TOTEM obtains better MSE under 15% block missingness. These details make the robustness results assessable.
Critical weaknesses
  1. The objective-level noise argument does not match the experiments. The paper motivates discrete prediction through noisy targets (p. 2, lines 68–74), but D.3/D.4 corrupt only inputs, leaving targets without additional corruption (pp. 17 and 19–20). D.3 nevertheless interprets the results as objective-level denoising. These experiments support learning from corrupted inputs, but do not directly establish better suppression of target noise through discrete supervision. Intrinsic target noise cannot replace a controlled target-corruption comparison, so the mechanistic interpretation remains unsupported despite the forecasting gains.
  2. Other denoising mechanisms lack direct evidence. The codebook combines nearest-neighbor residual quantization with sparse separation and orthogonality/frequency-ordering constraints (Eqs. 2–8). Ablations show utility, but reconstruction from decoded codes plus a sparse branch does not establish that the branch captures noise rather than genuine signals. The three noise dimensions are plausible pipeline views, though inference-level noise conflates input perturbations with prediction and feedback errors. CPC needs a simple-averaging control with the same prediction count; optimization, baseline initialization, and training-seed variability also require clarification.
Questions for authors
  1. Objective-level noise. How do D.3/D.4 establish target-noise suppression without additional target corruption? Please provide matched input-only, target-only, and joint-corruption comparisons, generating discrete labels from the same corrupted targets and measuring degradation against training without additional corruption. Otherwise, please limit the conclusions to robustness under input corruption.
  2. Implementation and comparison protocols. Please clarify stage-wise trainable parameters and losses, CPC weight learning and causal prediction-window construction, and baseline initialization and training budgets.
  3. Mechanisms and reliability. Is there a capacity-matched standard residual-codebook comparison and direct evidence of noise–signal separation? Does CPC consistently outperform simple averaging across training seeds with the same number of predictions?
Reviewer_DDJ4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Time DeCode, a codebook-based framework for robust time-series analysis that aims to mitigate noise throughout representation learning and forecasting. It combines discrete representation learning with code prediction and prediction calibration. The framework is evaluated on various downstream tasks and under synthetic corruption settings.

Critical strengths

S1. Relevant problem motivation

  • The paper considers noise related challenges at multiple stages of time-series modeling, including representation learning, pretraining objectives, and forecasting inference. This provides a clear motivation for exploring noise-aware time-series modeling.

S2. Competitive results on evaluated benchmarks

  • Time DeCode achieves competitive results on the evaluated forecasting benchmarks. According to the reported results, it obtains the lowest average MAE on all seven datasets and the lowest average MSE on five.
Critical weaknesses

W1. Insufficient validation of the claimed denoising mechanism.

  • The tokenizer is trained without clean signal supervision and reconstructs the original observations. So the sparsity and orthogonality constraints do not establish that the learned sparse component captures noise rather than meaningful temporal patterns. Despite evaluations under synthetic corruptions, the learned noise decomposition is not directly validated.

W2. Inconsistency between the architecture diagram and method description

  • The architecture diagram (Figure 2) does not accurately reflect the method described in the text (Section 3). It conflates tokenizer learning and forecasting into a single pipeline, omits the shadow encoder used for orthogonality regularization, and fails to show the decoder required to generate continuous forecasts in Eq. (13). These inconsistencies obscure the actual training and inference procedures.

W3. Limited evidence for the distinct contributions of the proposed components

  • The proposed framework combines several building blocks, but the distinct benefits of frequency-ordering regularization and code-space prediction calibration remain insufficiently established. The ablation studies do not directly validate the intended frequency hierarchy across codebook layers. Moreover, CPC yields only marginal improvements under the tested inference-corruption settings.
Questions for authors

Q1. Validation of the denoising mechanism

  • How to verify that the learned sparse component captures noise rather than meaningful temporal patterns? Can you provide direct evidence validating the learned decomposition?

Q2. Comparison with recent methods

  • The evaluation lacks comparisons with relevant recent methods, particularly those addressing robustness to noisy or incomplete observations, such as LGA and ChannelTokenFormer (ICLR 26). Also comparisons with recent general time-series models, such as FACT and GTM (ICLR 26), and representation learning methods like PatchCLE (KDD 26), would further strengthen the evaluation.

References:

[1] LGA: Local Geometry Attention for Time Series Forecasting under Realistic Corruptions. ICLR 2026.

[2] ChannelTokenFormer: Towards Robust Real-World Multivariate Time Series Forecasting: A Unified Framework for Dependency, Asynchrony, and Missingness. ICLR 2026.

[3] GTM: GTM: A General Time-series Model for Enhanced Representation Learning of Time-Series Data. ICLR 2026.

[4] FACT: FACT: Fine-grained Across-variable Convolution for Multivariate Time Series Forecasting. ICLR 2026.

[5] PatchCLE: PatchCLE: Breaking the Linear Representation Bottleneck in Time Series Forecasting via Soft Contrastive Learning. KDD 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.