acceptodds
Under review as a conference paper at ICLR 2027

DLL: Deep Latent Learning via Layer-Wise Latent Supervision

Abstract

Traditional autoregressive predictive learning paradigms rely primarily on surface-level observable objectives, such as next discrete tokens or future continuous values. This paper argues that observable samples induce additional targets in the deep latent space, namely the contextualized representations embedded in deep layers. We propose . DLL trains the model to simultaneously predict what the next observable target is and how that target should be represented in the deep latent space. We emphasize the necessity of supervising deep layer-wise representations, as deep layers leverage sequence-wide context through attention mechanisms far more effectively than shallow layers. To complement deep latent supervision, DLL introduces a self-purification module that anchors latent targets to the observables to prevent representation collapse. DLL yields robust improvements on established benchmarks across three domains, such as ImageNet-1K in computer vision, TableBench in natural language processing, and GIFT-Eval in time series forecasting.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Predictive Learning, Deep Latent Representation, Layer-Wise Supervision, Cross-Domain

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_z4LXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Deep Latent Learning (DLL), a training framework that extends conventional autoregressive prediction by introducing layer-wise latent supervision. In addition to predicting the next observable target, DLL encourages the model to predict the latent representations of the next step across multiple network layers. The proposed method introduces a self-purification module to align latent targets with observable targets, together with latent prediction modules that predict these representations from the current hidden state. These auxiliary modules are used only during training and introduce no additional inference-time overhead. The authors evaluate DLL across computer vision, natural language processing, and time-series forecasting tasks and report improvements over conventional training baselines on several benchmarks.

Critical strengths
  1. The paper explores using latent representation prediction as an auxiliary training objective for autoregressive models. The proposed framework is relatively straightforward to integrate into existing architectures without modifying their inference procedures.

  2. The experiments cover multiple domains and model architectures. The authors also provide ablation studies and computational overhead measurements to investigate the contributions and costs of the proposed components.

Critical weaknesses
  1. The proposed method appears to be an incremental combination of existing techniques, including latent representation prediction, representation alignment, deep supervision, and stop-gradient-based training. These ideas have been extensively explored in previous representation learning and self-distillation methods. The proposed objective resembles the use of multiple auxiliary prediction heads for next-step prediction. The paper does not convincingly demonstrate that this particular combination provides a fundamentally new learning mechanism or a substantial methodological contribution beyond existing latent prediction and representation alignment approaches.

  2. Although the authors report improvements across multiple benchmarks, the performance gains are generally modest. For example, the ImageNet-1K accuracy improves from 83.8% to 83.9% on ViT-Base compared with NEPA. The reported gains do not convincingly establish the superiority or necessity of the proposed training framework, particularly given its limited methodological novelty.

  3. The full DLL framework increases training FLOPs per step by approximately 51–52% and peak memory usage by 28–31% compared with the baseline. Despite this additional computational cost, the reported performance improvements remain relatively small across most benchmarks. Given the additional training complexity and computational requirements, the current experimental evidence does not convincingly justify the practical benefits of DLL over simpler training approaches.

Questions for authors
  1. What is the fundamental methodological novelty of DLL compared with existing representation alignment, latent prediction, and deep supervision methods? In particular, how does the proposed framework differ from introducing multiple auxiliary next-step prediction heads? Could the authors provide more direct comparisons with these simpler alternatives?

  2. Can the authors provide stronger empirical evidence that deep layer-wise latent supervision offers substantial benefits beyond existing training objectives? The current improvements are generally modest, and the ablation studies do not fully establish that the proposed latent supervision mechanism is necessary. Comparisons against alternative auxiliary supervision methods would help clarify the contribution.

  3. How does DLL perform under matched training compute budgets? Given the approximately 51–52% increase in training FLOPs per step for the full method, could the authors compare DLL with baselines trained using equivalent total FLOPs or training time? This would help determine whether the reported improvements justify the additional computational cost.

3 comments

Dear reviewer,

Thank you for the valuable comments. We have submitted a revised PDF (changes marked in blue).

(i) Methodological contribution: deep supervision, representation alignment, and latent prediction.

Deep supervision. Deeply-Supervised Nets (Lee, Xie et al., AISTATS 2015) attach auxiliary heads to hidden layers and train them using ground-truth targets, providing direct supervision to intermediate representations and helping mitigate vanishing-gradient issues during training. Subsequent residual connections and normalization schemes offer more direct and elegant ways to address this issue. DLL instead constructs additional predictive targets. Its distinction is explicit:

  • Deep supervision: Aℓ(ht(ℓ))→xt+1A_\ell(h_t^{(\ell)}) \to x_{t+1}—different source layers predict the same observable.
  • DLL: Pℓ(ht(L))→sg(Qℓ(ht+1(ℓ)))P_\ell(h_t^{(L)}) \to \mathrm{sg}(Q_\ell(h_{t+1}^{(\ell)}))—the current final-layer state predicts next-step targets from different depths.

Representation alignment. For example, CLIP aligns image/text pairs; distillation matches teacher/student outputs or features. DLL instead predicts next-position, depth-specific targets that incorporate xt+1x_{t+1}, using only the prefix through tt: a causal prediction task, not same-input teacher imitation or cross-modal matching.

Layer-wise latent prediction. NEPA targets input patch embeddings; NITP uses shallow representations. Original I-JEPA/V-JEPA predict target-encoder outputs, and LLM-JEPA uses last-token/last-layer embeddings. To the best of our knowledge, DLL is the first autoregressive training framework to jointly predict observation-grounded next-step latent targets separately at every depth of the same backbone, using its current final-layer state and without maintaining a separate teacher backbone.

Hierarchical modeling. H-JEPA, first posted in October 2026, after our submission, independently supports the motivation for hierarchical latent prediction. It builds separate JEPA levels at coarser temporal strides; DLL reuses one backbone's depth hierarchy at a fixed next-step offset.

(ii) Effectiveness of layer-wise latent supervision.

We additionally evaluated final-layer-only supervision and have added the result in Table 6 and Figure 4:

Supervision settingMSE ↓MAE ↓
Observable-only baseline0.70550.1484
First-layer-only latent target0.70360.1480
Final-layer-only latent target (new)0.70480.1482
DLL without purification0.70470.1482
Full DLL: layer-wise latent targets0.69810.1457

Full DLL outperforms both single-layer alternatives, supporting the utility of targets from multiple depths in this setting.

(iii) Magnitude and consistency of the improvements.

We agree that the +0.1 percentage-point ImageNet gain on ViT-B is small in magnitude. However, small in magnitude does not imply triviality. Table 1's top-tier ImageNet baselines differ by tenths of a percentage point: 85.2% (BEiT-L/JEPA-L), 85.3% (NEPA-L), 85.6% (MAE-L). DLL reaches 86.0%, reversing the NTP-versus-MIM paradigm ordering in this comparison; its gains over NEPA are +0.1/+0.7 points on ViT-B/L. The larger gain on ViT-L suggests that DLL’s benefits increase with model scale, consistent with the depth and embedding-size ablations in Section 4.6.

According to Table 4, adjacent foundation-model entries on GIFT-Eval leaderboard differ at the third decimal: MASE 0.699/0.698/0.696 for Toto-1B/Chronos-2/Toto-2.5B, and CRPS 0.478/0.476/0.474 for Zeus/Toto-2.5B/t0-beta. Our gains are of the same scale but non-trivial.

(iv) Training overhead and matched-compute evaluation.

Section 5 and Appendix A quantify the trade-off: full DLL adds 51–52% forward FLOPs and 28–31% peak memory. Layer subsampling (r=0.3r=0.3), motivated by representation similarity between neighboring layers, reduces these overheads to 16% and 13–18%, retaining 90% of the TableBench gain and 81%/92% of the Toto MASE/CRPS reductions.

We have not yet conducted an equal-total-FLOPs or wall-clock comparison. However, Table 14 shows that DLL with 3 layers slightly outperforms the 7-layer baseline (MSE 0.7022 vs. 0.7059), using 43% as many layers. With half as many training slices (2 vs. 4), DLL also achieves comparable MSE (0.7049 vs. 0.7059). Table 1 also shows that DLL outperforms MAE on ViT-L (86.0% vs. 85.6%) with only half the epochs (800 vs. 1600).

I thank the authors for the response. However, my main concerns are still unsolved. The novelty of the proposed method is limited. It largely increases computational cost, while the gains are small. I will maintain my original ratings.

Dear reviewer,

We have addressed all your concerns individually. More specifically, we have explained how layer subsampling hugely reduces computational cost while maintaining most of the gains; we also explained why gains small in magnitude do not imply triviality in the context of the benchmarks. Could you please specify which concerns remain unsolved? Thank you.

Reviewer_6N3GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper adds an auxiliary loss to autoregressive training. Besides predicting the next observable (token, patch or value), the model uses its last-layer state at step tt to predict the hidden state of every layer at step t+1t+1. These targets first pass through small per-layer decoders that are trained to reconstruct xt+1x_{t+1} (the "self-purification" step, with stop-gradient), which is meant to prevent collapse. All extra modules are training-only. The method is tested on ImageNet ViT pre-training (vs. NEPA), LoRA fine-tuning of Qwen3-4B on TableBench and CTI-Bench, Toto-2.0 post-training on GIFT-Eval, and smaller forecasting models, with ablations.

Critical strengths
  1. The idea is simple, costs nothing at inference, and the same recipe is applied to vision, language and time series. The sign of the effect is mostly consistent (e.g., 22 wins, 4 ties and 4 losses over 30 sub-datasets in Table 13). If the gains hold up under the issues below, it could be a useful add-on.
Critical weaknesses
  1. Several of the headline numbers have consistency or protocol problems:

    • On TableBench fact-checking, DLL's 78.22% (Table 2) is not a possible score on 96 questions, while the baseline's 78.12% is exactly 75/96.
    • Figure 2 shows the default setting at about 0.94% MSE gain, but Tables 5 and 14 give 1.05% (from 0.7055 to 0.6981).
    • The Toto-2.0 post-training data (Table 12) includes GIFT-Eval test sources (M4, KDD Cup 2018, COVID deaths, Saugeen), yet the ranking is compared with leaderboard entries that have no test leakage, and this is not discussed.
    • The forecasting experiments use an 80:20 train/test split with no validation set, and Sec. 4.5 reports the "best test loss".

    Since all gains are small, each of these can change the conclusion of its comparison.

  2. The paper presents a general training principle for three domains, but it does not name a concrete problem that this principle solves, and there is little analysis of why it works. The method extends NEPA's next-embedding prediction to every layer and adds small decoders, and the evidence is mostly small gains on many benchmarks: +0.1 top-1 over NEPA-B on ImageNet (+0.7 for ViT-L) and GIFT-Eval changes in the third decimal. The latent part itself is also not tested. QℓQ_\ell learns to reconstruct xt+1x_{t+1} from ht+1(ℓ)h^{(\ell)}_{t+1}, which already sees xt+1x_{t+1} as input under teacher forcing, so the "latent" targets may end up close to the observable itself. There is no check of this, and no comparison with plain observable targets at each layer or with a last-layer-only target, so it is not shown that deep latent supervision is what helps.

Questions for authors

Clarification questions:

  1. How many of the 96 fact-checking questions did each model answer correctly?
  2. Were the GIFT-Eval test sources removed from the Toto-2.0 post-training data? Without a validation split, how were checkpoints and default settings chosen?
  3. The latent part itself is also not tested. QℓQ_\ell learns to reconstruct xt+1x_{t+1} from ht+1(ℓ)h^{(\ell)}_{t+1}, which already sees xt+1x_{t+1} as input under teacher forcing, so the "latent" targets may end up close to the observable itself. There is no check of this, and no comparison with plain observable targets at each layer or with a last-layer-only target, so it is not shown that deep latent supervision is what helps.
3 comments

Dear reviewer,

Thank you for the valuable comments. We have submitted a revised PDF (changes marked in blue).

(i) Concern about TableBench fact-checking

Both DLL and DLL-baseline answered 75/96 questions correctly (78.12%). DLL's 78.22% was a transcription error; the overall mixed score was computed from the correct evaluation outputs and remains 26.41. We have corrected Table 2 and replaced "close" with "tied" in the text.

(ii) Figure 2 and Table 14: inconsistent reporting of the EMA variant.

During the early development of DLL, we introduced EMA updates to stabilize latent targets and mitigate representation collapse. The entire hyperparameter sweep used DLL w/ EMA, whose default MSE/MAE are 0.6989/0.1458 (Table 6), corresponding to 0.94%/1.75% improvements over baseline. Later component ablations (Table 6) showed that removing EMA improved these to 0.6981/0.1457 (1.05%/1.82%) while eliminating EMA parameter maintenance. We therefore adopted the variant without EMA in the other experiments (NEPA’s EMA is not used for target generation). Since the EMA variant performed comparably, we retained the earlier hyperparameter study as an evaluation of this DLL variant.

During final cross-checking, a co-author who joined midway, unaware that the unlabeled sweep used EMA, assumed its default results should match the final-model results in Table 5 and believed the discrepancy to be a transcription error. The co-author then replaced Table 14's default entries. Figure 2 and the remaining sweep entries still used EMA results, creating the mismatch. We apologize for the missing label and proofreading error.

We have marked DLL w/ EMA in Section 4.6 and Appendix C.2, and restored all three default rows in Table 14 to 0.6989 MSE/0.1458 MAE. The final-model results in Tables 5 and 6 remain unchanged.

(iii) Toto-2.0 post-training: concern about test-data leakage.

Post-training uses only the train/validation portions, while leaderboard evaluation uses the test portions. We directly use the official splitting procedure in gift_eval.data.Dataset, preserving its temporal boundaries. Thus, M4, KDD Cup 2018, COVID deaths and Saugeen in Table 12 refer to permitted historical portions. The official guidance explicitly permits train/validation use without test-data leakage. Both variants train for two epochs (Table 11), based on validation early stopping. We have clarified this protocol in Appendix B.3.

(iv) Training/test split and the wording “best test loss.”

With the 80:20 train/test split, both variants train for the same prescribed epochs based on preliminary training-loss convergence, and use the final checkpoint, without validation-based early stopping or test-based selection. We have replaced "best test loss" with "final-checkpoint test loss" in Section 4.5

(v) Concrete problem addressed and magnitude of the improvements.

DLL addresses the absence of explicit next-step supervision for depth-dependent contextual representations in observable-only autoregressive training.

Table 1's top-tier ImageNet baselines differ by tenths of a percentage point: 85.2% (BEiT-L/JEPA-L), 85.3% (NEPA-L), 85.6% (MAE-L). DLL reaches 86.0%, reversing the NTP-versus-MIM paradigm ordering in this comparison; its gains over NEPA are +0.1/+0.7 points on ViT-B/L. Appendix C.4 also reports better patch-projection conditioning: approximately 1,800 for DLL-Base versus 7,800 for NEPA-Base, consistent with alleviated representation collapse at this projection.

According to Table 4, adjacent foundation-model entries on GIFT-Eval leaderboard differ at the third decimal: MASE 0.699/0.698/0.696 for Toto-1B/Chronos-2/Toto-2.5B, and CRPS 0.478/0.476/0.474 for Zeus/Toto-2.5B/t0-beta. Our gains are of the same scale.

(vi) Concern about purified latent targets reducing to observable targets and necessity of going deep .

Detaching the input confines LpurL_{pur} to the lightweight QℓQ_\ell modules (normally MLPs): no gradient from LpurL_{pur} reaches the backbone. This makes exact reconstruction difficult.

As discussed in Section 2, Transformer representations evolve across depth, motivating targets drawn from different levels of contextual abstraction. We additionally evaluated final-layer-only supervision and have added the result in Table 6 and Figure 4:

Supervision settingMSE ↓MAE ↓
Observable-only baseline0.70550.1484
Final-layer-only latent target (new)0.70480.1482
First-layer-only latent target0.70360.1480
Full DLL: layer-wise latent targets0.69810.1457

Full DLL outperforms both single-layer alternatives, supporting the utility of targets from multiple depths.

Reviewer_6N3GOpenReview ↗

Response to Authors' Rebuttal

Thank you for the response and the revision. The corrections to Table 2, the EMA labelling of the sweep, the GIFT-Eval split and the final-checkpoint protocol clarify those points, and the new final-layer-only result is helpful.

However, my main concern remains and I still tend to rejection.

  1. The "latent" targets may just be the observable. By Eq. (3), the purification module of every layer is trained to output the same thing, xt+1x_{t+1}. So the targets from all layers may be nearly identical, and predicting them is close to predicting xt+1x_{t+1} several times. The response does not check this: it does not measure how different the targets are across layers, and it does not compare with heads that predict xt+1x_{t+1} directly. Table 6 also shows that the gain disappears without purification (0.7047 vs. 0.7055 for the baseline).
  2. The evidence for depth is thin. The new comparison differs only in the third or fourth decimal (0.7048, 0.7036, 0.6981), and these experiments have no validation split.

Dear Reviewer,

Thank you for acknowledging the clarifications. We would like to clarify the purpose of self-purification and the non-triviality of the reported improvements and evaluation protocol.

1. Why self-purification is needed, and whether it perfectly reconstructs the observable.

We examined the purification loss LpurL_{pur} at the end of the 800-epoch ImageNet ViT-L pre-training run. Under the negative-cosine objective, the average purification loss across layers is approximately −0.91, rather than the optimum of −1, indicating that the purified outputs have not achieved perfect directional reconstruction. The first-layer purification loss is -0.99, while the last-layer purification loss is -0.83.

As discussed in “Representation Collapse and Target Stabilization” in Section 2, self-purification is introduced to keep latent targets grounded in the observations and reduce the risk of degenerate representations. The result without purification—MSE 0.7047, compared with 0.7055 for the observable-only baseline and 0.6981 for full DLL—shows that implementations without this grounding yield little benefit in our setting. This supports the importance of purification for the observed improvement.

Appendix C.4 provides complementary evidence: near the end of training, the patch-projection condition number is approximately 1,800 for DLL-Base versus 7,800 for NEPA-Base, consistent with better spectral conditioning and alleviated collapse at this projection.

Importantly, the purification inputs are detached, so the purification loss directly updates only the lightweight modules QℓQ_\ell (normally MLPs), without directly updating the backbone to make its representations easier to reconstruct. The measured purification losses show that reconstruction fidelity varies across depth: the first-layer output is nearly aligned with the reconstruction target, whereas the final-layer output retains a substantially larger reconstruction residual.

2. Small absolute differences should be interpreted in context, and fixed-budget evaluation does not select the best test checkpoint.

Small gains in magnitude do not imply triviality. Our experiments use large-scale, heterogeneous data from the Chronos collection, which was introduced for time-series foundation-model training and evaluation [1]. In such settings, small absolute differences can distinguish competitive models. For example, the adjacent GIFT-Eval foundation-model entries reported in Section 4.4, Table 4 have MASE values of 0.699/0.698/0.696, with differences on the same scale as our improvements. Furthermore, achieving consistent improvements across heterogeneous datasets with a shared configuration is challenging. Without dataset-specific hyperparameter tuning or additional inference-time cost, DLL achieves 22 wins, 4 ties, and 4 losses across 30 sub-datasets, with a 2.20% reduction in average MSE, supporting its broad applicability.

Regarding the absence of a validation split, our comparison uses the same prescribed training epochs and the final checkpoint for DLL and its baseline, without test-based checkpoint selection. Under such large-scale training data, fixed-epoch/step training practice is not uncommon: NEPA reports 1600/800 pre-training epochs for ViT-B/L [3]; Chronos reports 200K training steps [1]; and the TimesFM scaling study trains models for 1.5M iterations [2]. Our protocol evaluates the final checkpoint under a common epoch budget; it does not choose the checkpoint with the best test loss, especially when training on large-scale data.

Thank you for your continued discussion.

References

[1] Ansari, A. F., et al. Chronos: Learning the Language of Time Series. 2024. See Sections 5.1–5.2 for the dataset collection and the 200K-step training protocol.

[2] Das, A., Kong, W., Sen, R., and Zhou, Y. A Decoder-Only Foundation Model for Time-Series Forecasting. ICML, 2024. See Section 6.2 for the 1.5M-iteration scaling study.

[3] Xu, S., et al. Next-Embedding Prediction Makes Strong Vision Learners. 2025. See Section 4.3 for the 1600/800-epoch pre-training schedules.

Reviewer_8nBPOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose deep latent learning (DLL), a topic that has gained significant traction in recent research. The proposed method predicts the next observation and the target representation in the deep latent space. A self-purification module is designed to anchor latent targets to the observables to prevent representation collapse in deep latent supervision. DLL yields improvements on established benchmarks across computer vision, natural language processing, and time series forecasting domains.

Critical strengths
  1. The proposed deep latent learning uses contextualised representations in deep network layers as auxiliary supervisory targets, to predict both further observations and lay states, via capturing rich sequence-wide context without sacrificing granular details.

  2. Extensive evaluations demonstrate that the latent prediction method yields superior performance on the benchmarks across three domains.

Critical weaknesses
  1. There are some similar studies of the latent learning in the recent time series research, such as LatentTSF [1] and WorldTS [2]. The idea looks very similar. Could the author explain the differences between your idea and existing latent models? What are the advantages of the proposed DLL?

  2. The code of this paper is not submitted. It would be better to show the code for enhancing reproducibility.

[1] From Observations to States: Latent Time Series Forecasting, 2026.

[2] WorldTS: World Modeling for Multimodal Covariate-aware Time Series Forecasting, 2026.

Questions for authors

Why choose the time series foundation models to validate the effectiveness of DLL? How does it work on classical time series models (such as iTransformer, PathTST, and TimesNet) as well as LLM-based models (including Time-LLM, TimeCMA, and LLMTime)?

3 comments

Dear reviewer,

Thank you for the valuable comments. We have submitted a revised PDF (changes marked in blue).

(i) Relationship to LatentTSF and WorldTS.

LatentTSF follows a standard autoencoder(AE)-based approach, using a pretrained and frozen point-wise autoencoder for target construction. DLL instead derives targets from the backbone's own contextualized representations across layers, avoiding the additional cost of separately pretraining an autoencoder while introducing no additional inference-time overhead. WorldTS learns covariate-conditioned latent dynamics with state-matching and VICReg objectives, then freezes the encoders and predictor to train an observation decoder. DLL combines three features: single-stage training without separate autoencoder pretraining, observation-grounded targets from multiple backbone depths, and no additional inference-time cost.

(ii) Code and reproducibility.

We provide the core DLL implementation below for inspection. We are also preparing the full codebase for public release by standardizing its structure and coding style, adding clear explanatory comments, and providing comprehensive documentation.

# Q and P: layer-wise MLPs (d -> 2d -> d), including depth 0.
def dll_losses(h, clean_embeddings, Q, P, loss_fn):
    q = torch.stack([Q_l(h_l.detach()) for Q_l, h_l in zip(Q, h)], dim=2)
    p = torch.stack([P_l(h[-1]) for P_l in P], dim=2)
    targets = clean_embeddings.unsqueeze(2).expand_as(q)
    L_pur = prediction_loss(q, targets, loss_fn)
    # Align predictions at t with detached targets at t+1.
    L_lat = prediction_loss(p, q.detach(), loss_fn, shift=True)
    return L_pur, L_lat

(iii) Why time-series foundation models, and what about classical and LLM-based forecasters?

Our choice of time-series foundation models is motivated by the broader goal of DLL: to expand the sources of supervision available during model training by introducing deep, layer-wise latent targets. We intend DLL to be applicable across modalities and application settings. The Transformer-based time-series foundation models evaluated in our paper share key architectural principles with our vision and language backbones, providing a natural setting for examining this common training mechanism across domains.

We did experiment on the classical time series model PatchTST in Section 4.5. In this section, we evaluated DLL across different Transformer backbones, in which we adopted PatchTST as the encoder-only implementation. As reported in Table 5, PatchTST's average MSE across four TSlib sub-datasets decreases from 0.2647 to 0.2630, a 0.64% relative reduction. Due to page limits, we did not put the detailed results in the main body. Below are the detailed results (which have been added to Appendix C.5):

DatasetDLL MSEDLL-Baseline MSEMSE Improvement (%)
ETTm10.30050.30371.06
ETTm20.17190.17461.54
Traffic0.42210.42240.08
Weather0.15730.15810.51
Average0.26300.26470.64

PatchTST provides temporally indexed patch representations. In contrast, iTransformer operates on variate tokens, while TimesNet models period-based 2D representations. Applying DLL to these architectures requires an explicit definition and alignment of next-step latent targets; we have not evaluated these specialized extensions yet.

Regarding LLM-based forecasters (Time-LLM, TimeCMA, and LLMTime), our validations of DLL on both causal LLM fine-tuning (Qwen3-4B in Sections 4.2–4.3) and native time-series Transformer backbones (Sections 4.4–4.5) directly support its compatibility with LLM-based forecasting pipelines. Extending DLL from our current cross-domain evaluations to joint multimodal reprogramming frameworks is a natural next step. Notably, with DLL, the next-token prediction (NTP) paradigm outperforms MAE, a representative masked image modeling (MIM) method, on ImageNet-1K in Section 4.1—demonstrating DLL's potential to strengthen representations across modalities within unified autoregressive architectures.

Thank you for the authors' response and clarifications. However, my main concerns remain insufficiently addressed. The discussion of LatentTSF and WorldTS mainly highlights architectural differences without providing empirical comparisons to demonstrate the advantages of DLL. More importantly, the additional experiments are still limited to PatchTST, with only a marginal average MSE improvement of 0.64%. The requested evaluations on other classical forecasting models, such as iTransformer and TimesNet, and LLM-based methods, such as Time-LLM and TimeCMA, have not been conducted. Furthermore, the revisions to the manuscript appear limited, and the full codebase remains unavailable for reproducibility.

Dear reviewer,

Regarding PatchTST, you mentioned "the additional experiments are still limited to PatchTST". Nevertheless, this is not an additional experiment. It was in Section 4.5 and Table 5 of our paper originally, before the rebuttal revision.

Regarding LatentTSF and WorldTS, your original questions are: "Could the author explain the differences between your idea and existing latent models? What are the advantages of the proposed DLL?" You were just asking about the differences between these models and DLL and how the differences serve as the advantages of DLL over these models. You did not explicitly ask for empirical comparisons here.

Regarding "revisions to the manuscript appear limited", the relevant issue is whether the changes address specific concerns, rather than how extensively the manuscript has been rewritten. Too much revisions would make it a different/new paper.

Regarding iTransformer, TimesNet, Time-LLM, TimeCMA and LLMTime, we acknowledge that we have not conducted the additional experiments - it is very demanding to conduct all these experiments within the limited rebuttal period. We have mentioned that applying DLL to iTransformer and TimesNet requires specialized extensions. We only demonstrated the cross-domain capability rather than the multimodal capability of DLL in our paper, so we mentioned that accommodating Time-LLM, TimeCMA and LLMTime would be a next step.

Regarding full codebase, we mentioned that we were "preparing the full codebase for public release by standardizing its structure and coding style, adding clear explanatory comments, and providing comprehensive documentation". This is in progress and time consuming. We also provided the core DLL implementation in our response.

We completely respect your right to lower the rating, but believe that the reasons given do not fully take into account the clarifications and evidence provided in our response.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.