acceptodds
Under review as a conference paper at ICLR 2027

ComPass: Demand-Guided Latent Compression for High-Dimensional Time Series Forecasting

Abstract

High-dimensional time series forecasting requires capturing cross-variable dependencies while maintaining computational efficiency. Channel-independent methods scale well but overlook cross-variable dependencies, whereas channel-dependent methods become increasingly expensive as the number of variables grows. Channel compression offers a promising solution, yet existing approaches apply a shared compression scheme without accounting for heterogeneous representational demands across variables. To address this limitation, we propose ComPass, a demand-guided latent compression framework that allocates limited latent capacity according to variable demands. Specifically, a demand analyzer evaluates each variable through a lightweight shared temporal bottleneck, and a budget router converts the resulting scores into continuous budget priors. Attention biases derived from demand guide learnable latent queries, while an alignment objective encourages attention usage to follow the budget priors. To preserve information useful for forecasting under differentiated capacity allocation, we further introduce a budget-conditioned prediction preservation mechanism. During training, this mechanism employs an auxiliary head to generate forecasts from recovered variable representations and their corresponding budget embeddings. The auxiliary forecasts are aligned with outputs from the main forecasting head, which is supervised by the forecasting targets. This prediction consistency favors compressed representations that retain information useful for forecasting under the allocated budgets, without adding inference overhead. Experiments on 16 high-dimensional forecasting benchmarks demonstrate that ComPass achieves strong forecasting performance while handling time series with up to 20,000 variables.

Primary area:
learning on time series and dynamical systems
Keywords:
High-dimensional time series forecasting, variable heterogeneity

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3tTNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

ComPass proposes demand-guided latent compression for high-dimensional time-series forecasting. Rather than compressing every variable uniformly, it estimates how much representation capacity each variable needs and uses that estimate to guide information aggregation into a smaller latent space.

Critical strengths

s1. The paper is well written and clearly organized. The proposed methodology is generally easy to follow, making the motivation, technical design, and experimental evaluation accessible to readers.

s2. The authors address an important challenge in high-dimensional time-series forecasting: how to balance forecasting accuracy, cross-variable dependency modeling, and computational efficiency when the number of variables reaches thousands or even tens of thousands. This is a highly relevant and worthwhile research problem.

s3.ComPass demonstrates promising empirical performance, and the experimental evaluation covers a broad range of benchmarks and analyses. The results suggest that the proposed method has potential for effective and efficient high-dimensional forecasting.

Critical weaknesses

w1. The experimental results do not appear to sufficiently substantiate one of the paper’s central claims: that ComPass adaptively allocates limited representation capacity across variables according to their individual needs. The authors should provide more direct evidence that different variables receive different amounts of effective representation capacity and that such allocation is beneficial.

w2. The paper claims that the proposed compression mechanism preserves key information relevant to forecasting while allocating capacity differentially across variables. However, it remains unclear how this preservation is demonstrated empirically.

w3. It is unclear whether the proposed demand-guided compression mechanism is specific to ComPass or can serve as a general-purpose technique to improve other forecasting architectures.

w4. The scalability study primarily reports the computational cost of ComPass, but does not appear to provide sufficiently comprehensive comparisons with competing methods under consistent experimental settings.

w5. The paper would benefit from a more explicit discussion of its limitations and the conditions under which the proposed approach may be less effective.

Questions for authors

See the Weaknesses.

7 comments
AuthorsOpenReview ↗

Response to W1: Analysis of variable-wise representation capacity allocation and its effectiveness.

We sincerely thank Reviewer 3tTN for the recognition of our work and for providing constructive comments. Below, we address your concerns and questions individually:

We quantify the realized attention usage across variables on Wiki-20k (C=20,000C=20{,}000, K=313K=313). As shown in Rebuttal Table Q, the top 10% highest-demand variables receive 50.40% ± 6.40% of the total attention, compared with only 10% under uniform allocation. The total variation distance (TVD) from uniform allocation is 0.449 ± 0.071, further confirming substantial differentiation in the realized attention distribution. These results show that ComPass realizes non-uniform attention allocation, with a substantially larger attention share assigned to the highest-demand variables.

We further compare demand-guided adaptive allocation against uniform and random budget allocation. As shown in Table 2, ComPass consistently achieves lower MSE and MAE than both alternatives across Neurolib, SP500, and Air Quality. These results demonstrate that the performance gains do not simply come from non-uniform allocation, but from allocating capacity according to estimated variable-wise demands.

Rebuttal Table Q. Realized attention allocation compared with uniform allocation on Wiki-20k.

MetricUniform allocationComPass
TVD from uniform00.449 ± 0.071
Attention share of the top 10% highest-demand variables10.00%50.40% ± 6.40%
AuthorsOpenReview ↗

Response to W2: Empirical evidence of forecasting-relevant information preservation.

Thank you for raising this important question. ComPass addresses differentiated capacity allocation and forecasting-relevant information preservation through two complementary mechanisms, which we further validate through variable-level experiments and ablation studies.

(1) Differentiated capacity allocation and forecasting benefits. To verify that the allocated budgets are reflected in actual attention usage, we measure the correlation between variable-wise budget priors and realized attention usage. As shown in Rebuttal Table R, the Spearman correlations are 0.865, 0.989, and 0.994 for the full model on Neurolib, SP500, and Air Quality, respectively, compared with 0.092, −0.168, and −0.160 without Lalign\mathcal L_{\mathrm{align}}. These results show that budget alignment mechanism effectively guides variable-wise capacity allocation.

(2) Budget-conditioned prediction preservation. The auxiliary prediction head generates forecasts from the recovered representations UrecU^{rec} and their corresponding budget embeddings eibe_i^b, and aligns them with the target-supervised main forecasts through Lpres\mathcal L_{\mathrm{pres}}.

As shown in Table 2 of the paper‾\underline{\text{Table 2 of the paper}}, removing either the preservation head or its budget embedding consistently degrades forecasting performance. These results demonstrate that explicitly constraining the predictive utility of recovered representations, conditioned on variable-wise budgets, contributes to preserving forecasting information under differentiated capacity allocation.

Together, these results provide empirical evidence. We will incorporate these additional analyses and clarifications into the revised manuscript.

Rebuttal Table R. Spearman correlation between variable-wise budget priors and realized attention usage.

DatasetComPassw/o Lalign\mathcal L_{\mathrm{align}}
Neurolib0.8650.092
SP5000.989−0.168
Air Quality0.994−0.160
AuthorsOpenReview ↗

Response to W3: The transferability of demand-guided compression mechanism across forecasting architectures.

Thank you for this valuable question. We clarify the applicability of our design and conduct additional plug-in experiments to evaluate its transferability.

To evaluate the transferability of ComPass, we integrate it into two representative forecasting architectures, iTransformer and DLinear. As shown in Rebuttal Table S, incorporating demand-guided compression improves both architectures, achieving lower MSE and MAE than their original counterparts across the evaluated datasets. These results demonstrate that the proposed mechanism can be transferred to different forecasting architectures in high-dimensional settings, rather than being restricted to the specific design of ComPass.

We also want to clarify that ComPass is designed to allocate limited latent capacity according to heterogeneous variable-wise representation requirements, rather than uniformly compressing all variables. We do not intend compression to be universally beneficial. For low-dimensional datasets, directly modeling all variables may better preserve their dependencies without introducing unnecessary information bottlenecks.

Rebuttal Table S. Forecasting results of iTransformer and DLinear with demand-guided compression, MSE / MAE.

ModelNeurolibSP500Air Quality
iTransformer1.718 / 0.3470.511 / 0.3060.447 / 0.431
iTransformer + demand-guided compression1.675 / 0.3400.496 / 0.3000.438 / 0.425
DLinear1.793 / 0.3810.630 / 0.3670.449 / 0.446
DLinear + demand-guided compression1.721 / 0.3700.608 / 0.3580.438 / 0.437
Reviewer_UU5rOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper targets multivariate time series forecasting in settings where channel counts are high, arguing that existing methods are expensive due to their computational complexity. The authors propose an architecture that allocates compute budget non-uniformly across channels, compressing low-information dimensions. Thanks to this, they are able to lower the cost while claiming state-of-the-art performance. The overall idea is interesting and shows promising results. However, the paper has issues with writing clarity, evaluation, and reproducibility (as described in the weaknesses).

Critical strengths
  1. The explored problem space is very relevant and has been relatively underexplored in the literature. ComPass architecture seems to provide a way to dynamically allocate computational budget across variables while achieving good predictive performance.
  2. The evaluation considers a few angles and ablations beyond simple benchmark predictive performance.
Critical weaknesses
  1. It seems that training uses a complex loss function comprising 5 loss terms, which are somewhat motivated by the authors in Section 3, but their actual role in the architecture is unclear. They are empirically ablated in a very limited capacity, as only LalignL_{align} and LdivL_{div} were held out. Optimization of such a complex Lagrangian, making sure that all loss terms are on appropriate scales and managing training dynamics, is probably quite difficult. The paper should discuss this in detail.

    Furthermore, sensitivity analysis of the compression factor (γ\gamma) and the bottleneck dimension (r) in Appendix B suggests that ComPass is sensitive to both and that the relationship between performance and these hyperparameters is non-monotonic and, quite interestingly, lower compression does not lead to better results (e.g., Neurolib’s γ=8\gamma=8 is worse than γ=16\gamma=16 but γ=32\gamma=32 is better than γ=64\gamma=64). In the case of parameter r, we see that the optimal bottleneck dimensions are different for each dataset. Overall, in this paper there are a lot of design choices and architectural modifications, and it is quite difficult to understand which parts contribute what to the good results we are observing. I think it would strengthen the paper if a thorough analysis of each component was conducted. For instance, correlating some information-theoretic measure of per-channel complexity with the predicted per-channel demand scores would help us believe that the demand analyzer indeed produces useful information and works as intended.

    Moreover, the architecture is very complex and it begs asking whether some components could be simplified without harming the performance while making the architecture simpler and the optimization more stable. For instance, the demand analyzer could be replaced with simple analytical measures of per-channel complexity.

  2. Section 3 is very loaded, and it is not easy to parse since there are numerous symbols introduced in each subsection and paragraph, some of which are vaguely or never defined. For instance ϕenc\phi_{enc}, ϕdec\phi_{dec}, ϕbias\phi_{bias}, ϕenc\phi_{enc}. The total budget, how we should understand it, and what the units are, is never explained. The attention biasing mechanism via adding +1Kϕbias(s)T+1_K\phi_{bias}(s)^T is never explained. Overall, the section is difficult to read, and the writing should be substantially improved and optimized for clarity before publication.

  3. Evaluation robustness is lacking. Table 1 does not average across randomness (seed), prediction lengths (it uses only one prediction length per dataset), and lookback window lengths (this parameter has not been specified anywhere). Moreover, the evaluation code is not open-source, and the results are not reproducible from the paper as baseline training, and their hyperparameters are not detailed.

Minor.

  1. The main forecasting branch applies inverse normalization to predictions and computes loss in the original space. Note that this approach will introduce implicit channel weighting based on the scales of the channels in a given dataset, which can be highly disparate (see, e.g., [1, 2]). Interestingly, the budget-conditioned prediction preservation branch computes the loss between the aux and main branches in the normalized space, which makes it free of implicit channel weighting. Have you considered this design choice and tried computing the main forecasting loss in the normalized space to avoid scale contamination in channel weighting?

Nitpicks:

  • L192: instance normalization is used, but not clearly defined or cited
  • L310: stop-gradient operator could be either explained or cited
  • L391: what is “requirement analyzer”?

[1] G. Berthelier, T. Nabil, E. L. Naour, R. Niamke, S. Perlaza, and G. Neglia “On the role of reversible instance normalization” (2026) https://arxiv.org/abs/2603.11869

[2] I. Stepka, W. Potosnak, K. G. Olivares, A. Dubrawski “Scale-Invariant Training for Time Series Foundation Models” (2026) https://arxiv.org/abs/2610.07324

Questions for authors
  1. Why does the main branch compute loss in the original space while the auxiliary branch in the normalized space?
8 comments
AuthorsOpenReview ↗

Response to W1.1: The roles, contributions, and optimization of the five loss terms.

We sincerely thank Reviewer UU5r for recognizing our work and for the constructive feedback. We address the concerns as follows.

To systematically evaluate the contribution of each loss term, We conduct a ablation of the four auxiliary loss terms. We further evaluate a forecasting-only variant that retains the main forecasting objective but removes all four auxiliary losses, allowing us to assess their collective contribution. The results are reported in Rebuttal Table I, removing any loss results in a decrease in performance.

We further clarify that the five losses serve distinct and complementary purposes within ComPass:

  • Lfore\mathcal L_{\mathrm{fore}} provides supervises the final forecasting outputs against ground-truth targets;
  • Ldem\mathcal L_{\mathrm{dem}} trains the shared temporal bottleneck to estimate variable-wise representational demands;
  • Lalign\mathcal L_{\mathrm{align}} aligns actual attention usage with the intended budget allocation;
  • Ldiv\mathcal L_{\mathrm{div}} encourages diverse attention patterns across latent queries, thereby reducing redundant variable aggregation;
  • Lpres\mathcal L_{\mathrm{pres}} encourages recovered representations to retain forecasting information under differentiated budgets.

To investigate whether jointly optimizing multiple losses requires delicate weight balancing, we conduct an additional loss-weight sensitivity analysis. Specifically, we introduce individual coefficients λi\lambda_i for the five loss terms and vary each coefficient independently over {0.5,1,2}\{0.5,1,2\}, while keeping the other four coefficients fixed at 1. As shown in Rebuttal Table J, ComPass maintains consistently competitive forecasting performance across different loss-weight configurations, with only minor variations in MSE and MAE. These results demonstrate that ComPass is robust to moderate changes in individual loss coefficients and does not require delicate loss-weight tuning.

Rebuttal Table I. Ablation of the four auxiliary losses and their collective contribution.

VariantNeurolibSP500Air Quality
ComPass1.667 / 0.3430.456 / 0.2820.423 / 0.415
w/o LdemL_{\mathrm{dem}}1.705 / 0.3530.464 / 0.2880.437 / 0.426
w/o LalignL_{\mathrm{align}}1.743 / 0.3610.476 / 0.2920.439 / 0.425
w/o LdivL_{\mathrm{div}}1.779 / 0.3680.466 / 0.2870.432 / 0.421
w/o LpresL_{\mathrm{pres}}1.727 / 0.3520.464 / 0.2860.430 / 0.420
Forecasting only1.787 / 0.3740.477 / 0.2960.446 / 0.436

Rebuttal Table J. Loss weight sensitivity results (MSE / MAE).

Varied lossWeightNeurolibSP500Air Quality
ComPass—1.667 / 0.3430.456 / 0.2820.423 / 0.415
LforeL_{\mathrm{fore}}0.51.680 / 0.3410.459 / 0.2830.419 / 0.414
LforeL_{\mathrm{fore}}2.01.687 / 0.3460.460 / 0.2840.425 / 0.417
LdemL_{\mathrm{dem}}0.51.692 / 0.3470.460 / 0.2830.422 / 0.412
LdemL_{\mathrm{dem}}2.01.680 / 0.3410.458 / 0.2830.429 / 0.418
LalignL_{\mathrm{align}}0.51.697 / 0.3480.461 / 0.2840.425 / 0.418
LalignL_{\mathrm{align}}2.01.682 / 0.3450.458 / 0.2830.428 / 0.418
LdivL_{\mathrm{div}}0.51.683 / 0.3460.456 / 0.2820.423 / 0.416
LdivL_{\mathrm{div}}2.01.688 / 0.3460.459 / 0.2830.425 / 0.418
LpresL_{\mathrm{pres}}0.51.699 / 0.3480.462 / 0.2840.428 / 0.419
LpresL_{\mathrm{pres}}2.01.678 / 0.3450.459 / 0.2830.420 / 0.414
AuthorsOpenReview ↗

Response to W1.2 & W1.3: Hyperparameter sensitivity, component-wise contributions, and architectural complexity.

(1) Hyperparameter Sensitivity. We extend the sensitivity analyses to additional datasets under both hyperparameters. As shown in Rebuttal Table K, ComPass maintains relatively stable performance across most datasets. As reported in Table 4 of the paper‾\underline{\text{Table 4 of the paper}}, Neurolib has a sampling interval of only 1 ms, substantially finer than those of the other datasets. Such a fine temporal resolution leads to more rapid temporal variations, making performance differences across hyperparameter configurations more pronounced.

(2) Effect of Compression Strength. We clarify that although weaker compression provides greater capacity to retain input information, it may also introduce more redundant information, potentially degrading forecasting performance. Therefore, weaker compression does not necessarily lead to better forecasting performance.

(3) Bottleneck Dimension rr. The rr controls the representation capacity of the shared temporal encoder in the Demand Analyzer. Different datasets exhibit distinct temporal dynamics and variable heterogeneity, requiring different levels of representation capacity to estimate variable-wise demands.

(4) Component Analysis. Our design addresses core challenges in high-dimensional forecasting, and each component serves a specific purpose:

  • Heterogeneous variable demands: A shared-bottleneck Demand Analyzer estimates variable-wise representational demands through reconstruction errors, avoiding uniform treatment of variables with different temporal characteristics.
  • Limited representation capacity: An Adaptive Budget Router translates estimated demands into differentiated capacity allocations under a fixed global budget, while Demand-Guided Latent Compression uses these allocations to prioritize informative variables during compact cross-variable interaction.
  • Information recovery: Variable-Level Recovery retrieves variable-specific information from the compressed latent representations, avoiding the use of a single shared representation for all variables.
  • Forecasting information preservation: Budget-Conditioned Prediction Preservation encourages the recovered representations to retain information under differentiated budgets, preventing compression from discarding useful predictive information.

Thus, the architecture is principled and necessary to address the heterogeneous representation demands of variables and the limited information capacity in high-dimensional time series forecasting.

(5) Replacing the Demand Analyzer with Simpler Analytical Measures. Following the reviewer's suggestion, we replace the learned Demand Analyzer with two simple analytical measures of per-channel complexity:

  • Permutation Entropy (PE): Captures the diversity of local temporal patterns.
  • Spectral Entropy (SE): Characterizes the dispersion of frequency-domain energy.

As shown in Rebuttal Table L, simple analytical indicators do not lead to performance improvement. ComPass aims not merely to identify which variables are more complex, but to determine how much representational capacity each variable requires under a limited latent budget. PE and SE analyze each variable independently, capturing only its intrinsic statistical complexity without considering its representational requirements under the shared bottleneck.

Rebuttal Table K. Results under different settings of the hyperparameters γ\gamma and rr across multiple datasets.

HyperparameterValueAir QualityM5
Metric—MSE / MAEMSE / MAE
γ\gamma80.422 / 0.4133.402 / 0.834
γ\gamma16 (default)0.423 / 0.4153.394 / 0.832
γ\gamma320.422 / 0.4153.380 / 0.829
γ\gamma640.425 / 0.4193.381 / 0.830
rr20.420 / 0.4123.409 / 0.832
rr40.422 / 0.4113.397 / 0.832
rr8 (default)0.423 / 0.4153.394 / 0.832
rr160.423 / 0.4103.383 / 0.831

Rebuttal Table L. Hypothetical forecasting results with permutation entropy (PE) and spectral entropy (SE) replacing the Demand Analyzer.

DatasetComPassPESE
Neurolib1.667 / 0.3431.740 / 0.3631.785 / 0.372
Wind1.076 / 0.6861.091 / 0.6911.093 / 0.692
AuthorsOpenReview ↗

Response to W2: Explanation of symbols in Section 3.

Thank you for pointing out the insufficient explanations of mathematical notations in Section 3. We have revised this section and streamlined the presentation to improve readability. Here, our detailed explanation is as follows:

  • The ϕenc\phi_{\mathrm{enc}} and ϕdec\phi_{\mathrm{dec}} denote the shared encoder and decoder of the demand analyzer, respectively, mapping each normalized variable sequence from RL\mathbb R^L to a bottleneck representation in Rr\mathbb R^r and reconstructing it back to RL\mathbb R^L.

  • The ϕbias\phi_{\mathrm{bias}} denotes a learnable mapping that transforms the standardized demand vector s∈RC\mathbf{s}\in\mathbb R^C into a variable-wise bias vector b=ϕbias(s)∈RC\mathbf{b}=\phi_{\mathrm{bias}}(\mathbf{s})\in\mathbb R^C, where each element represents the demand-related bias of the corresponding variable. The vector 1K∈RK×1\mathbf{1}_K\in\mathbb R^{K\times1} is an all-ones column vector. Its outer product with b⊤\mathbf b^\top produces a K×CK\times C bias matrix, broadcasting the same variable-wise biases across all KK latent queries. This matrix is added to the content-based attention logits before softmax normalization, allowing the demand signal to modulate attention allocation while retaining query-specific content-based interactions.

  • We further define ϕbud:R→Rd\phi_{\mathrm{bud}}:\mathbb R\rightarrow\mathbb R^d as the shared mapping that transforms each standardized continuous budget prior into a dd-dimensional budget embedding.

  • The BtotB_{\mathrm{tot}} represents the total continuous capacity budget distributed across all variables, satisfying ∑i=1Cki=Btot\sum_{i=1}^{C}k_i=B_{\mathrm{tot}}. Both BtotB_{\mathrm{tot}} and kik_i are expressed in the same continuous budget units. In contrast, KK denotes the actual number of latent queries.

Reviewer_snLiOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ComPass is for high-dimensional forecasting with 1k–20k variables. It compresses the C variable tokens into K=⌈C/16⌉ latent queries using cross-attention biased by a per-variable "demand" score, defined as the reconstruction error of a shared temporal autoencoder. Demands are turned into softmax budget priors. A KL loss aligns average attention usage with the priors, and a diversity loss is added. The pipeline continues with a latent Transformer, cross-attention recovery and residual fusion. A training-only, budget-embedded auxiliary head is distilled from the stop-gradient main forecast. It is best on 14 of 16 datasets from the U-CAST benchmark and runs at 8.2 ms per sample on Wiki-20k.

Critical strengths
  1. It addresses a practically relevant and under-studied regime (C up to 20k), with 16 datasets and 9 baselines.
  2. The design is modular, and the auxiliary branch adds no inference cost. Wins are consistent if modest, and efficiency is comparable to U-CAST.
  3. The analyses are reasonable: uniform and random budgets, alternative compression strategies, and sub-linear or fixed K (App. B.3).
Critical weaknesses
  1. The definition of demand is weakly justified. Demand is the reconstruction error of a shared temporal autoencoder, which mainly reflects how irregular a variable is within the input window, as the visualization in Appendix B.4 also suggests. This does not directly measure how much cross-variable capacity a variable needs for forecasting, and noisy, hard-to-predict variables may receive more capacity. The budget is also enforced only through an attention bias and a KL term on average attention usage, and no per-variable analysis links demand to forecasting gains.
  2. The novelty over U-CAST is incremental. The paper itself describes U-CAST as already compressing variables into a compact latent space, and the additions here are an attention bias, two regularizers and an auxiliary self-distillation head. Removing the preservation head, its budget embedding or either regularizer raises MSE by about 2% to 7% in Table 2, which is similar to the 7% change on Neurolib when the bottleneck dimension r changes from 8 to 4.
  3. The empirical evidence is limited. No seeds or standard deviations are reported, and several margins are small, for example 0.937 versus 0.941 on Meters and 10.249 versus 10.273 on Wiki-20k. The method is also sensitive to its hyperparameters: Neurolib MSE rises from 1.667 to 2.458 when γ changes from 16 to 8, and the paper does not state whether the defaults were chosen on a validation set. Since K grows linearly with C, the compression cost is still quadratic in C, and peak memory on Wiki-20k exceeds that of U-CAST.
Questions for authors
  1. Can results be reported over multiple seeds, and were the hyperparameters selected on a validation set?
  2. Does a variable's demand score correlate with how much it benefits from cross-variable modeling?
  3. At C = 20,000, how far is the realized attention usage from uniform?
  4. How does ComPass compare with TimeFilter, which is cited in the introduction?
  5. What look-back length is used for each dataset, and why are DUET results missing on Solar and Traffic-CA?
6 comments
AuthorsOpenReview ↗

Response to W2: Compare with U-CAST.

U-CAST already employs hierarchical latent queries to compress high-dimensional variables into a compact latent space, and we acknowledge this prior contribution in Section 1 of our paper‾\underline{\text{Section 1 of our paper}}. We would like to clarify that the novelty of ComPass does not lie in latent compression itself, but in its demand-guided capacity allocation and budget-conditioned prediction preservation under heterogeneous variable-wise representational demands. The main differences are as follows:

(1) Demand-guided Capacity Allocation.

U-CAST employs hierarchical latent queries to learn compact variable representations. However, it does not explicitly estimate the representational demands of individual variables or translate them into variable-wise capacity budgets.

In contrast, ComPass introduces a demand-guided allocation mechanism that evaluates each variable through a shared temporal bottleneck and converts the resulting reconstruction discrepancies into continuous budget priors. These priors are incorporated into latent attention through demand-based attention biases, while the budget–usage alignment loss LalignL_{\mathrm{align}} encourages the realized attention usage to follow the intended allocation. Thus, ComPass goes beyond learning compact latent representations by explicitly connecting variable-wise representational demands to the allocation of limited latent capacity.

(2) Budget-conditioned Prediction Preservation.

U-CAST learns compact variable representations but lacks an explicit mechanism to preserve forecasting information throughout the compression–recovery process.

In contrast, ComPass introduces a budget-conditioned prediction preservation mechanism. Specifically, an auxiliary prediction head takes the recovered variable representations together with their corresponding budget embeddings as input, and aligns its forecasts with the outputs of the main forecasting head. By explicitly incorporating budget embeddings into the auxiliary prediction head, ComPass conditions the prediction preservation objective on each variable's allocated budget. Aligning its forecasts with those of the main forecasting head encourages the recovered representations to retain forecasting information under differentiated capacity allocation.

(3) Ablation Evidence.

The ablation results in Table 2 of our paper‾\underline{\text{Table 2 of our paper}} demonstrate that the proposed mechanisms consistently contribute to the forecasting performance improvements of ComPass. Specifically, replacing adaptive budgets with uniform allocation or demand-guided latent attention with linear compression degrades forecasting performance, validating the effectiveness of demand-guided capacity allocation. Similarly, removing the prediction preservation head, its budget embedding, or either regularizer also degrades performance, confirming their respective contributions to prediction preservation, budget–usage alignment, and latent diversity.

Regarding the comparison with the performance variation caused by changing rr, we clarify that component ablations and hyperparameter sensitivity evaluate different aspects of ComPass. All component ablations in Table 2 of our paper‾\underline{\text{Table 2 of our paper}} are conducted under the same bottleneck dimension rr and identical experimental settings, thereby isolating the contributions of individual mechanisms. In contrast, varying rr changes the representation capacity of the Demand Analyzer. Therefore, similar performance variations do not imply that the proposed mechanisms are replaceable by hyperparameter tuning.

AuthorsOpenReview ↗

Response to W3 & Q1: Empirical reliability, hyperparameter sensitivity, and scalability.

Thank you for the insightful comment. We further strengthen the empirical evaluation with multi-seed experiments and clarify the hyperparameter selection protocol and scalability of ComPass.

(1) Statistical reliability and performance comparison.

  • Multi-seed evaluation: Following the reviewer's suggestion, we conduct five-seed experiments on SP500 and Air Quality. As shown in Rebuttal Table D, ComPass achieves the lowest mean MSE and MAE, together with the smallest standard deviations, among the three compared methods on both datasets. These results support its performance advantage and stability across random seeds.
  • Overall performance: As shown in Table 1 of our paper and Rebuttal Table E, ComPass achieves the best or tied MSE on 14 of 16 datasets, with an average and median improvement of 6.00% and 2.77%, respectively. In comparison, U-CAST achieves mean and median MSE improvements of 3.42% and 0.14%, over the strongest baseline excluding both ComPass and U-CAST. These results support the overall performance advantage of ComPass.

(2) Hyperparameter Sensitivity and Selection.

  • We extend the sensitivity experiments to additional datasets under both hyperparameters, γ\gamma and rr. As shown in Rebuttal Table F, ComPass maintains relatively stable performance across most datasets. As reported in Table 4 of the paper‾\underline{\text{Table 4 of the paper}}, Neurolib has a sampling interval of only 1 ms, substantially finer than those of the other datasets. Such a fine temporal resolution leads to more rapid temporal variations, making performance differences across configurations more pronounced.
  • The default hyperparameters are selected based on their performance on the validation set, rather than being tuned on the test set.

(3) Computational complexity and memory efficiency. We clarify that under the default proportional budget policy, K=⌈C/γ⌉K=\lceil C/\gamma\rceil, the compression attention has a complexity of O(KCd)=O(C2d/γ)O(KCd)=O(C^2d/\gamma), which remains quadratic in CC when γ\gamma is fixed. However, the latent interaction operates on only KK queries, with complexity O(K2d)=O(C2d/γ2)O(K^2d)=O(C^2d/\gamma^2). Thus, for γ=16\gamma=16, the respective attention computations are reduced by factors of 16 and 256 compared with full C×CC\times C channel attention.

Importantly, the proportional budget policy is a default configuration rather than an inherent requirement of ComPass. The framework also supports a fixed query budget, K=constantK=\text{constant}, under which the compression and recovery costs scale linearly with CC, i.e., O(KCd)=O(Cd)O(KCd)=O(Cd). The global query budget determines the total latent capacity, while the demand-guided mechanism determines how this capacity is allocated across variables. Therefore, adopting a fixed KK does not remove the demand-guided allocation mechanism.

As shown in Appendix Table 8 of the revised paper‾\underline{\text{Appendix Table 8 of the revised paper}}, fixed K=512K=512 achieves an MSE of 10.257 on Wiki-20k, slightly outperforming U-CAST (10.273), while reducing peak memory by 31.0% and inference latency by 42.9%. These results demonstrate that ComPass can maintain competitive forecasting performance while substantially improving memory efficiency under a fixed query budget.

Rebuttal Table D. Results of five-seed evaluation (mean ± standard deviation).

DatasetModelMSE Mean ± StdMAE Mean ± Std
Air QualityiTransformer0.429 ± 0.0110.418 ± 0.008
Air QualityU-CAST0.432 ± 0.0080.421 ± 0.005
Air QualityComPass0.423 ± 0.0020.414 ± 0.001
SP500iTransformer0.467 ± 0.0250.288 ± 0.010
SP500U-CAST0.553 ± 0.0290.329 ± 0.016
SP500ComPass0.460 ± 0.0080.284 ± 0.004

Rebuttal Table E. Aggregate relative improvements across 16 datasets. ComPass is compared with the strongest baseline on each dataset, U-CAST is compared with the strongest baseline excluding both ComPass and U-CAST.

MetricComPass vs. Best BaselineU-CAST vs. Best Other Baseline
Mean MSE improvement6.00%3.42%
Median MSE improvement2.77%0.14%
Mean MAE improvement4.31%2.94%
Median MAE improvement1.83%0.56%

Rebuttal Table F. Results under different settings of the hyperparameters γ\gamma and rr across multiple datasets.

HyperparameterValueAir QualityM5
Metric—MSE / MAEMSE / MAE
γ\gamma80.422 / 0.4133.402 / 0.834
γ\gamma16 (default)0.423 / 0.4153.394 / 0.832
γ\gamma320.422 / 0.4153.380 / 0.829
γ\gamma640.425 / 0.4193.381 / 0.830
rr20.420 / 0.4123.409 / 0.832
rr40.422 / 0.4113.397 / 0.832
rr8 (default)0.423 / 0.4153.394 / 0.832
rr160.423 / 0.4103.383 / 0.831
AuthorsOpenReview ↗

Response to Q3: At C = 20,000, how far is the realized attention usage from uniform?

We quantify the realized attention usage on C=20,000 using the total variation distance (TVD) from uniform allocation. As illustrated in Rebuttal Table G, the realized attention usage exhibits a TVD of 0.449 ± 0.071, indicating a substantial deviation from uniform allocation. Moreover, the top 10% highest-demand variables receive 50.40% ± 6.40% of the total attention, compared with only 10% under uniform allocation.

These findings indicate that ComPass achieves differentiated attention allocation, with higher-demand variables receiving greater attention.

Rebuttal Table G. Realized attention allocation compared with uniform allocation on Wiki-20k.

MetricUniform allocationComPass
TVD from uniform00.449 ± 0.071
Attention share of the top 10% highest-demand variables10.00%50.40% ± 6.40%

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.