acceptodds
Under review as a conference paper at ICLR 2027

REQUA: RESIDUAL-GROUNDED QUERY ADAPTATION FOR COVARIATE-AWARE FORECASTING

Abstract

Recent time-series foundation models (TSFMs) increasingly support native multivariate forecasting and have demonstrated substantial gains over univariate modeling on multivariate tasks. However, they always treat all covariates equally, which implies that adapting to downstream tasks, where the focus is on predicting a subset of target covariates, may require fundamentally altering the model’s internal structure. It is reasonable to expect that such adaptation cannot be achieved under the conventional fine-tuning paradigm on the pretrained models. Therefore, in this paper, we introduce ReQuA, a lightweight covariate-aware adaptation framework that keeps the TSFM and its original prediction pathway frozen. ReQuA constructs queries conditioned on the target history and forecast horizon to aggregate task-relevant auxiliary information. The backbone’s forecast-generating latent states attend to this representation to obtain a residual correction, whose magnitude is controlled by a single coefficient calibrated on held-out data. Experiments on eight widely used time-series datasets demonstrate that ReQuA delivers substantial and consistent improvements across frozen TSFM backbones with few additional parameters and little computational overhead, while global calibration effectively suppresses harmful corrections.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Time-Series Foundation Models, Multivariate Time Series, Query-Based Adaptation, Residual Learning, Frozen-Backbone Adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_nVs2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a novel module, ReQuA, which helps time-series foundation models (TSFMs) leverage auxiliary variables to capture residual prediction errors that the original model has not yet eliminated. The proposed framework consists of query aggregation, residual grounding, and global calibration.

Critical strengths

The method is well-designed, with a coherent set of complementary mechanisms. In addition to the Query Aggregator, the authors introduce residual grounding and a second-stage global calibration procedure. Residual grounding encourages the query representation to learn error structures that remain unexplained by the frozen backbone, while calibration uses a global scalar to control the magnitude of the correction and mitigate the risk of overcorrection.

Critical weaknesses
  1. The related work section discusses several existing covariate-aware adaptation methods for time-series foundation models (TSFMs), yet these methods are not included in the experimental comparisons. The authors' justification for excluding them is insufficient. I would expect these recent and relevant methods to be included in the evaluation.

  2. Given that the reported results vary across different TSFMs, evaluating the proposed method on only a limited number of time-series foundation models as backbones is insufficient. The authors should consider a broader range of models to better establish the generalizability of the proposed approach.

  3. The authors should provide a more in-depth analysis of the conditions under which residuals remain predictable. If a sufficiently powerful model produces residuals between its predictions and the ground truth that are nearly unpredictable and resemble random noise, the proposed method may have corresponding limitations. This possibility should be investigated and discussed more thoroughly.

  4. Figure 1 requires further refinement, particularly in its visual presentation. The overlap between arrows and figure boundaries detracts from the clarity and polish of the figure, making it fall short of the standard expected of a formal framework diagram in a research paper.

Questions for authors

See in the Critical Weakness part.

2 comments
AuthorsOpenReview ↗

**Response to Reviewer nVs2**

Thank you for the helpful suggestions, and in particular for the question of when residuals remain predictable. We examined this with a negative control, and we also added the comparisons and backbones you requested.

Q1. Comparisons with existing adaptation methods

We added UniCA, ChronosX, and CoRA, together with supervised iTransformer and TimeXer. The table reports the mean of per-dataset percentage error reductions against native-multivariate frozen Chronos-2 on the same eight tasks (higher is better).

MethodΔMAE (%)ΔMSE (%)Trainable params
iTransformer−1.10+12.016.54M
TimeXer−0.85+11.182.30M
ChronosX+7.50+18.902.17M
UniCA+12.09+19.2215.9M
CoRA+1.73+9.229.42–11.7M
ReQuA+8.69+19.501.70M

UniCA is ahead on average MAE. The average MSE values are close (19.50 for ReQuA, 19.22 for UniCA), and ReQuA uses 1.70M trainable parameters and 5% of the downstream training origins. Because CoRA is run through a channel-independent Chronos-2 interface and ReQuA adapts the native multivariate model, the comparison is not strictly like-for-like.

Q2. Broader backbone evaluation

In addition to Chronos-2, TimesFM-3, and Moirai-1.1, we evaluated ReQuA on Chronos-Bolt-base and Timestral-28M (Mistral AI, October 2026). Improvements are computed against each model's own frozen baseline and then averaged across datasets.

BackboneΔMAE (%)ΔMSE (%)Datasets with MSE gains
Chronos-2+8.69+19.508/8
TimesFM-3+3.95+20.338/8
Moirai-1.1+11.36+19.507/8
Chronos-Bolt-base+11.84+23.268/8
Timestral-28M+3.41+10.828/8

These five backbones differ in architecture and output head. Chronos-Bolt is evaluated at horizon 64 (24 for ILI) instead of 96. The Moirai-1.1 average MSE gain equals the Chronos-2 value by coincidence.

Q3. Conditions for residual predictability

Let I denote the information available to the adapter at inference time, and define the frozen-model residual as r = y − ŷ_frozen. Under squared error, the optimal unconstrained additive correction is g*(I) = E[r | I]. If r is independent zero-mean noise, then g*(I) = 0.

We tested this boundary on ETTh1 with an MSE-matched negative control. We replaced the training, validation, and test targets with ỹ = ŷ_frozen + ε, where ε is independent Gaussian noise matched to the second moment of the real frozen residual. ReQuA's training, checkpoint selection, residual grounding, and held-out scalar calibration were unchanged.

Residual targetStage-1 ΔMSECalibrated αFinal ΔMSE
Real ETTh1 residual+8.21%0.755+7.52%
MSE-matched independent noise−0.093%0.092−0.0010%

Frozen-model test MSE was closely matched (0.3999 vs. 0.4022). On independent noise, the MSE gain disappears and calibration shrinks the correction scale to 0.092, whereas with real residuals ReQuA retains a +7.52% MSE improvement. ReQuA's gain depends on the frozen residual being predictable from the auxiliary history. Our horizon study (Appendix E.2) shows the same pattern. On ETTh1 at H = 720 the gain disappears (ΔMSE −1.04%), consistent with the residual becoming less predictable from historical covariates at long horizons.

Q4. Figure 1

We redesigned the framework diagram so that arrows no longer overlap module boundaries and frozen and trainable modules are visually distinct. The horizon-conditioning path is now explicit, and the correction is shown to modify only the target's forecast-generating latent states.

Thanks to the author for the additional experiments, this solved most of my concerns, and I’ll be raising my score.

Reviewer_zHm1OpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ReQuA, a lightweight method for adapting frozen multivariate time-series foundation models (TSFMs). The core idea is to use the target and forecast horizon to query historical auxiliary-variable states, producing a representation that acts as a residual correction to the target's forecast states. The module is trained using a joint forecast and auxiliary residual-prediction loss, with a scalar correction weight calibrated afterward on held-out data. Empirically, ReQuA improves Chronos-2 by an average of ~8.7% in terms of MAE and ~19.5% in MSE across eight datasets, and the authors round out the evaluation by testing on TimesFM-3 and Moirai-1.1 as well as comparing against parameter-matched residual adapters and LoRA across eight datasets.

Critical strengths
  1. The target-conditioned query mechanism is a really clean design choice. Because it compresses an arbitrary number of auxiliary series into a fixed-size representation to nudge the target forecast states, it keeps the backbone completely frozen. Relying strictly on historical covariates also makes the method much more practical for real-world settings where future covariates aren't available at inference time.
  2. Rigorous and transparent empirical setup: I appreciate that the evaluation goes well beyond simply showing that adaptation beats a frozen backbone. The authors run three-seed evaluations across three distinct families (Chronos-2, TimesFM-3, and Moirai-1.1), include fair parameter-matched baselines in Appendix E.3, and carefully split the data used for checkpoint selection versus post-hoc calibration. Being upfront about failure modes—especially at longer horizons—makes the empirical story much more credible.
Critical weaknesses
  1. Missing comparisons and positioning against recent covariate adapters: The paper would benefit from a much closer comparison with existing covariate-adaptation methods. For instance, UniCA (ICLR 2026) is cited, but not included as an empirical baseline. Additionally, closely related approaches like ChaTSFM (ICML 2026) and ChronosX (AIstats 2025) are omitted entirely. The authors should tone-down the "first-framework" novelty claim, even accounting for differences in backbones and training regimes.
  2. It is not clear whether the residual baselines receive calibration comparable to ReQuA's. Section 5.2 and Appendix D.1 make a compelling case that scalar shrinkage prevents over-correction, yet Appendix E.3 only describes applying this calibration step to ReQuA. This is especially important when interpreting the single-seed results in Table 14: LoRA and ReQuA achieve nearly identical average MAE gains (6.9% vs 6.84%), with ReQuA's main advantage being fewer negative transfer cases. Applying the exact same held-out calibration step to the baseline adapters is essential to separate the contribution of the query design from the contribution of correction scaling.
Questions for authors
  1. Please compare with UniCA, ChaTSM and ChronosX on the existing forecasting tasks.
  2. If you apply post-hoc scalar calibration on the same validation split to the residual baselines (or tune a global scale on LoRA's updates), does ReQuA still yield fewer negative cases? Additionally, to separate the impact of conditional query aggregation from simply having direct access to auxiliary states in the adapter (beyond what the multivariate backbone passes through), how does ReQuA compare against a simple parameter-matched correction branch given the same auxiliary inputs, residual loss, and calibration?
  3. I found Figure 1 difficult to follow without repeatedly checking the equations. The horizon input is missing, the correction appears to modify the full hidden representation rather than only the forecast-generating states, and some labels differ from the text.
  4. Try not to number equations if you are not going to referring back to them in the text.
  5. Please list the profiling details (hardware, batch size, warm-up, etc.) behind the ~0.003% overhead reported in Figure 2(a).
2 comments
AuthorsOpenReview ↗

**Response to Reviewer zHm1**

We thank the reviewer for the careful assessment, particularly the suggestion to test calibration across different adapters. This prompted a revised comparative analysis alongside new covariate-adapter experiments and more detailed profiling results.

Q1. Existing covariate adapters and novelty positioning

We evaluated UniCA and ChronosX on the same eight forecasting tasks, reporting percentage error reductions relative to frozen Chronos-2 (mean ± standard deviation across seeds; higher is better).

MethodΔMAE (%)ΔMSE (%)
ChronosX+7.50 ± 0.28+18.90 ± 0.47
UniCA+12.09 ± 0.27+19.22 ± 0.41
ReQuA+8.69 ± 0.50+19.50 ± 0.65

ReQuA achieves the highest average MSE improvement among these methods (+19.50%), leading UniCA on six of eight datasets, while also obtaining an +8.69% average MAE improvement. This performance requires only 1.70M trainable parameters and 5% of downstream training origins, compared with UniCA's 15.9M parameters and full training pool.

ChaTSFM could not be evaluated because its public repository currently provides neither runnable code nor checkpoints; its zero-shot channel adaptation of cross-domain-pretrained models also differs from ReQuA's dataset-specific adaptation of native multivariate TSFMs. We will expand Related Work to discuss ChronosX and refine the original "first-framework" wording.

Q2. Fair calibration and the contribution of query aggregation

Applying held-out scalar calibration to residual MLP and LoRA improved both baselines, confirming its broader utility for controlling correction strength. Since the original negative-adaptation count comparison does not isolate the query architecture from this general calibration effect, we have removed that comparison and its uniqueness claim while retaining scalar calibration in ReQuA's two-stage procedure.

To distinguish query aggregation from direct auxiliary-state access, we additionally implemented Aux-Direct, a correction branch based on future-to-auxiliary cross-attention. Aux-Direct and ReQuA share the frozen backbone, historical auxiliary inputs, injection point, residual-grounding objective, 5% training-origin pool, and held-out calibration, with comparable parameter counts (1.79M and 1.70M, respectively). The table reports three-seed mean reductions relative to frozen Chronos-2 (ΔMAE / ΔMSE; higher is better).

DatasetReQuA ΔMAE / ΔMSEAux-Direct ΔMAE / ΔMSE
ILI39.23 / 64.0024.08 / 44.56
ETTh10.85 / 6.631.24 / 6.23
ETTm11.40 / 9.881.94 / 7.26
ZafNoo4.37 / 15.054.42 / 10.51
AQShunyi4.09 / 12.633.99 / 9.55
Weather1.85 / 11.132.44 / 8.71
PEMS0416.95 / 31.296.51 / 11.49
Electricity0.78 / 5.421.05 / 3.52
Mean8.69 / 19.505.71 / 12.73

Under matched auxiliary inputs, residual grounding, and calibration, ReQuA improves MSE more than Aux-Direct on every dataset and achieves a 6.77-percentage-point larger mean MSE reduction (19.50% versus 12.73%). Its mean MAE reduction is also higher (8.69% versus 5.71%), providing direct evidence for the value of conditional query aggregation.

Q3. Figure 1

We redesigned Figure 1 to make the forecast horizon h and target- and horizon-conditioned query initialization explicit, while showing that the correction affects only the target's forecast-generating states; its notation now follows Section 4 throughout.

Q4. Equation numbering

We have removed unnecessary equation numbers and added explicit textual references to the equations that remain numbered.

Q5. Profiling methodology and Figure 2(a)

The ~0.003% shown in Figure 2(a) refers to relative FLOPs overhead rather than measured inference latency, and we will relabel the panel accordingly. We used PyTorch's FlopCounterMode on an RTX 4090 in BF16 with dataset-matched synthetic inputs and batch sizes of 32 for Chronos-2 and 16 for TimesFM-3. Weighting dataset-level ratios by frozen-backbone inference time gives overheads of 0.00333% and 0.0031%, respectively.

For latency measurements, we used five warm-up and 15 timed iterations with CUDA synchronization; observed differences ranged from approximately −5% to +11% across datasets, with negative values attributable to timing noise.

Thank you for the additional experiments. Comparing with Aux-Direct, UniCA and ChronosX have addressed my concerns. I will raise my rating to 3.

Reviewer_4LmDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces an adapter for multivariate time series foundation models named ReQuA. ReQuA uses target- and horizon-conditioned queries to correct the latent states used to generate target forecasts, while keeping the TSFM backbone and its predictions frozen. The goal of the method is to recover useful auxiliary information from each dataset that the pretrained model has not sufficiently explored during inference. Training is supervised by prediction loss and residual supervision, while the correction strength is calibrated. The experimental results show generally positive improvements across 8 datasets and 3 backbones, including comparisons against other simple adapters.

Critical strengths
  • The method provides a simple way to further improve the performance of pretrained time series foundation models by training an adapter while keeping the backbone frozen.
  • The proposed method shows consistent improvements.
Critical weaknesses
  • First of all, the proposed method is a per-dataset method, which means the adapter needs to be trained separately for each dataset. Therefore, the experiments need to demonstrate improvements not only over TSFMs but also in comparison with supervised methods.
  • Second, the method requires per-dataset optimization like supervised methods which introduces additional overhead over the inference of TSFMs. This is crucial, as the one goal of TSFMs is to remove the need for repeated per-dataset training and hp tuning. Therefore, a comparison of the training and hyperparameter tuning costs of supervised methods against those of training this adapter (ReQuA) on top of TSFMs is essential to demonstrate its practical usability.
  • Third, the Residual MLP and Tiny Transformer adapter baselines lack a separate branch that directly reads historical auxiliary states. Therefore, it is not fully established whether the benefits come from the architectural design itself or from direct access to historical auxiliary states. I believe a matched auxiliary-aware comparison is essential to support the contribution of this central design.
  • Fourth, I believe the evidence supporting the design components should be trustworthy, especially for a method like ReQuA, which includes diverse components. However, while the main results are averaged over three seeds, comparisons with Residual MLP, Tiny Transformer, and LoRA use only one seed, if I understand correctly. Since the paper emphasizes avoiding negative adaptation, I believe these comparisons should be repeated across multiple seeds.
Questions for authors

Responses to the critical weaknesses raised above would help clarify my assessment.

2 comments
AuthorsOpenReview ↗

**Response to Reviewer 4LmD**

We thank the reviewer for the constructive feedback. To assess the practical value of dataset-specific adaptation, we have added comparisons with supervised forecasters, training-budget measurements, and a matched auxiliary-aware baseline. Unless otherwise stated, results are averaged over three seeds (42, 1731, 2024) on the same eight datasets, target channels, and test windows.

Q1–Q2. Forecasting performance, training cost, and hyperparameter tuning

We compare ReQuA with supervised iTransformer (2 layers, width 512, FFN 2048) and TimeXer (1 layer), as well as the covariate-aware adapters ChronosX and UniCA. Using each baseline's own optimization settings and early-stopping protocol, we report trainable parameters, training/search budgets, and percentage error reductions relative to frozen Chronos-2 (mean ± standard deviation across seeds; higher is better).

MethodΔMAE (%)ΔMSE (%)ParamsLR trialsUpdates (k)GPU-h
iTransformer−1.10 ± 0.25+12.01 ± 0.256.54M169.7≈0.32
TimeXer−0.85 ± 0.24+11.18 ± 0.102.30M185.3≈3.59
ChronosX+7.50 ± 0.28+18.90 ± 0.472.17M3≤360≈27.9
UniCA+12.09 ± 0.27+19.22 ± 0.4115.9M1≤120≈10.4
ReQuA+8.69 ± 0.50+19.50 ± 0.651.70M1≤72≤4.8

ReQuA achieves the highest average MSE reduction among the compared methods (+19.50%), outperforming UniCA on six of eight datasets, while also delivering an +8.69% average MAE improvement. It does so with just 1.70M trainable parameters, compared with UniCA's 15.9M, and only 5% of training origins rather than the full training pools used by the other methods.

To make the training costs transparent, we report learning-rate trials, optimizer updates, and approximate GPU-hours. The supervised baselines follow validation-based early stopping without being capped at ReQuA's update budget, although the GPU-hour estimates have different accounting scopes: supervised figures exclude validation and testing, ChronosX and UniCA use maximum search budgets, and ReQuA includes calibration and evaluation. Under this protocol, ReQuA achieves competitive accuracy with 1.70M trainable parameters, 5% of training origins, and at most 72k optimizer updates.

Q3. Matched auxiliary-aware comparison

To isolate the contribution of query aggregation, we implemented Aux-Direct, which replaces ReQuA's 15 conditioned queries with a two-layer auxiliary-token encoder and direct cross-attention from the target's future states. Both methods use the same frozen backbone, historical auxiliary inputs, injection point, prediction and residual-grounding losses, 5% training-origin pool, and held-out scalar calibration; Aux-Direct is slightly larger (1.79M versus 1.70M trainable parameters). The table reports their three-seed mean improvements over frozen Chronos-2 (ΔMAE / ΔMSE; higher is better).

DatasetReQuA ΔMAE / ΔMSEAux-Direct ΔMAE / ΔMSE
ILI39.23 / 64.0024.08 / 44.56
ETTh10.85 / 6.631.24 / 6.23
ETTm11.40 / 9.881.94 / 7.26
ZafNoo4.37 / 15.054.42 / 10.51
AQShunyi4.09 / 12.633.99 / 9.55
Weather1.85 / 11.132.44 / 8.71
PEMS0416.95 / 31.296.51 / 11.49
Electricity0.78 / 5.421.05 / 3.52
Mean8.69 / 19.505.71 / 12.73

With auxiliary access, losses, and calibration held constant, ReQuA outperforms Aux-Direct in MSE on all eight datasets, achieving an average improvement of 19.50% versus 12.73% (a 6.77-percentage-point margin). ReQuA also leads in average MAE improvement (8.69% versus 5.71%), demonstrating the benefit of target-conditioned query aggregation beyond direct auxiliary cross-attention.

Q4. Robustness and fair calibration

Beyond the three-seed comparisons above, we applied held-out scalar calibration to residual MLP and LoRA and observed improvements in both, indicating that correction-strength calibration is broadly useful across adapters. We have therefore removed the original negative-adaptation count comparison and its uniqueness claim, while retaining calibration in ReQuA; the matched comparison in Q3 also uses identical held-out calibration for both methods.

Thanks for adding the supervised baselines. They make the accuracy comparison clearer. The reported GPU hours are hard to compare because they cover different parts of the pipeline, and ReQuA’s per-dataset training budget is much larger than I expected. Given that overhead, I see limited practical advantage over training a supervised model. I’m keeping my original score of 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.