acceptodds
Under review as a conference paper at ICLR 2027

Multimodal Foundation Priors for Diffusion-Calibrated Zero-Shot Time Series Reconstruction

Abstract

Zero-shot time series reconstruction asks a model trained where a target is observed to reconstruct it where it never was, from exogenous inputs alone. Prior work builds an informed prior from a single exogenous source and calibrates it with diffusion, so the prior is only as informative as that source. We ask whether a multimodal foundation model can supply a richer prior. We pretrain a transformer by masked reconstruction over heterogeneous inputs, daily gridded fields, station-scale series, and static attributes, tokenized at their native resolutions and fused by joint space-time attention, with a query decoder that returns a representation at any coordinate, day, variable, and depth, including locations that have no labels. Downstream, this representation enters a recurrent prior model through a small learned map, and a conditional diffusion calibrator refines the prior into a predictive distribution; prior, map, and calibrator are fine-tuned jointly. On four zero-shot reconstruction benchmarks (streamflow, stream temperature, soil moisture, and dissolved oxygen, under spatial cross-validation), the multimodal prior improves two tasks, with the largest gains where the single-source prior is weakest, and we characterize why the other two do not benefit: their target statistics at unseen locations are not identifiable from any input. Ablations show that the width of the representation map matters little, that fine-tuning it jointly with the diffusion calibrator matters a great deal, and that the point read-out of the calibrator must be chosen with care.

Primary area:
learning on time series and dynamical systems
Keywords:
Time series reconstruction, Zero-shot learning, Diffusion models, Multimodal foundation models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_H17JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies zero-shot reconstruction to reconstruct a target series at locations with no observations. To reduce the errors of direct fusion of multimodal representations using concatenation, this paper presents RepCal to take the fused model as a prior and calibrate it with a conditioned diffusion model. Experiments on four tasks, compared with nine models, show that direct fusion is inconsistent acorss models and tasks, while the calibrator improves effectively.

Critical strengths
  1. The calibrator is devised in a model-agnostic manner and is evaluated on top of nine models. The improvement over +rep is overall consistent.
  2. This paper is well organised and easy to follow, with several illustrative figures to facilitate comprehension.
Critical weaknesses
  1. Table 1 compares base / +rep / +RepCal, but does not include a calibrator without the representation (base + calibration). To support the argument in Sec. 4.3 that "only WrsWr_s lets the calibration tell them apart", it is suggested to add two more ablations: 1) +rep prior with a calibrator whose condition excludes WrsWr_s; 2) a base prior with a calibrator conditioned on WrsWr_s.
  2. The prior is a 1-layer, 20-unit LSTM. The second stage adds a 2-layer, 512-wide transformer plus a denoiser. Improvements may therefore come from extra capacity rather than from calibration per se.
Questions for authors
  1. Please consider reporting base + calibration in Table 1 and two additional ablations.
  2. Please design experiments to separate the performance improvement from calibration and justify the necessity of diffusion.
  3. Please explain how the bound in Appendix C addresses the transfer from observed stations O\mathcal{O} to unobserved stations U\mathcal{U}, which should be the core difficulty in zero-shot reconstruction.
6 comments
AuthorsOpenReview ↗

Response to Reviewer H17J: Summary

We sincerely appreciate the reviewer's constructive review, positive feedback on the clarity of our paper and on our calibration framework, which improves consistently over direct fusion across nine models, and insightful questions on the design and theory of RepCal. The main findings are summarized here, with full details in the three replies that follow.

R1. Repeated representation conditioning: WrsWr_s in the prior and in the calibrator. RepCal uses the pretrained multimodal representation WrsWr_s in the prior to reconstruct the 1D target series y^sp\hat{y}_s^p, and again in the calibrator to recover the local environmental context lost during this 1D projection. Stage-wise ablations confirm that calibration with the basic inputs xsx_s alone already improves the prior, while feeding WrsWr_s to the calibrator brings further gains on three of the four tasks. (Weakness 1; Question 1)

R2. Why calibration comes from diffusion rather than fϕf_\phi alone. Even with the same network fϕf_\phi, training it with direct L2L_2 loss to calibrate a prior overfits rare extreme peaks at sparse training locations and biases day-to-day predictions. RepCal resolves this via diffusion: multi-scale noise training regularizes fϕf_\phi so its initial prediction y^0\hat{y}_0 generalizes better to unseen locations (higher test R2R^2), while iterative reverse denoising refines y^0\hat{y}_0 toward typical daily observations to lower MAE. Going one step further, we removed the two-stage setup and used fϕf_\phi directly as a standalone prior. It underperforms the lightweight LSTM prior and still benefits from RepCal, so the gains do not come from network capacity. (Weakness 2; Question 2)

R3. Why calibration transfers reliably from observed O\mathcal{O} to unobserved U\mathcal{U} locations. A standard diffusion model generates from pure noise, so at an unseen location its score errors accumulate over a long chain. RepCal instead runs a short denoiser chain that starts from the regression prediction y^0=fϕ(cs)\hat{y}_0 = f_\phi(c_s). As a result, its distribution error bound on U\mathcal{U} (Appendix C) depends primarily on the regression error of y^0\hat{y}_0 and gives only a small weight to the denoiser's score error. Held-out results show that both components transfer to U\mathcal{U}: y^0\hat{y}_0 alone improves test R2R^2 over the +rep prior in 34 of 36 settings, and the reverse denoising steps reduce MAE by similar margins at training and unseen locations. (Question 3)

We thank the reviewer again for the constructive suggestions, which led us to add the stage-wise ablations on WrsWr_s and the comparison with direct regression (RepCal without diffusion) across all nine priors, and to clarify how the bound in Appendix C applies to unobserved locations. The paper is clearer and stronger as a result. If our replies and revisions address the reviewer's concerns, we would be grateful if the reviewer would consider raising the score. We are happy to answer any further questions.

AuthorsOpenReview ↗

R1. Repeated representation conditioning: in the prior and in the calibrator

When predicting the 1D target series y^sp\hat{y}_s^p, the prior compresses its high-dimensional inputs (xsx_s and WrsWr_s) into a scalar trajectory and loses local terrain, soil, and climate details (Section 4.3). However, in zero-shot reconstruction, how the prior errs at a given location depends directly on these local environmental conditions, so the calibrator still needs WrsWr_s to correct location-specific biases. Tables R1a and R1b add or remove WrsWr_s at both stages on the 1-layer LSTM prior, including the three settings you suggested (rows 3–5 of Table R1a), and the last row of Table R1b repeats the calibrator ablation across all nine priors of Table 1.

Table R1a. Median test MAE ↓ / R2R^2 ↑ across held-out locations when varying WrsWr_s in the prior and calibrator.

PriorCalibratorStreamflowStream temp.SWESnow depth
Basenone0.79 / 0.6051.88 / 0.92852.5 / 0.820153.0 / 0.868
+repnone0.80 / 0.6441.90 / 0.92950.3 / 0.839141.3 / 0.889
Basewithout WrsWr_s0.68 / 0.6281.89 / 0.94248.9 / 0.837136.9 / 0.889
+repwithout WrsWr_s0.66 / 0.6621.80 / 0.93447.8 / 0.848138.2 / 0.899
Basewith WrsWr_s0.68 / 0.6591.84 / 0.94347.0 / 0.852141.9 / 0.887
+repwith WrsWr_s (RepCal)0.71 / 0.6561.76 / 0.95146.1 / 0.863133.2 / 0.905

Table R1b. Percentage of held-out locations where RepCal achieves lower test MAE / higher R2R^2 than each setting without WrsWr_s in the prior or calibrator.

RepCal vs.StreamflowStream temp.SWESnow depth
Prior without WrsWr_s, calibrator without WrsWr_s53% / 69%52% / 59%57% / 65%57% / 68%
Prior with WrsWr_s, calibrator without WrsWr_s51% / 52%59% / 73%57% / 65%54% / 59%
Prior without WrsWr_s, calibrator with WrsWr_s58% / 58%52% / 51%49% / 59%61% / 70%
Calibrator without WrsWr_s across all nine priors of Table 151% / 58%52% / 55%50% / 53%50% / 52%

Calibration alone corrects shared systematic errors, while adding WrsWr_s corrects location-specific errors. Although calibrating with the basic inputs xsx_s alone already raises test R2R^2 on all four tasks by fixing shared prior biases (Table R1a), adding WrsWr_s provides the local terrain, soil, and climate context missing from xsx_s, further improving test R2R^2 at 59–69% of held-out locations (Table R1b).

Feeding WrsWr_s to the calibrator is especially effective when the prior underutilizes it. On stream temperature, the prior is dominated by seasonal air temperature in xsx_s and barely benefits from WrsWr_s (R2R^2: 0.928 to 0.929), whereas feeding WrsWr_s to the calibrator recovers the bypassed terrain and microclimate signals and lifts R2R^2 from 0.934 to 0.951. On streamflow, the prior already absorbs most of the WrsWr_s signal into y^sp\hat{y}_s^p (R2R^2: 0.605 to 0.644). The two calibrator variants then perform comparably on the +rep prior (RepCal has the lower MAE at 51% of held-out locations), and the calibrator extracts new information from WrsWr_s mainly when the prior is trained without it (R2R^2: 0.628 to 0.659).

The gain remains when the nine priors of Table 1 are pooled. As shown in the last row of Table R1b, feeding WrsWr_s to the calibrator improves test R2R^2 at 52–58% of held-out locations without increasing MAE (lower at 50–52% of locations).

AuthorsOpenReview ↗

R2. Why calibration comes from diffusion rather than alone

In RepCal, a regression network fϕf_\phi first predicts an initial center y^0=fϕ(cs)\hat{y}_0 = f_\phi(c_s), and the diffusion denoiser then refines y^0\hat{y}_0 over the reverse process. To separate the training and inference contributions of diffusion from the capacity of fϕf_\phi, we compare RepCal across all nine priors against two variants that use the exact same fϕf_\phi architecture without reverse denoising steps (Tables R2a–R2c):

  • RepCal w/o diffusion: fϕf_\phi trained standalone with direct L2L_2 loss, without the diffusion module.
  • RepCal w/o reverse steps: the initial center y^0\hat{y}_0 taken directly from the trained RepCal checkpoint with zero reverse steps.

Across the 36 prior–task settings, RepCal achieves a lower MAE than RepCal without diffusion in 29 settings and a higher R2R^2 in 26 (Table R2c).

Table R2a. Test MAE ↓ per prior (RepCal w/o diffusion / RepCal w/o reverse steps / RepCal).

PriorStreamflowStream temp.SWESnow depth
CNP0.685 / 0.687 / 0.6521.598 / 1.563 / 1.58858.3 / 54.8 / 52.0152.3 / 141.3 / 133.7
TNP0.721 / 0.723 / 0.6781.543 / 1.548 / 1.53954.6 / 53.1 / 52.5140.6 / 146.0 / 141.7
TE-TNP0.694 / 0.683 / 0.6471.486 / 1.520 / 1.50349.0 / 50.5 / 49.7137.7 / 136.7 / 132.9
ConvCNP0.721 / 0.769 / 0.7241.481 / 1.465 / 1.46657.7 / 52.5 / 48.6146.0 / 148.8 / 145.1
Gridded TNP0.681 / 0.638 / 0.6151.469 / 1.479 / 1.43452.6 / 52.1 / 49.9139.6 / 149.6 / 143.5
SADI0.658 / 0.653 / 0.6431.718 / 1.614 / 1.59257.1 / 57.9 / 57.6155.4 / 157.7 / 154.6
GiFlow0.807 / 0.794 / 0.7181.632 / 1.649 / 1.63164.0 / 62.3 / 59.7180.5 / 170.0 / 164.2
RP Flow0.738 / 0.732 / 0.6801.768 / 1.761 / 1.76959.4 / 57.3 / 57.0177.6 / 171.8 / 167.3
ZeroDiff0.749 / 0.766 / 0.7121.766 / 1.741 / 1.76151.0 / 47.4 / 46.1138.0 / 145.3 / 133.2

Table R2b. Median over the 9 priors (RepCal w/o diffusion / RepCal w/o reverse steps / RepCal).

MetricStreamflowStream temp.SWESnow depth
MAE ↓0.721 / 0.723 / 0.6781.598 / 1.563 / 1.58857.1 / 53.1 / 52.0146.0 / 148.8 / 143.5
RMSE ↓1.874 / 1.831 / 1.7652.016 / 2.008 / 1.98098.8 / 96.1 / 96.2243.2 / 240.9 / 243.5
R2R^2 ↑0.654 / 0.664 / 0.6600.951 / 0.951 / 0.9510.847 / 0.854 / 0.8540.888 / 0.894 / 0.893

Table R2c. Number of priors (out of 9) for which the first variant beats the second.

ComparisonStreamflowStream temp.SWESnow depthTotal
MAE: RepCal beats RepCal w/o diffusion8/97/97/97/929/36
MAE: RepCal beats RepCal w/o reverse steps9/95/99/99/932/36
RMSE: RepCal beats RepCal w/o diffusion6/96/96/95/923/36
RMSE: RepCal beats RepCal w/o reverse steps8/93/96/96/923/36
R2R^2: RepCal beats RepCal w/o diffusion7/96/96/97/926/36
R2R^2: RepCal w/o reverse steps beats RepCal w/o diffusion6/96/96/97/925/36
R2R^2: RepCal w/o reverse steps beats +rep prior8/99/98/99/934/36

Joint diffusion training regularizes fϕf_\phi to generalize better to unseen locations. Although both RepCal without diffusion and RepCal without reverse steps are direct single-step outputs of fϕf_\phi, the latter achieves a higher test R2R^2 in 25 of 36 settings (Table R2c). Training fϕf_\phi alone with L2L_2 loss heavily penalizes rare extreme peaks and overfits sparse training locations, whereas training fϕf_\phi jointly with the denoiser across multiple noise levels acts as multi-scale regularization, reducing sensitivity to isolated peaks and improving unseen-location R2R^2.

Iterative reverse denoising at inference corrects tail bias to lower day-to-day MAE. As a single-step regression output approximating the conditional mean E[ys∣cs]\mathbb{E}[y_s \mid c_s], the initial center y^0\hat{y}_0 in RepCal without reverse steps is pulled toward the tail by rare extreme peaks in skewed series like streamflow, inflating day-to-day baseline errors. Starting from y^0\hat{y}_0, the reverse steps follow the learned score of p(ys∣cs)p(y_s \mid c_s) back toward the high-density mode where typical daily observations concentrate. This lowers test MAE in 32 of 36 settings while keeping median test R2R^2 within 0.004 (Tables R2b and R2c), with the largest median MAE reduction on heavily skewed streamflow (5.8%) and the smallest on near-symmetric stream temperature (0.6%).

Reviewer_R9n9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies zero-shot reconstruction of environmental time series at sites without observations of the target variable. RepCal uses a pretrained, queryable multimodal encoder to extract frozen representations from weather, spatial, and static inputs. The frozen representations condition both the upstream time-series predictor and the downstream calibrator. The calibrator first produces a regression-center estimate and then applies informed-prior diffusion. The evaluation covers four environmental targets and multiple upstream model families under spatially held-out site splits, including frozen-prior and joint-training comparisons.

Critical strengths
  1. A relevant reconstruction task supported by broad comparisons. Reconstructing target series at sites without target measurements addresses a practically important problem distinct from future forecasting. The evaluation covers four environmental variables and multiple model classes using spatially grouped sites. In Table 1, +RepCal improves upon +rep in 32 of 36 tabulated MAE comparisons, with two ties and two degradations. The unfavorable cases are reported, and comparisons across additional prior families support the pipeline's broader applicability.

  2. A useful representation interface with relevant supporting analyses. The queryable frozen representation combines information across spatial supports, providing a flexible interface for multimodal inputs. The experiments include representation and calibration ablations, frozen-versus-joint prior comparisons, and a trained native decoder baseline. The supplied code also masks held-out sites from supervised losses and selects checkpoints using training loss. These controls address relevant concerns, although the attribution and preprocessing questions below remain unresolved.

Critical weaknesses
  1. The gains from diffusion and repeated representation conditioning are not isolated from the regression center's contribution. Before diffusion, the calibrator's function (f_\phi(c)) already produces a point estimate from the full conditioning input. Appendix Table 10 specifies a two-layer, width-512 Transformer for this center, compared with a one-layer, width-20 LSTM for the principal upstream prior. The paper does not report this regression center's performance on its own. It also lacks a comparison that fixes the +rep prior and removes only the calibrator's direct representation input. The existing component studies and native decoder comparison do not resolve these specific attribution questions. The code already exposes a y0_hat output, making the regression-center comparison particularly feasible.

  2. The preprocessing boundary for held-out target statistics requires verification. The paper uses spatially held-out sites, and the implementation excludes those sites from supervised losses. However, the supplied preprocessing and loader path appears to compute global target normalization from the full y_raw_trn array before fold-specific filtering. In this path, the array includes target labels from held-out sites. A source-level perturbation check confirms that changing held-out targets can alter normalized quantities on the training side. This concerns aggregate target statistics; it does not establish direct supervised training on held-out observations. Its effect on the reported MAE has not been measured. The authors should clarify the actual preprocessing protocol to establish whether the reported experiments satisfy the zero-target-observation setting.

Questions for authors
  1. Could the authors report the performance of the existing y0_hat regression-center output and the deterministic diffusion chain using the same checkpoints and held-out sites? Could they also fix the same +rep prior and remove only the calibrator's direct multimodal representation input? These controls would separate the contributions of additional regression capacity, the second representation pathway, and diffusion inference.

  2. Were target normalization statistics computed using only the permitted training sites within each outer split? If so, please identify the corresponding preprocessing step in the runs used for Table 1. Otherwise, please report the main paired comparisons after recomputing those statistics without held-out target labels. Please also clarify whether hyperparameter selection on folds 8–10 was independent of target labels in the final reported folds 1–7.

5 comments
AuthorsOpenReview ↗

Response to Reviewer R9n9: Summary

We sincerely appreciate the reviewer's thorough review, positive feedback on our problem formulation and two-stage calibration framework, and insightful questions after examining our code so carefully. The main findings are summarized here, with full details in the three replies that follow.

R1. Diffusion calibration vs. direct L2L_2 regression with fϕf_\phi (RepCal without diffusion). Training fϕf_\phi alone with L2L_2 loss is dominated by squared errors on rare extreme peaks in skewed series like streamflow, which causes fϕf_\phi to overfit sparse training locations and biases its day-to-day predictions. RepCal addresses this in both training and inference. During training, denoising across multiple noise levels regularizes fϕf_\phi so its initial prediction y^0\hat{y}_0 generalizes better to unseen locations (higher test R2R^2). At inference, iterative reverse denoising refines y^0\hat{y}_0 toward typical daily observations. (Weakness 1; Question 1)

R2. Reusing the multimodal representation WrsWr_s in conditional calibration. RepCal uses the pretrained multimodal representation WrsWr_s at both stages: first for the prior to reconstruct the 1D target series y^sp\hat{y}_s^p, and second for the calibrator to generalize error correction across locations. Time-series reconstruction forces the prior to compress high-dimensional inputs (xsx_s and WrsWr_s) into a 1D scalar trajectory y^sp\hat{y}_s^p. Because this 1D projection loses environmental and multimodal details, we also feed WrsWr_s directly to the calibrator so it can still see the local context. This helps on three of the four tasks. (Weakness 1; Question 1)

R3. Global scaling statistics and the held-out locations. In the submitted runs, only the two task-wide scaling constants (m,s)(m, s) were computed across all locations before fold splitting. We recomputed (m,s)(m, s) without the held-out locations of each fold and reran RepCal on all nine priors of Table 1, and our main conclusions remain unchanged. This is now the default in our released code, and we have updated all tables in the revised paper with the rerun results. (Weakness 2; Question 2)

We thank the reviewer again for examining both our paper and our code so carefully. This feedback made our evaluation setup and released code more rigorous. We would be glad to discuss any further questions, and we hope the reviewer will consider reassessing our work in light of these results and revisions.

AuthorsOpenReview ↗

R1. Diffusion calibration vs. direct regression with (RepCal without diffusion)

The diffusion calibrator of RepCal works in two stages: a regression network fϕf_\phi first predicts an initial center y^0=fϕ(cs)\hat{y}_0 = f_\phi(c_s), and the diffusion denoiser then refines y^0\hat{y}_0 over the reverse process (using the deterministic chain at inference). To test whether RepCal's gains come from this diffusion calibration or simply from the capacity of fϕf_\phi, we compare RepCal against two variants that use the same fϕf_\phi architecture but run no reverse denoising steps, across all nine priors (Tables R1a–R1c):

  • RepCal w/o diffusion: fϕf_\phi trained standalone with direct L2L_2 loss, without the diffusion module.
  • RepCal w/o reverse steps: the initial center y^0\hat{y}_0 taken directly from the trained RepCal checkpoint with zero reverse denoising steps.

Comparing the three variants separates the training and inference contributions of diffusion: across the 36 settings, RepCal outperforms RepCal without diffusion in 29/36 cases on MAE and 26/36 on R2R^2, and improves over RepCal without reverse steps in 32/36 cases on MAE (remaining within 3% in the other four; Table R1c).

Table R1a. Test MAE ↓ per prior (RepCal w/o diffusion / RepCal w/o reverse steps / RepCal).

PriorStreamflowStream temp.SWESnow depth
CNP0.685 / 0.687 / 0.6521.598 / 1.563 / 1.58858.3 / 54.8 / 52.0152.3 / 141.3 / 133.7
TNP0.721 / 0.723 / 0.6781.543 / 1.548 / 1.53954.6 / 53.1 / 52.5140.6 / 146.0 / 141.7
TE-TNP0.694 / 0.683 / 0.6471.486 / 1.520 / 1.50349.0 / 50.5 / 49.7137.7 / 136.7 / 132.9
ConvCNP0.721 / 0.769 / 0.7241.481 / 1.465 / 1.46657.7 / 52.5 / 48.6146.0 / 148.8 / 145.1
Gridded TNP0.681 / 0.638 / 0.6151.469 / 1.479 / 1.43452.6 / 52.1 / 49.9139.6 / 149.6 / 143.5
SADI0.658 / 0.653 / 0.6431.718 / 1.614 / 1.59257.1 / 57.9 / 57.6155.4 / 157.7 / 154.6
GiFlow0.807 / 0.794 / 0.7181.632 / 1.649 / 1.63164.0 / 62.3 / 59.7180.5 / 170.0 / 164.2
RP Flow0.738 / 0.732 / 0.6801.768 / 1.761 / 1.76959.4 / 57.3 / 57.0177.6 / 171.8 / 167.3
ZeroDiff0.749 / 0.766 / 0.7121.766 / 1.741 / 1.76151.0 / 47.4 / 46.1138.0 / 145.3 / 133.2

Table R1b. Median over the 9 priors (RepCal w/o diffusion / RepCal w/o reverse steps / RepCal).

MetricStreamflowStream temp.SWESnow depth
MAE ↓0.721 / 0.723 / 0.6781.598 / 1.563 / 1.58857.1 / 53.1 / 52.0146.0 / 148.8 / 143.5
RMSE ↓1.874 / 1.831 / 1.7652.016 / 2.008 / 1.98098.8 / 96.1 / 96.2243.2 / 240.9 / 243.5
R² ↑0.654 / 0.664 / 0.6600.951 / 0.951 / 0.9510.847 / 0.854 / 0.8540.888 / 0.894 / 0.893

Table R1c. Number of priors (out of 9) for which the first variant beats the second.

ComparisonStreamflowStream temp.SWESnow depthTotal
MAE: RepCal beats RepCal w/o diffusion8/97/97/97/929/36
MAE: RepCal beats RepCal w/o reverse steps9/95/99/99/932/36
RMSE: RepCal beats RepCal w/o diffusion6/96/96/95/923/36
RMSE: RepCal beats RepCal w/o reverse steps8/93/96/96/923/36
R²: RepCal beats RepCal w/o diffusion7/96/96/97/926/36
R²: RepCal w/o reverse steps beats RepCal w/o diffusion6/96/96/97/925/36

Joint diffusion training regularizes fϕf_\phi to generalize better to unseen locations. Although RepCal without diffusion and RepCal without reverse steps are direct single-step outputs of the same 2-layer, 512-wide Transformer fϕf_\phi, the latter achieves a higher test R2R^2 in 25 of 36 settings (Table R1c). Training fϕf_\phi alone with L2L_2 loss heavily penalizes rare extreme peaks, causing it to overfit extreme events at sparse training locations. In RepCal, training fϕf_\phi jointly with the denoiser across multiple noise levels acts as multi-scale noise regularization, reducing sensitivity to isolated peaks and improving test R2R^2 at unseen locations.

Iterative reverse denoising at inference corrects tail bias to lower day-to-day MAE. Because the initial center y^0\hat{y}_0 in RepCal without reverse steps is a single-step regression output approximating the conditional mean E[ys∣cs]\mathbb{E}[y_s \mid c_s], rare extreme peaks in heavy-tailed series such as streamflow pull y^0\hat{y}_0 toward the tail and inflate day-to-day baseline predictions. Starting from y^0\hat{y}_0, the reverse steps in RepCal iteratively follow the learned score field of p(ys∣cs)p(y_s \mid c_s) back toward the high-density mode where daily observations concentrate. As shown in Tables R1b and R1c, this lowers test MAE in 32 of 36 settings while keeping median test R2R^2 within 0.004, with the largest median MAE reduction on heavily skewed streamflow at 5.8% and the smallest on near-symmetric stream temperature at 0.6%.

AuthorsOpenReview ↗

R2. Reusing the multimodal representation in conditional calibration

In RepCal, the prior uses WrsWr_s to predict the target series y^sp\hat{y}_s^p, whereas the calibrator conditions on WrsWr_s to learn how the prior's residual errors depend on local multimodal context (Section 4.3). Without WrsWr_s in the calibrator, this context must come from y^sp\hat{y}_s^p or xsx_s. However, time-series reconstruction forces the prior to compress high-dimensional inputs into a 1D scalar trajectory y^sp\hat{y}_s^p, creating an information bottleneck that records the predicted value but loses much of the underlying terrain and climate regime. Meanwhile, the basic basin inputs xsx_s provide only coarse local summaries, lacking the gridded weather, terrain, and soil maps encoded in WrsWr_s (Section 5.1). Conditioning the calibrator directly on WrsWr_s bypasses this 1D bottleneck and restores the spatial context for correcting location-specific errors. Following your suggestion, we tested this by keeping the frozen +rep LSTM prior and removing only WrsWr_s from the calibrator's condition (Table R2).

Table R2. Removing WrsWr_s from the calibrator's condition cs=(xs,Wrs,m~s,y^sp)c_s = (x_s, Wr_s, \tilde{m}_s, \hat{y}_s^p) on the frozen +rep LSTM prior (medians across held-out locations).

Metric / VariantStreamflowStream temp.SWESnow depth
MAE ↓, w/o WrsWr_s0.661.8047.8138.2
MAE ↓, RepCal0.711.7646.1133.2
R2R^2 ↑, w/o WrsWr_s0.6620.9340.8480.899
R2R^2 ↑, RepCal0.6560.9510.8630.905
Locations where RepCal has lower MAE51%59%57%54%

In Table R2, keeping WrsWr_s in the calibrator lowers median MAE and raises R2R^2 on stream temperature, SWE, and snow depth, improving MAE at 54–59% of held-out locations. More importantly, comparing Table R2 with the prior ablations in Table 8 of the revised paper shows how the calibrator and the prior complement each other in using WrsWr_s.

When the prior underutilizes WrsWr_s, the calibrator recovers the missing signal. On stream temperature, adding WrsWr_s to the LSTM prior barely changes its R2R^2 from 0.928 to 0.929, because the prior's point prediction is dominated by the strong seasonal air-temperature cycle in xsx_s. Feeding WrsWr_s directly to the calibrator allows it to exploit the fine-grained terrain and shading context in WrsWr_s for residual correction, raising test R2R^2 from 0.934 to 0.951 and lowering MAE at 59% of locations.

When the prior already extracts strong signal from WrsWr_s, both calibrator variants perform comparably. On streamflow, the LSTM prior already gains substantially from WrsWr_s, with R2R^2 jumping from 0.605 to 0.644. Since y^sp\hat{y}_s^p already carries most of the context in WrsWr_s, feeding WrsWr_s to the calibrator again adds little: RepCal has the lower MAE at 51% of held-out sites, and w/o WrsWr_s has the lower median MAE.

Reviewer_zTauOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces RepCal, a two-stage framework for zero-shot time-series reconstruction at unmeasured locations. a two-stage framework for zero-shot time-series reconstruction that takes a time-series model fused with a frozen multimodal representation as a prior and calibrates its output with a diffusion model conditioned on that same representation. The task is to estimate the full daily series of an environmental variable, such as streamflow, stream temperature, or snow, at locations where it has never been measured. Experiments on four environmental tasks with nine published time-series models show consistent improvements over direct fusion. The authors also ablate the representation and calibrator across several priors, and provide a theoretical bound on the calibrated distribution.

Critical strengths

The paper studies a clear problem: how to find the right evidence for a given query. The authors check the LLM judge against other LLM backbones, which is a reasonable sanity check. It would help to also see whether the judge agrees with human annotators.

Critical weaknesses

The benchmark covers a fairly narrow set of tasks. This area has many sensor setups and task types, and it would help to see which tasks are easy and which are hard. The ReAct setup is not described clearly. If ReAct can run code in a sandbox, I would expect D1 to be high. Could the authors explain the setup, and say more about why the agent still does badly on D1? PHM is tested on its own, with no comparison to other memory-retrieval methods, not even a naive RAG baseline. The writing needs some more work. Several results are stated without saying why. For example, Section 5.2 does not explain the reasoning behind Finding 1.

Questions for authors

Why are the PHM results not in Table 3? In Figure 3, how are fine-grained, multiscale, and coarse-grained defined?

6 comments
AuthorsOpenReview ↗

Response to Reviewer zTau: Summary

We thank the reviewer for the positive feedback and the concrete suggestions. We ran new experiments for each point. The main results are summarized here, with details in the four replies that follow.

  • R1. Retrieval baseline: Reusing the series of the 10 most similar training locations works well on stream temperature and snow (R2R^2 0.82–0.93) but poorly on streamflow (R2R^2 0.29). RepCal outperforms it on all four tasks. (Weaknesses 1 and 3)
  • R2. Distance to training locations: Streamflow accuracy drops for locations farther from the training locations in the representation space. RepCal still improves the prior at the farthest locations. (Weaknesses 1 and 3)
  • R3. Why direct fusion is inconsistent: With the representation, the LSTM and EA-LSTM priors fit the training locations better, but much of this gain does not carry over to held-out locations. (Weakness 4)
  • R4. Pretrained head and terminology: We report the head's MAE and R2R^2 in Table R1, discuss why it underperforms on snow, and define fine-grained, coarse-grained, and multiscale. (Weakness 2; Questions 1 and 2)

The reviewer's feedback helped us clarify our experimental findings and substantially strengthened the paper. We hope the new experiments below address the reviewer's questions, and we would be glad to discuss further if any questions remain.

AuthorsOpenReview ↗

R1. Retrieval baseline

At an unobserved location, the only target observations come from other sensors, so the most direct baseline is to borrow them. For each held-out location in U\mathcal{U}, we take the 10 nearest training locations in O\mathcal{O} (Euclidean distance in the time-averaged, standardized representation space) and average their standardized series day by day. Because the mean and scale at an unobserved location are unknown, we rescale this series with (μ^s,σ^s)(\hat{\mu}_s, \hat{\sigma}_s) predicted by the same conditional VAE that RepCal uses.

Table R1. Median test MAE ↓ / R2R^2 ↑ over held-out locations. We use 10-fold cross-validation over locations; folds 8–10 were used for hyperparameter tuning, so all results are reported on folds 1–7. The last row gives the share of held-out locations where RepCal has lower MAE / higher R2R^2 than retrieval.

MethodStreamflowStream temp.SWESnow depth
Retrieval (10 nearest)1.00 / 0.2901.88 / 0.92850.5 / 0.818153.3 / 0.864
Pretrained head0.80 / 0.6331.64 / 0.94168.6 / 0.835178.7 / 0.878
+rep LSTM prior0.80 / 0.6441.90 / 0.92950.3 / 0.839141.3 / 0.889
RepCal0.71 / 0.6561.76 / 0.95146.1 / 0.863133.2 / 0.905
RepCal better than retrieval92% / 96%54% / 62%68% / 74%70% / 73%

Retrieval already does well on stream temperature, SWE, and snow depth (R2R^2 0.82–0.93, close to the LSTM prior). These variables follow seasonal cycles (air temperature, snow accumulation and melt) that similar locations share. Streamflow is different: it responds to local storms through basin-specific rainfall–runoff processes, and retrieval reaches only R2=0.29R^2 = 0.29.

RepCal outperforms retrieval on all four tasks, by a large margin on streamflow (lower MAE at 92% of locations) and by a smaller one on the other three (54–70%). Retrieval uses the target location's representation only to choose which stations to copy, so its daily series comes entirely from other sensors. RepCal instead predicts the series from the inputs at the target location (daily weather, terrain, and soil).

AuthorsOpenReview ↗

R2. Distance to training locations

The frozen representation lets us measure, without target labels, how far a held-out location is from O\mathcal{O}: its mean distance to the 10 nearest training locations. We split the held-out locations of each task into three equal groups (Near, Middle, Far) by this distance. Retrieval is the baseline from Reply 1; the +rep prior and +RepCal columns pool the nine priors from Table 1.

Table R2. Median test R2R^2 ↑ by distance group. The last column gives the share of prior–location pairs where +RepCal has lower MAE / higher R2R^2 than the +rep prior.

TaskDistanceRetrieval+rep prior+RepCal+RepCal vs. +rep prior (MAE / R2R^2)
StreamflowNear0.5290.7280.75556% / 72%
StreamflowMiddle0.2500.6560.69261% / 72%
StreamflowFar0.0160.4900.49970% / 61%
Stream temp.Near0.9250.9270.93453% / 60%
Stream temp.Middle0.9420.9570.96258% / 67%
Stream temp.Far0.9180.9360.94561% / 70%
SWENear0.8470.8580.86760% / 60%
SWEMiddle0.7980.8400.85359% / 64%
SWEFar0.7930.8080.83161% / 67%
Snow depthNear0.8810.8850.89759% / 60%
Snow depthMiddle0.8710.8840.89565% / 66%
Snow depthFar0.8330.8680.88962% / 66%

Distance matters most for streamflow: from Near to Far, retrieval falls from R2R^2 0.53 to 0.02 and the priors from 0.73 to 0.49. SWE and snow depth decline less, and stream temperature shows no clear trend.

RepCal raises the median R2R^2 in all 12 groups and achieves lower MAE / higher R2R^2 on 53–70% / 60–72% of prior–location pairs. At Far streamflow locations the R2R^2 gain is small, from 0.490 to 0.499, but RepCal lowers MAE for 70% of prior–location pairs there, the highest share in Table R2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.