- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies zero-shot reconstruction to reconstruct a target series at locations with no observations. To reduce the errors of direct fusion of multimodal representations using concatenation, this paper presents RepCal to take the fused model as a prior and calibrate it with a conditioned diffusion model. Experiments on four tasks, compared with nine models, show that direct fusion is inconsistent acorss models and tasks, while the calibrator improves effectively.
Critical strengths
- The calibrator is devised in a model-agnostic manner and is evaluated on top of nine models. The improvement over +rep is overall consistent.
- This paper is well organised and easy to follow, with several illustrative figures to facilitate comprehension.
Critical weaknesses
- Table 1 compares base / +rep / +RepCal, but does not include a calibrator without the representation (base + calibration). To support the argument in Sec. 4.3 that "only lets the calibration tell them apart", it is suggested to add two more ablations: 1) +rep prior with a calibrator whose condition excludes ; 2) a base prior with a calibrator conditioned on .
- The prior is a 1-layer, 20-unit LSTM. The second stage adds a 2-layer, 512-wide transformer plus a denoiser. Improvements may therefore come from extra capacity rather than from calibration per se.
Questions for authors
- Please consider reporting base + calibration in Table 1 and two additional ablations.
- Please design experiments to separate the performance improvement from calibration and justify the necessity of diffusion.
- Please explain how the bound in Appendix C addresses the transfer from observed stations to unobserved stations , which should be the core difficulty in zero-shot reconstruction.
6 comments
R1. Repeated representation conditioning: in the prior and in the calibrator
When predicting the 1D target series , the prior compresses its high-dimensional inputs ( and ) into a scalar trajectory and loses local terrain, soil, and climate details (Section 4.3). However, in zero-shot reconstruction, how the prior errs at a given location depends directly on these local environmental conditions, so the calibrator still needs to correct location-specific biases. Tables R1a and R1b add or remove at both stages on the 1-layer LSTM prior, including the three settings you suggested (rows 3–5 of Table R1a), and the last row of Table R1b repeats the calibrator ablation across all nine priors of Table 1.
Table R1a. Median test MAE ↓ / ↑ across held-out locations when varying in the prior and calibrator.
| Prior | Calibrator | Streamflow | Stream temp. | SWE | Snow depth |
|---|---|---|---|---|---|
| Base | none | 0.79 / 0.605 | 1.88 / 0.928 | 52.5 / 0.820 | 153.0 / 0.868 |
| +rep | none | 0.80 / 0.644 | 1.90 / 0.929 | 50.3 / 0.839 | 141.3 / 0.889 |
| Base | without | 0.68 / 0.628 | 1.89 / 0.942 | 48.9 / 0.837 | 136.9 / 0.889 |
| +rep | without | 0.66 / 0.662 | 1.80 / 0.934 | 47.8 / 0.848 | 138.2 / 0.899 |
| Base | with | 0.68 / 0.659 | 1.84 / 0.943 | 47.0 / 0.852 | 141.9 / 0.887 |
| +rep | with (RepCal) | 0.71 / 0.656 | 1.76 / 0.951 | 46.1 / 0.863 | 133.2 / 0.905 |
Table R1b. Percentage of held-out locations where RepCal achieves lower test MAE / higher than each setting without in the prior or calibrator.
| RepCal vs. | Streamflow | Stream temp. | SWE | Snow depth |
|---|---|---|---|---|
| Prior without , calibrator without | 53% / 69% | 52% / 59% | 57% / 65% | 57% / 68% |
| Prior with , calibrator without | 51% / 52% | 59% / 73% | 57% / 65% | 54% / 59% |
| Prior without , calibrator with | 58% / 58% | 52% / 51% | 49% / 59% | 61% / 70% |
| Calibrator without across all nine priors of Table 1 | 51% / 58% | 52% / 55% | 50% / 53% | 50% / 52% |
Calibration alone corrects shared systematic errors, while adding corrects location-specific errors. Although calibrating with the basic inputs alone already raises test on all four tasks by fixing shared prior biases (Table R1a), adding provides the local terrain, soil, and climate context missing from , further improving test at 59–69% of held-out locations (Table R1b).
Feeding to the calibrator is especially effective when the prior underutilizes it. On stream temperature, the prior is dominated by seasonal air temperature in and barely benefits from (: 0.928 to 0.929), whereas feeding to the calibrator recovers the bypassed terrain and microclimate signals and lifts from 0.934 to 0.951. On streamflow, the prior already absorbs most of the signal into (: 0.605 to 0.644). The two calibrator variants then perform comparably on the +rep prior (RepCal has the lower MAE at 51% of held-out locations), and the calibrator extracts new information from mainly when the prior is trained without it (: 0.628 to 0.659).
The gain remains when the nine priors of Table 1 are pooled. As shown in the last row of Table R1b, feeding to the calibrator improves test at 52–58% of held-out locations without increasing MAE (lower at 50–52% of locations).
R2. Why calibration comes from diffusion rather than alone
In RepCal, a regression network first predicts an initial center , and the diffusion denoiser then refines over the reverse process. To separate the training and inference contributions of diffusion from the capacity of , we compare RepCal across all nine priors against two variants that use the exact same architecture without reverse denoising steps (Tables R2a–R2c):
- RepCal w/o diffusion: trained standalone with direct loss, without the diffusion module.
- RepCal w/o reverse steps: the initial center taken directly from the trained RepCal checkpoint with zero reverse steps.
Across the 36 prior–task settings, RepCal achieves a lower MAE than RepCal without diffusion in 29 settings and a higher in 26 (Table R2c).
Table R2a. Test MAE ↓ per prior (RepCal w/o diffusion / RepCal w/o reverse steps / RepCal).
| Prior | Streamflow | Stream temp. | SWE | Snow depth |
|---|---|---|---|---|
| CNP | 0.685 / 0.687 / 0.652 | 1.598 / 1.563 / 1.588 | 58.3 / 54.8 / 52.0 | 152.3 / 141.3 / 133.7 |
| TNP | 0.721 / 0.723 / 0.678 | 1.543 / 1.548 / 1.539 | 54.6 / 53.1 / 52.5 | 140.6 / 146.0 / 141.7 |
| TE-TNP | 0.694 / 0.683 / 0.647 | 1.486 / 1.520 / 1.503 | 49.0 / 50.5 / 49.7 | 137.7 / 136.7 / 132.9 |
| ConvCNP | 0.721 / 0.769 / 0.724 | 1.481 / 1.465 / 1.466 | 57.7 / 52.5 / 48.6 | 146.0 / 148.8 / 145.1 |
| Gridded TNP | 0.681 / 0.638 / 0.615 | 1.469 / 1.479 / 1.434 | 52.6 / 52.1 / 49.9 | 139.6 / 149.6 / 143.5 |
| SADI | 0.658 / 0.653 / 0.643 | 1.718 / 1.614 / 1.592 | 57.1 / 57.9 / 57.6 | 155.4 / 157.7 / 154.6 |
| GiFlow | 0.807 / 0.794 / 0.718 | 1.632 / 1.649 / 1.631 | 64.0 / 62.3 / 59.7 | 180.5 / 170.0 / 164.2 |
| RP Flow | 0.738 / 0.732 / 0.680 | 1.768 / 1.761 / 1.769 | 59.4 / 57.3 / 57.0 | 177.6 / 171.8 / 167.3 |
| ZeroDiff | 0.749 / 0.766 / 0.712 | 1.766 / 1.741 / 1.761 | 51.0 / 47.4 / 46.1 | 138.0 / 145.3 / 133.2 |
Table R2b. Median over the 9 priors (RepCal w/o diffusion / RepCal w/o reverse steps / RepCal).
| Metric | Streamflow | Stream temp. | SWE | Snow depth |
|---|---|---|---|---|
| MAE ↓ | 0.721 / 0.723 / 0.678 | 1.598 / 1.563 / 1.588 | 57.1 / 53.1 / 52.0 | 146.0 / 148.8 / 143.5 |
| RMSE ↓ | 1.874 / 1.831 / 1.765 | 2.016 / 2.008 / 1.980 | 98.8 / 96.1 / 96.2 | 243.2 / 240.9 / 243.5 |
| ↑ | 0.654 / 0.664 / 0.660 | 0.951 / 0.951 / 0.951 | 0.847 / 0.854 / 0.854 | 0.888 / 0.894 / 0.893 |
Table R2c. Number of priors (out of 9) for which the first variant beats the second.
| Comparison | Streamflow | Stream temp. | SWE | Snow depth | Total |
|---|---|---|---|---|---|
| MAE: RepCal beats RepCal w/o diffusion | 8/9 | 7/9 | 7/9 | 7/9 | 29/36 |
| MAE: RepCal beats RepCal w/o reverse steps | 9/9 | 5/9 | 9/9 | 9/9 | 32/36 |
| RMSE: RepCal beats RepCal w/o diffusion | 6/9 | 6/9 | 6/9 | 5/9 | 23/36 |
| RMSE: RepCal beats RepCal w/o reverse steps | 8/9 | 3/9 | 6/9 | 6/9 | 23/36 |
| : RepCal beats RepCal w/o diffusion | 7/9 | 6/9 | 6/9 | 7/9 | 26/36 |
| : RepCal w/o reverse steps beats RepCal w/o diffusion | 6/9 | 6/9 | 6/9 | 7/9 | 25/36 |
| : RepCal w/o reverse steps beats +rep prior | 8/9 | 9/9 | 8/9 | 9/9 | 34/36 |
Joint diffusion training regularizes to generalize better to unseen locations. Although both RepCal without diffusion and RepCal without reverse steps are direct single-step outputs of , the latter achieves a higher test in 25 of 36 settings (Table R2c). Training alone with loss heavily penalizes rare extreme peaks and overfits sparse training locations, whereas training jointly with the denoiser across multiple noise levels acts as multi-scale regularization, reducing sensitivity to isolated peaks and improving unseen-location .
Iterative reverse denoising at inference corrects tail bias to lower day-to-day MAE. As a single-step regression output approximating the conditional mean , the initial center in RepCal without reverse steps is pulled toward the tail by rare extreme peaks in skewed series like streamflow, inflating day-to-day baseline errors. Starting from , the reverse steps follow the learned score of back toward the high-density mode where typical daily observations concentrate. This lowers test MAE in 32 of 36 settings while keeping median test within 0.004 (Tables R2b and R2c), with the largest median MAE reduction on heavily skewed streamflow (5.8%) and the smallest on near-symmetric stream temperature (0.6%).
Response to Reviewer H17J: Summary
We sincerely appreciate the reviewer's constructive review, positive feedback on the clarity of our paper and on our calibration framework, which improves consistently over direct fusion across nine models, and insightful questions on the design and theory of RepCal. The main findings are summarized here, with full details in the three replies that follow.
R1. Repeated representation conditioning: Wrs in the prior and in the calibrator. RepCal uses the pretrained multimodal representation Wrs in the prior to reconstruct the 1D target series y^sp, and again in the calibrator to recover the local environmental context lost during this 1D projection. Stage-wise ablations confirm that calibration with the basic inputs xs alone already improves the prior, while feeding Wrs to the calibrator brings further gains on three of the four tasks. (Weakness 1; Question 1)
R2. Why calibration comes from diffusion rather than fϕ alone. Even with the same network fϕ, training it with direct L2 loss to calibrate a prior overfits rare extreme peaks at sparse training locations and biases day-to-day predictions. RepCal resolves this via diffusion: multi-scale noise training regularizes fϕ so its initial prediction y^0 generalizes better to unseen locations (higher test R2), while iterative reverse denoising refines y^0 toward typical daily observations to lower MAE. Going one step further, we removed the two-stage setup and used fϕ directly as a standalone prior. It underperforms the lightweight LSTM prior and still benefits from RepCal, so the gains do not come from network capacity. (Weakness 2; Question 2)
R3. Why calibration transfers reliably from observed O to unobserved U locations. A standard diffusion model generates from pure noise, so at an unseen location its score errors accumulate over a long chain. RepCal instead runs a short denoiser chain that starts from the regression prediction y^0=fϕ(cs). As a result, its distribution error bound on U (Appendix C) depends primarily on the regression error of y^0 and gives only a small weight to the denoiser's score error. Held-out results show that both components transfer to U: y^0 alone improves test R2 over the +rep prior in 34 of 36 settings, and the reverse denoising steps reduce MAE by similar margins at training and unseen locations. (Question 3)
We thank the reviewer again for the constructive suggestions, which led us to add the stage-wise ablations on Wrs and the comparison with direct regression (RepCal without diffusion) across all nine priors, and to clarify how the bound in Appendix C applies to unobserved locations. The paper is clearer and stronger as a result. If our replies and revisions address the reviewer's concerns, we would be grateful if the reviewer would consider raising the score. We are happy to answer any further questions.