- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
FAIP-Diff proposes a probabilistic forecasting framework to address temporal pattern distortion during forward and reverse diffusion. The authors introduce three mechanisms: adaptive noise remodulation, which increases forward perturbation according to volatility-normalized forecasting residuals; constrained-space guided sampling, which modifies predicted denoising noise using a PCA subspace constructed from training future windows; and a volatility-aware weighted objective, which emphasizes high-residual positions during volatility estimation.
Critical strengths
1. Empirical gains are good compared to diffusion baselines (Table 1).
2. Useful component-wise and targeted empirical analyses (Table 2).
Critical weaknesses
1. The adaptive forward diffusion process is not sufficiently justified, raising concerns about the soundness of the proposed formulation.
Firstly, the explanation in lines 257–269 (Section 4.2) is unclear. Equation (8) introduces the base noise variance, , without properly motivating its formulation. Although the authors briefly mention as the true local volatility, they do not clearly explain how it is calculated or why and are combined using these specific coefficients. While Appendix A.1 provides an algebraic expansion of the coefficients, it does not sufficiently explain the reasoning behind this particular variance formulation.
Furthermore, Equation (11) modifies the forward diffusion process by adding noise based on the ground-truth prediction error. However, Section 4.3 describes the reverse denoising process without clearly explaining how it accounts for this modified noise. It is unclear whether the reverse process is properly aligned with the adaptive forward diffusion process introduced in Section 4.2. Since the forward and reverse processes are closely related in diffusion models, the authors should provide a clearer explanation of how these two processes work together and justify the proposed formulation. It is very difficult to follow the paper in current form
2. The PCA-based guidance in Section 4.3 is not clearly justified.
In Section 4.3, the authors use PCA to obtain the main patterns from the ground-truth future sequences in the training data. These patterns are then used to correct the predicted noise during reverse diffusion. However, it is unclear why the PCA components obtained from the future sequences are suitable for correcting the predicted noise. The authors do not clearly explain the relationship between these two. Also, PCA mainly captures the dominant patterns in the training data, so it may miss some rare but important fluctuations. Since the main goal of the paper is to preserve temporal patterns, this raises a concern about whether the proposed guidance can preserve such fluctuations. Although the results show improvements, the paper does not clearly demonstrate how the PCA-based correction helps reduce denoising trajectory drift.
Questions for authors
Q1. In Equation (8), could the authors provide the exact calculation of and further justify why the predicted and target variances are combined using these particular coefficients?
Q2. How is the reverse denoising process mathematically consistent with the adaptive forward diffusion process in Equation (11)?
Q3. Although the results show improvements, could the authors further explain why increasing noise in regions with larger prediction errors helps preserve temporal information instead of suppressing it further?
Q4. In Section 4.3, what justifies using PCA components obtained from clean future sequences to correct predicted noise, given that these represent different quantities?
Q5. Could the authors provide a direct quantitative analysis showing that the proposed PCA-based guidance reduces denoising trajectory drift, beyond improvements in general forecasting metrics?
Q6. Could the authors clarify whether all baseline methods were evaluated using the same preprocessing, number of generated samples, and evaluation settings, and explain how fair comparisons were ensured?
8 comments
Response to Weakness 1 and Questions 1–2(Ⅱ)
Weakness 1b and Q2
Training-time corruption and objective. During training, the observed future is available. Its residual relative to the conditional mean determines the score and the elementwise modulation factor . The forward noise variance is therefore .
The denoiser receives states constructed using this adaptive variance and is trained to predict the injected standard Gaussian variable , together with the reverse-transition variance. Its noise-prediction target is , rather than the scaled perturbation . Thus, residual-aware modulation affects both the training states and the learned denoising function.
Inference-time calculation. At inference, is unknown. Therefore, the true residual-dependent factor cannot be evaluated. The sampler retains the diffusion schedule, conditional priors, learned noise prediction , and predicted reverse-transition variance. Using the inferred base-process noise variance , it calculates the clean-future estimate as .
We therefore do not claim that this update is the exact analytical reverse of the residual-modulated forward process: the unavailable factor is not reinstated in the clean-state reconstruction. Nor do we assume that its effect is necessarily negligible. Eqs. (14)–(15) describe an inference-time learned conditional sampling calculation, whose practical sensitivity to the choice of factor must be assessed empirically.
Scope of related precedents. Previous diffusion studies illustrate that sampling design can be considered separately from the formulas used to construct training states. DDIM derives non-Markovian processes that share a noise-prediction training objective with DDPM [1]. DPM-Solver samples from pretrained diffusion models using a dedicated diffusion-ODE solver [2]. EDM studies sampling choices separately, including their application to pretrained score networks [3]. Each of these methods has its own sampling rationale. We cite them as precedents for separating training and sampling design, rather than as proof that omitting our residual-dependent factor yields an exact reverse process. We evaluate this specific design choice through the following paired ablation.
Paired sampling-factor ablation
We compared the current sampler () with a training-set mean and a history-predicted , changing only the factor used in clean-state reconstruction. The checkpoint, test windows, random numbers, and CSG setting were fixed. The implementation reproduced the original sampler with a maximum output difference below . For each dataset, we used three training seeds and generated 100 trajectories per test window. The table reports the three-seed mean percentage change relative to the current sampler; negative values indicate lower and therefore better CRPS or MAE.
| Dataset | Test windows per seed | Training-mean : CRPS | Training-mean : MAE | History-predicted : CRPS | History-predicted : MAE |
|---|---|---|---|---|---|
| ETTh1 | 11 | +0.033% | +0.056% | +0.033% | +0.054% |
| ETTh2 | 11 | +0.078% | +0.099% | +0.080% | +0.101% |
| ETTm1 | 44 | +0.326% | +0.157% | +0.262% | +0.123% |
| ETTm2 | 44 | +0.267% | +0.358% | +0.282% | +0.377% |
| ExchangeRate | 6 | −0.013% | −0.017% | −0.015% | −0.020% |
| ILI | 1 | −0.085% | −0.082% | −0.087% | −0.085% |
Neither implementable factor approximation provides a consistent improvement, and every reported mean change has a magnitude below . Under the tested checkpoints, windows, and metrics, omitting an explicit residual factor from the sampler has a limited observed effect on final forecasting performance relative to the two feasible proxy-factor samplers. These results support the narrower practical claim that the present non-exact but learned conditional sampling procedure is feasible in the evaluated settings.
References
[1] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. In International Conference on Learning Representations, pp. 1–20, 2021.
[2] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. In Advances in Neural Information Processing Systems, volume 35, pp. 5775–5787, 2022.
[3] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the Design Space of Diffusion-Based Generative Models. In Advances in Neural Information Processing Systems, volume 35, pp. 26565–26577, 2022.
Response to Weakness 2 and Questions 4–5
Weakness 2. In Section 4.3, the authors use PCA to obtain the main patterns from the ground-truth future sequences in the training data. These patterns are then used to correct the predicted noise during reverse diffusion. However, it is unclear why the PCA components obtained from the future sequences are suitable for correcting the predicted noise. The authors do not clearly explain the relationship between these two. Also, PCA mainly captures the dominant patterns in the training data, so it may miss some rare but important fluctuations. Since the main goal of the paper is to preserve temporal patterns, this raises a concern about whether the proposed guidance can preserve such fluctuations. Although the results show improvements, the paper does not clearly demonstrate how the PCA-based correction helps reduce denoising trajectory drift.
Q4. In Section 4.3, what justifies using PCA components obtained from clean future sequences to correct predicted noise, given that these represent different quantities?
Q5. Could the authors provide a direct quantitative analysis showing that the proposed PCA-based guidance reduces denoising trajectory drift, beyond improvements in general forecasting metrics?
Response to Weakness 2a and Q4
Thank you for pointing out that the connection between the training-derived PCA space and predicted-noise correction was insufficiently explained. CSG is designed to mitigate denoising trajectory drift, rather than to use the principal components as direct predictions of the future.
Multivariate future windows are high-dimensional, but their dominant trends, periodic patterns, scale changes, and cross-variable dependencies are often concentrated in a lower-dimensional correlated structure. The PCA basis extracted from the training futures therefore provides a data-driven coordinate system: its span represents directions supported by the dominant training structure, while directions orthogonal to represent deviations from that structure.
This structural coordinate system can guide noise because the predicted noise determines the direction of each reverse update and therefore affects the shape, scale, and cross-variable relationships of the generated trajectory. CSG does not assume that the noise itself follows the future distribution. Instead, it uses to identify and attenuate the component of the predicted noise that drives the reverse trajectory away from the training-supported structure. Let denote the projection onto . The corrected noise satisfies
With , the orthogonal component is reversed and reduced to 20% of its original magnitude. Because is used to calculate the next state , this correction limits the accumulation of off-subspace deviations over subsequent reverse steps. In short, PCA supplies the structural reference, while noise correction acts on the direction that actually drives reverse sampling.
Response to Weakness 2b
Preserving all such details is not the specific objective of CSG. The role of CSG is limited to mitigating denoising trajectory drift during reverse sampling by reducing directions that are weakly supported by the dominant training-future structure.
Response to Weakness 2c and Q5
We added a matched-sample structural diagnostic comparing the same trained model with CSG disabled and enabled. The initial and stepwise sampling random numbers are paired. Each dataset uses 30 test windows and 50 predictive samples per window. Lower values indicate smaller structural deviation.
Each cell reports without CSG -> with CSG (reduction).
| Dataset | Out-of-subspace error | Scale discrepancy | ACF error | Spectral error | Cross-variable correlation error |
|---|---|---|---|---|---|
| ETTh2 | 0.433409→0.420508 (2.977%) | 0.438193→0.407700 (6.959%) | 0.268660→0.256018 (4.706%) | 0.011426→0.010904 (4.567%) | 0.353027→0.306317 (13.231%) |
| ETTm1 | 0.784229→0.779955 (0.545%) | 0.796002→0.791456 (0.571%) | 0.099580→0.099348 (0.232%) | 0.006224→0.006207 (0.269%) | 0.526262→0.522342 (0.745%) |
| Weather | 0.235438→0.233135 (0.978%) | 1.641348→1.632744 (0.524%) | 0.391509→0.385824 (1.452%) | 0.095826→0.094987 (0.875%) | 0.762517→0.757847 (0.612%) |
The largest reductions are observed on ETTh2, where trajectory-scale discrepancy, autocorrelation error, spectral-shape error, and cross-variable correlation error decrease by 6.96%, 4.71%, 4.57%, and 13.23%, respectively. These results quantitatively support that CSG can reduce structural deviation in the resulting forecast trajectories.
Response to Weakness 1 and Questions 1–2
Response. We thank the reviewer for identifying that the manuscript omitted important derivation details.
Weakness 1a and Q1
1. Exact calculation of the target local variance
Let the historical input and observed training future be concatenated as z=[x1:L,y1:H], where L and H denote the input and forecasting lengths. For each future position i and variable n, the target local variance is computed using a trailing window of length W. Specifically, the local mean is μi,n(W)=W1∑j=i−Wi−1zj,n, and the corresponding local variance is σy,i,n2=W1∑j=i−Wi−1(zj,n−μi,n(W))2+ε.
The implementation uses the population variance (
unbiased=False). The first forecasting position is estimated from the most recent historical observations, while later positions additionally use preceding ground-truth future values. Consequently, σy2 is available only during training and is never computed from the unknown test future at inference.2. Derivation of the base forward-noise variance
At diffusion step k, the clean future is attenuated by αˉk, so the total perturbation mass is βˉk=1−αˉk. The base conditional process decomposes this perturbation into two non-negative components, giving σbase,k2=(βˉk−β~k)σx2+β~kσy2.
Here, β~k is determined entirely by the diffusion schedule, and the implementation verifies that 0≤β~k≤βˉk. Therefore, both coefficients are non-negative and satisfy (βˉk−β~k)+β~k=βˉk.
The formula is consequently a schedule-controlled variance decomposition rather than an arbitrary interpolation. The term (βˉk−β~k)σx2 represents the portion supported by the history-conditioned variance prior, whereas β~kσy2 injects the realized local variability available during training. Importantly, the coefficients preserve the total perturbation coefficient βˉk. If σx2=σy2=σ2, the expression reduces to σbase,k2=βˉkσ2, which is the corresponding homoscedastic conditional-diffusion form.