acceptodds
Under review as a conference paper at ICLR 2027

ANaLOG: Anisotropic Native-Latent Operator Guidance for Solving Inverse Problems

Abstract

Native-latent guidance is a recent paradigm for solving inverse problems with latent diffusion models. It replaces repeated evaluations of the image-space forward model, each requiring a decoder pass, with efficient guidance computed using a learned latent-space surrogate. However, existing methods apply guidance uniformly across latent dimensions, ignoring that measurements are informative only along certain directions and that the reliability of model predictions varies across inputs and timesteps. We propose ANaLOG, a framework for efficient uncertainty-aware guidance with pretrained latent diffusion models. ANaLOG models uncertainty by learning an anisotropic, input- and time-dependent covariance that is integrated into the guidance mechanism to emphasize reliable directions and downweight uncertain ones. We theoretically analyze this framework in a linear model setting and prove that anisotropic, uncertainty-aware weighting is necessary for correct sampling, whereas isotropic guidance induces sampling errors. Experiments across five challenging inverse problems show that ANaLOG improves perceptual reconstruction quality over existing methods while preserving efficiency.

Primary area:
generative models
Keywords:
inverse problems, latent diffusion model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_auRTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ANaLOG, a native-latent guidance method for solving inverse problems with latent diffusion models. Building on SILO, which guides sampling with a learned latent forward operator, it interprets that guidance as an isotropic Gaussian likelihood on the encoded measurement and extends it with a learned, input- and time-dependent diagonal covariance (in the spatial or DCT basis), trained by Gaussian negative log-likelihood. Guidance then uses a variance-weighted residual. A theoretical analysis in a linear Gaussian model with deconvolution operators shows that the true latent likelihood is anisotropic and that isotropic approximations incur a strictly positive KL gap. Experiments on FFHQ and ImageNet show improved perceptual quality over SILO on most tasks at essentially the same cost, with substantially faster sampling than decode-and-guide methods.

Critical strengths
  1. The paper is easy to read and frames the work clearly relative to covariance-aware guidance for pixel-space diffusion and to decode-and-guide and native-latent methods for latent diffusion models. Recasting SILO as an isotropic Gaussian likelihood is a clean way to show the need for a direction-wise covariance.
  2. Adding the learned covariance improves perceptual quality over SILO on most tasks and on both datasets at essentially the same inference cost, while remaining substantially faster than decode-and-guide methods.
Critical weaknesses

W1. The ablation does not support the claim that anisotropy, input dependence and time dependence are each needed.

  • The "time-independent" variant freezes the variance computed at the first sampling step. This seems to remove input dependence as well as time dependence, since the sample is pure noise at that point and the paper notes that early variance maps are relatively uniform. It therefore does not isolate the role of time dependence.
  • The intermediate variants give mixed results. For example, the time-independent variant improves FID over SILO but worsens LPIPS, and the theory-predicted covariance worsens FID. Only the full model improves all metrics, and only one task is ablated. A cleaner ablation, such as a variance head trained without the time input or a time-dependent but input-independent covariance, ideally on more than one task, would better support this central claim.

W2. Limited novelty and limited theoretical support.

  • Methodologically, the contribution is SILO plus a heteroscedastic variance head trained with a standard Gaussian NLL. The method also inherits the limitations of native-latent guidance, as the authors acknowledge. This includes task-specific training, and measurements that must be image-like.
  • The theory covers a restricted case with a linear Gaussian model with a linear autoencoder and deconvolution operators, which the authors present as a motivating example. Within it, the main results are standard. The sufficiency argument does not apply to inpainting or JPEG, and the theory-predicted covariance gives only mixed improvements over SILO in the ablation, so the theory does motivate covariance modeling but offers limited guidance for the method used in practice.
Questions for authors
  1. How should the basis (spatial vs. DCT) be chosen for a new operator? The better choice appears to vary across tasks; is there some principled criterion, or could an input-adaptive basis be learned?
  2. Figure 1 (right) is hard to interpret. The ellipse is labeled Cov(E(y)∣zt)\mathrm{Cov}(\mathcal{E}(y) \mid z_t) but is drawn around ztz_t in latent space, alongside the z0z_0 manifold. Please make this figure more clear.
  3. Cleaner covariance ablations as described in W1, ideally on more than one task.
  4. A comparison with ReSample, which is cited but not compared, as a standard latent-diffusion baseline.
3 comments
AuthorsOpenReview ↗

Rebuttal Response [1 / 2] (Weaknesses 1-2)

We thank the reviewer for their insightful feedback. We now respond in detail to each weakness and question.

“Ablation does not support claims” (Weakness 1 and Question 3)

We include a more thorough ablation study of covariance structure on both Gaussian deblurring and inpainting below. The ablation study in the paper used a small measurement noise, which we believe led to the mixed performance between the covariance variants that the reviewer pointed out. Specifically, with a convolutional operator the measurement noise determines which frequencies remain recoverable, and at the small noise used in the paper's ablation the posterior is tight enough that all covariance variants behave similarly. Below, we use measurement noise σy=0.08\sigma_y=0.08 for deblurring.

Following the reviewers' suggestions we add two rows: (a) a variance head trained without the timestep conditioning input, and (b) an isotropic variant that, at each step, replaces ANaLOG's predicted variance by its spatial mean, so that it retains the input and time dependence of ANaLOG but discards the anisotropy. Removing the time input largely degrades performance, demonstrating the importance of time-dependence of the covariance. Replacing ANaLOG with its spatial mean is the strongest alternative, as expected because it keeps the input and time-dependence. ANaLOG still yields the best perceptual quality on both tasks, indicating that input-dependence, time-dependence, and anisotropy are important. We have updated the paper to include this ablation study.

Gaussian deblurring

CovarianceFID ↓LPIPS ↓PSNR ↑
SILO52.310.39723.72
Covariance fixed to first timestep44.510.38224.71
Theory-predicted (input-independent, time-dependent)57.160.41724.57
ANaLOG without time input42.940.39624.99
Isotropic, spatial mean of ANaLOG variance34.630.35426.32
ANaLOG32.830.34726.62

Center inpainting

CovarianceFID ↓LPIPS ↓PSNR ↑
SILO32.830.14222.56
Covariance fixed to first timestep28.570.13922.27
ANaLOG without time input35.840.16018.59
Isotropic, spatial mean of ANaLOG variance25.670.13720.85
ANaLOG22.760.13521.28

“Limited novelty” (Weakness 2)

Please refer to global comment.

"Theory covers a restricted case” (Weakness 2)

We would like to clarify the novelty of our theoretical work. Notably, there is no existing work that studies native-latent guidance, so our work represents the first formalization. In our model, we study the approximations of native-latent guidance and when they are justified (Proposition 7). Further, we provide an exact characterization of approximation error for SILO and for our method (Theorem 8). Relative to prior latent diffusion posterior-sampling theory [1], our spectral data model further allows us to isolate the role of covariance structure and motivate our method. We believe these results are not standard and shed light on the importance of covariance modeling for native-latent guidance.

[1] L. Rout et al., “Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models,” NeurIPS 2023.

AuthorsOpenReview ↗

Rebuttal Response [2 / 2] (Questions 1-4)

“How should basis be chosen?” (Question 1)

We believe that the principled criterion is to match the covariance structure to the structure of the forward operator. Many inverse problems have forward operators that have frequency-domain structure (such as low pass filtering for blurring and downsampling), which is why we believe the DCT-diagonal covariance is generally a good choice for the covariance model. Indeed, in our experiments, across all tasks, ANaLOG-DCT yields competitive performance, often outperforming ANaLOG especially for JPEG decompression. The idea of learning an input-adaptive basis is a great idea, and the natural generalization is to parameterize the covariance as UΣUTU \Sigma U^T where UU and Σ\Sigma are both learned, perhaps with low-rank structure. The main constraint is that this introduces computational cost both at train time and inference time since we must invert this covariance and efficient representations must be developed. Nonetheless, this represents a fascinating direction for future work.

“Figure 1 is hard to interpret” (Question 2)

Thank you for the feedback. We have updated the figure to be more interpretable and we have explained its significance in the caption.

“Comparison to Resample” (Question 4)

We additionally compare against ReSample [2], a widely used decode-and-guide LDM inverse solver, on JPEG decompression. ReSample’s hard data-consistency procedure obtains slightly better FID and PSNR, whereas ANaLOG-DCT obtains substantially better LPIPS. Most importantly, ANaLOG-DCT is approximately 24× faster, demonstrating a favorable quality–efficiency trade-off.

MethodTime (s)FID ↓LPIPS ↓PSNR ↑
ReSample57625.70.42126.96
ANaLOG-DCT24.127.30.32626.26

[2] B. Song et al., “Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency,” ICLR 2024.

I thank the authors for the response. Most of my concerns have been addressed. I still hold my position on the limited novelty and theoretical contribution, thus, I will maintain my score.

Reviewer_bykpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ANaLOG, a guidance method for solving inverse problems with latent diffusion models. Building on native-latent guidance (SILO), it models the latent measurement likelihood as a Gaussian and learns an anisotropic, input- and time-dependent covariance (diagonal), which reweights the guidance. For a simplified linear encoder-decoder and circulant A, the authors show that the true likelihood is anisotropic and that isotropic approximations, such as SILO's, incur a positive KL gap. Experiments show perceptual gains over SILO at similar runtime.

Critical strengths
  • Probabilistic view of native-latent guidance that identifies the covariance as a missing degree of freedom.
  • Minimal overhead for diagonal covariance: only a small variance head is added to SILO, at the same sampling cost.
  • Perceptual gains over SILO and other examined baselines on most tasks.
Critical weaknesses
  • The algorithmic contribution is relatively minor: ANaLOG keeps SILO’s native-latent framework, operator architecture, and a similar guidance procedure. ANaLOG only adds a variance head that reweights SILO’s residual.

  • Efficiency claims are overstated and incomplete. In Table 1 you refer to methods based on pixel-space models as inefficient. Yet, many pixel-space methods are faster than latent-space methods on common benchmarks.
    Moreover, few-step solvers, e.g., LATINO (ICCV'25, https://arxiv.org/abs/2503.12615), which also uses a latent prior, and CM4IR (CVPR'25, https://arxiv.org/abs/2412.20596), are not discussed.
    Also, the runtime comparison does not account for the per-operator training required by native-latent methods (about 32 GPU-hours on ImageNet), whereas LDPS, PSLD and STSL are training-free.

  • The gains are not shown to come directly from anisotropy. SILO’s covariance is both isotropic and constant over time and input, while ANaLOG removes both restrictions at once. Because the learned variances (mostly below 1, Fig. 5a) also rescale the guidance step over time and input, part of the gain may come from this rescaling alone, i.e., from an isotropic but time-varying variance. The ablation in Fig. 5b does not separate the two. Please add an isotropic variant that uses, at each step, the spatial mean of ANaLOG’s predicted variance.

  • Ablation does not seem to support the claimed progressive trend. The theory-predicted covariance has worse FID than SILO, and the time-independent variant has worse LPIPS. The gains of these two variants are also rather minor and do not support the claim that ``progressively richer covariance models improve reconstruction quality'' (Fig. 5). The ablation also covers a single task.

  • Weak and possibly under-tuned baselines. LDPS and PSLD are run with 100/250 steps without re-tuning, while their code defaults to 1000, which may explain the artifacts in Figs. 4, 9--11 and 13. Please report their results for 1000 steps and consider newer methods.

  • Overstated results. The abstract claims perceptual gains ``over existing methods,'' but STSL is clearly better on ImageNet super-resolution (FID 38.2 vs. 71.1 at 4×4\times, 69.6 vs. 105.2 at x8), and spatial ANaLOG is worse than SILO on FFHQ SRx8 in FID and PSNR.

  • Limited evaluation. Only σy=0.02\sigma_y=0.02 is tested (pixel range not stated), whereas SILO and PSLD report higher noise levels. This matters especially here, since σy\sigma_y drives the anisotropy in the theory, and testing other levels is straightforward, as the networks are trained over σy∈[0,0.1]\sigma_y\in[0,0.1]. Moreover, a single LDM backbone (an SD-1.5 fine-tune) is used.

  • Your approach requires task/operator-specific training. I suggest that you add comparison with 1-2 task specific method. More importantly, examine sensitivity to mismatch in A or in the observation noise level (under and over estimation). Compare the sensitivity to SILO.

  • Some visual examples do not look very impressive (e.g., not sharp for both SILO and your method). To put the results in context, please report metrics and present visual results also for 1-2 state-of-the-art pixel-space methods.

Questions for authors

In addition to the above:

  • Can you report LDPS and PSLD both at 1000 steps (their code default) and at 100/250 steps with re-tuned guidance scales, with runtimes?
  • What is the pixel range for σy\sigma_y, and how does ANaLOG perform at higher noise levels?
  • How does ANaLOG compare with LATINO in quality and total cost?
4 comments
AuthorsOpenReview ↗

Rebuttal Response [1 / 3] (Weaknesses 1-2)

We thank the reviewer for their comments. We now respond in detail to each weakness and question.

“Algorithmic contribution is relatively minor” (Weakness 1)

Please refer to global comment.

“Efficiency claims are overstated” (Weakness 2)

We would like to clarify the distinction between efficient diffusion and efficient guidance in Table 1. Since pixel diffusion models operate at ambient image resolution, each diffusion step requires more computation than an LDM, which performs diffusion in latent space. However, most pixel-space guidance algorithms are indeed faster than latent-space guidance algorithms, which typically rely on expensive decoder evaluations at every guidance step. This is precisely the motivation of native-latent guidance, which permits the use of efficient diffusion models as well as efficient guidance algorithms.

“Comparison to LATINO” (Weakness 2 and Question 3)

We thank the reviewer for pointing us to solvers such as LATINO, which use latent consistency models (LCMs) to develop few-step inverse problem solvers. Specifically, at every timestep t, LATINO performs 3 steps:

  1. Use a distilled LCM to predict z0z_0.
  2. Decode to pixel space and apply a proximal step to enforce consistency with measurement.
  3. Renoise.

While this method still relies on the use of decoder at each inference step, the use of the consistency model can afford improved denoiser estimates, allowing for fewer sampling steps. The use of the LCM is orthogonal to our contribution of covariance modeling and thus can be combined with ANaLOG. Specifically, we run ANaLOG-LCM, which follows the same steps as LATINO while adding ANaLOG guidance. Specifically, at every timestep t, ANaLOG-LCM performs 4 steps:

  1. Use a distilled LCM to predict z0z_0.
  2. Apply ANaLOG guidance step using our pretrained mean and covariance head.
  3. Decode to pixel space and apply proximal step to enforce consistency.
  4. Renoise.

We run ANaLOG and ANaLOG-LCM on a Gaussian deblurring task on the FFHQ dataset below. Our results show two main conclusions. First, LATINO outperforms ANaLOG, likely due to the use of the more powerful LCM as opposed to the latent diffusion model denoiser. Second, ANaLOG-LCM improves reconstruction quality over LATINO, and ANaLOG-LCM shows potential for combining pixel-space and native-latent guidance. However, we note this requires more inference time than LATINO, so there is a tradeoff. We believe that uncertainty modeling remains critical in developing accurate native-latent guidance algorithms, and the interplay between native-latent and pixel guidance represents an intriguing direction for future work.

MethodPSNR ↑LPIPS ↓FID ↓Time (s/gen) ↓
LATINO28.770.31024.85.4
ANaLOG27.340.32629.823.7
ANaLOG-LCM (new)30.160.26622.19.3
AuthorsOpenReview ↗

Rebuttal Response [2 / 3] (Weaknesses 3-5)

“Improved Ablations” (Weakness 3 and 4)

We include a more thorough ablation study of covariance structure on both Gaussian deblurring and inpainting below. The ablation study in the paper used a small measurement noise, which we believe led to the similar performance between the covariance variants that the reviewer pointed out. Specifically, with a convolutional operator the measurement noise determines which frequencies remain recoverable, and at the small noise used in the paper's ablation the posterior is tight enough that all covariance variants behave similarly. Below, we use measurement noise σy=0.08\sigma_y=0.08 for deblurring.

Following the reviewers' suggestions we add two rows: (a) a variance head trained without the timestep conditioning input, and (b) an isotropic variant that replaces ANaLOG's predicted variance by its spatial mean, so that it retains the input and time dependence of ANaLOG but discards the anisotropy. Removing the time input largely degrades performance, demonstrating the importance of time-dependence of the covariance. Replacing ANaLOG with its spatial mean is the strongest alternative, as expected because it keeps the input and time-dependence. ANaLOG still yields the best perceptual quality on both tasks, indicating that input-dependence, time-dependence, and anisotropy are important. We have updated the paper to include this ablation study.

Gaussian deblurring

CovarianceFID ↓LPIPS ↓PSNR ↑
SILO52.310.39723.72
Covariance fixed to first timestep44.510.38224.71
Theory-predicted (input-independent, time-dependent)57.160.41724.57
ANaLOG without time input42.940.39624.99
Isotropic, spatial mean of ANaLOG variance34.630.35426.32
ANaLOG32.830.34726.62

Center inpainting

CovarianceFID ↓LPIPS ↓PSNR ↑
SILO32.830.14222.56
Covariance fixed to first timestep28.570.13922.27
ANaLOG without time input35.840.16018.59
Isotropic, spatial mean of ANaLOG variance25.670.13720.85
ANaLOG22.760.13521.28

“Under-tuned Baselines” (Weakness 5 and Question 1)

Thank you for pointing this out. We agree that LDPS and PSLD hyperparameters could be further tuned than the default choices from their paper. In the rebuttal period, we performed a sweep over LDPS and PSLD hyperparameters on both FFHQ and Imagenet. For LDPS on 8x super-resolution and inpainting and for PSLD on 8x super-resolution, we found the optimal hyperparameters for 100 steps were identical to the hyperparameters chosen in the paper. On the other tasks, we were able to tune the hyperparameters and the updated results for FFHQ are shown below. Across all tasks, ANaLOG and ANaLOG-DCT still maintain a strong advantage in perceptual quality of reconstructions (as measured by FID and LPIPS). We report the improved results in the revised version of the paper. Note that all claims in the text remain valid as our main comparison was to the native-latent baseline, SILO.

Gaussian deblurring

MethodFID ↓LPIPS ↓PSNR ↑
LDPS (retuned)70.40.535 (.003)22.18 (.13)
PSLD (retuned)44.10.463 (.002)26.72 (.08)
SILO47.30.367 (.002)25.03 (.06)
ANaLOG29.80.326 (.002)27.34 (.09)
ANaLOG-DCT30.90.338 (.003)26.36 (.10)

4x Super-Resolution

MethodFID ↓LPIPS ↓PSNR ↑
LDPS (retuned)59.20.541 (.003)21.76 (.08)
PSLD (retuned)39.60.432 (.003)26.46 (.09)
SILO38.80.331 (.002)26.26 (.06)
ANaLOG27.00.303 (.002)27.56 (.08)
ANaLOG-DCT26.40.302 (.002)26.92 (.09)

Center Inpainting

MethodFID ↓LPIPS ↓PSNR ↑
LDPS66.50.256 (.001)16.67 (.12)
PSLD (retuned)63.50.224 (.001)19.08 (.10)
SILO32.80.142 (.001)22.58 (.08)
ANaLOG22.80.135 (.001)21.31 (.11)
ANaLOG-DCT22.10.132 (.001)22.08 (.10)

JPEG Decompression

MethodFID ↓LPIPS ↓PSNR ↑
LDPS (retuned)67.90.562 (.003)20.92 (.08)
SILO68.30.471 (.002)20.87 (.06)
ANaLOG46.30.393 (.002)23.92 (.06)
ANaLOG-DCT27.30.326 (.002)26.26 (.09)

Using 1000 steps, LDPS and PSLD can perform better on PSNR but still lag to AnaLOG in FID and LPIPS. However, we do not believe this is a fair comparison since inference time is 26x slower per image, which is why we matched the number of diffusion step counts for all methods.

Gaussian deblurring

MethodStepsFID ↓LPIPS ↓PSNR ↑Time (s)
LDPS100083.40.358 (.006)28.62 (.24)518
PSLD100073.30.369 (.006)29.14 (.23)625
AuthorsOpenReview ↗

Rebuttal Response [3 / 3] (Weaknesses 6-9)

“Overstated results in abstract” (Weakness 6)

Thank you for pointing this out. We have adjusted the wording in the abstract in the revised paper.

“Limited evaluation and sensitivity to mismatch” (Weakness 7 and 8)

Below, we comprehensively test our covariance model at inference time on degradations it was not explicitly trained for. Our evaluation includes different blur kernels, JPEG quality factors, and measurement noise levels. We use the same checkpoints and guidance scales for all these tasks, so this experiment measures generalization to different degradations.

AxisDegradationvs. trainMethodFID↓LPIPS↓PSNR↑
BlurGaussian σ=1.5sharperSILO57.40.27824.06
ANaLOG-DCT40.60.22925.10
BlurGaussian σ=5.0widerSILO70.00.37524.39
ANaLOG-DCT64.40.42424.57
BlurMotionout-of-familySILO65.50.33323.65
ANaLOG-DCT53.60.33324.11
JPEGQ=5harderSILO87.40.42220.27
ANaLOG-DCT59.30.31923.76
JPEGQ=20easierSILO80.50.37520.95
ANaLOG-DCT33.70.17825.87
JPEGQ=40much easierSILO80.60.37220.83
ANaLOG-DCT35.50.19524.86
Noiseσ_y=0.05in-rangeSILO62.70.29724.27
ANaLOG-DCT47.40.28525.60
Noiseσ_y=0.10edge of rangeSILO67.00.31423.39
ANaLOG-DCT47.60.26725.43
Noiseσ_y=0.20out-of-rangeSILO75.40.39720.71

ANaLOG-DCT preserves the advantage over SILO in almost all tasks across all metrics, indicating successful generalization. The one exception is on stronger Gaussian blur where the learned covariance model (trained with a less severe Gaussian blur) tends to result in oversmoothed reconstructions, resulting in a slightly degraded LPIPS. In general, our experiments show that the measurement and guidance algorithm provide strong signal even when the latent forward operator is miscalibrated.

“Other LDM Backbones” (Weakness 8)

ANaLOG’s gains persist with a different LDM architecture and latent resolution. We evaluate Stable Diffusion 2.1, which uses vv-prediction and 4×96×964\times96\times96 latents, rather than the 4×64×644\times64\times64 RV-5.1 backbone used in the paper. Using the same architectural and training choices on Gaussian deblurring, ANaLOG improves all three metrics over SILO.

MethodFID ↓LPIPS ↓PSNR ↑
SILO61.470.45625.41
ANaLOG48.120.41426.42

“Putting the Results in Context” (Weakness 9)

We have added LPIPS metrics to each of the qualitative figures in the Appendix to make the comparison clearer.

“Pixel Range” (Question 2)

The pixel range is [-1, 1]. We have added this to experimental details in Appendix E.

Dear authors, thank you for the detailed rebuttal and the new experiments. The requested isotropic control shows that replacing ANaLOG’s variance by its spatial mean recovers most of the gain over SILO (about 90% on deblurring and 70% on inpainting). The improvement therefore comes mainly from a learned time- and input-dependent rescaling of the guidance rather than from anisotropy, whose role seems overstated in the abstract and throughout the paper. Given this and the somewhat limited novelty relative to SILO, I lean toward keeping my original score.

Reviewer_LsZxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ANaLOG, an uncertainty-aware inverse solver that builds on latent space guidance. Building on a prior work called SILO, it learns an input and timestep dependent covariance matrix for the encoded measurement likelihood. It then uses this covariance to weight guidance residuals during inference. Spatial and DCT-diagonal parameterizations are provided as efficient implementations of this guidance term. Experiments cover image restoration problems like deblurring, super-resolution, inpainting, and JPEG decompression.

Critical strengths
  1. The improvement over SILO is clearly motivated i.e., a uniform residual penalty ignores differences in prediction uncertainty across latent directions and diffusion timesteps.
  2. The method appears to achieve substantial perceptual improvements with similar inference times and a small parameter overhead.
  3. Ablations examine covariance structure, temporal dependence, calibration, and covariance gradients.
  4. The linear-Gaussian analysis provides a useful setting in which anisotropic weighting is justified.
Critical weaknesses
  1. The method is primarily an extension of SILO [1]. The learned latent operator and avoidance of repeated decoding are inherited; the principal addition is Gaussian likelihood modeling and uncertainty-weighted guidance using the Mahalanobis distance.
  2. Stopping gradients through the input-dependent covariance omits terms from the Gaussian score. The empirical stability argument is reasonable, but the resulting method should be distinguished more clearly from exact likelihood guidance methods.
  3. The advantage of DCT parameterization appears to be task-dependent. It substantially helps JPEG and some super-resolution settings, but underperforms spatial covariance on FFHQ deblurring. The evidence supports covariance learning more strongly than a generally superior frequency-domain parameterization.
  4. Can the likelihood approximation guarantees be connected to any errors incurred in posterior sampling. Ideally, including the effects of covariance stop-gradients?

Overall: This work is a useful extension of SILO with convincing empirical gains shown in image inverse problems. My main reservations are due to incremental novelty and the gap between the likelihood theory and the practical posterior sampling claims.

References:

  1. Raphaeli, Ron, Sean Man, and Michael Elad. "Silo: Solving inverse problems with latent operators." In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10570-10580. IEEE, 2025.
Questions for authors

Please see weaknesses.

2 comments
AuthorsOpenReview ↗

Rebuttal Response

We thank the reviewer for their positive evaluation of our work. We respond to weaknesses and questions below.

“Primarily an extension of SILO” (Weakness 1)

Please refer to global comment.

"Gap between Likelihood Theory and Practical Posterior Sampling” (Weakness 2 and Weakness 4)

In Table 7 of Appendix F.4, we demonstrate using our method as an exact likelihood guidance method. Indeed, for carefully tuned scale, the exact likelihood can also give the same performance as our practical algorithm. However, we found that the practical version of our algorithm with stop-gradients for the covariance was more stable to scale selection, which is why we adopted it for the experiments. We thus view the practical algorithm we propose as not being very far from the exact likelihood, but allowing for practical guidance stability.

Note that in our linear-Gaussian setting, Proposition 7 gives an exact answer to the stop-gradient question. In that case, the true likelihood covariance CtC_t varies with the timestep but is independent of the noisy latent ztz_t, so holding CtC_t fixed when differentiating with respect to ztz_t gives exactly the same likelihood score as full differentiation.

Dear authors: Thank you for the clarification. I will maintain my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision Summary and Global Response

Revision Summary

The insightful feedback by the reviewers (LsZx, bykp, auRT) has improved the experimental depth and clarity of our paper. We summarize below the changes made to the main paper (shown in blue text in the revision).

  1. In Appendix F.5, we add a comparison with LATINO and demonstrate how its use of latent consistency models can further improve ANaLOG. (bykp Weakness 2 and Question 3)

  2. In Section 5.2 and Appendix F.4, we provide a more detailed ablation study of covariance structure on two tasks that shows the importance of input dependence, time-dependence, and anisotropy of the covariance. (bykp Weakness 3 and 4, auRT Weakness 1 and Question 3)

  3. We update Table 1 after further tuning the hyperparameters of LDPS and PSLD baselines. The claims in the text remain unchanged as our main comparison is against the existing native-latent guidance algorithm, SILO. (bykp Weakness 5 and Question 1)

  4. We clarify baseline comparisons in the abstract. (bykp Weakness 6)

  5. In Appendix F.6, we test the sensitivity of ANaLOG and SILO to other forward operators mismatched with the training setup and varying noise levels. (bykp Weakness 7 and 8)

  6. In Appendix F.1, we include metrics alongside all qualitative results for better comparison. (bykp Weakness 9)

  7. In Appendix G, we include a discussion of choosing the basis for new operators. (auRT Question 1)

  8. We make Figure 1 clearer to better illustrate the method. (auRT Question 2)

On Novelty (LsZx Weakness 1, bykp Weakness 1, auRT Weakness 2)

The reviewers raised concerns about the novelty of our approach. We would like to clarify that our contribution is both conceptual and algorithmic. Our central conceptual contribution is developing a new probabilistic framework of native-latent guidance, which involves reinterpreting SILO as approximate inference under a Gaussian likelihood. This contribution is what identifies covariance modeling as the hidden degree of freedom, which we view as a novel contribution. The simple algorithmic implementation that follows is a strength of our work, demonstrating that these covariance models can lead to substantial improvements in reconstruction quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.