acceptodds
Withdrawn from ICLR 2027

Adaptive Weighted -Transform Sampling for Coarse-Guided Visual Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Coarse-guided visual generation, which synthesizes fine visual samples from degraded or low-fidelity coarse references, is essential for various real-world applications. While training-based approaches are effective, they are inherently limited by high training costs and restricted generalization due to paired data collection. Accordingly, recent training-free works propose to leverage pretrained diffusion models and incorporate guidance during the sampling process. However, these training-free methods either require knowing the forward (fine-to-coarse) transformation operator, e.g., bicubic downsampling, or are difficult to balance between guidance and synthetic quality. To address these challenges, we propose a novel guided method by using the -transform, a tool that can constrain stochastic processes (e.g., sampling process) under desired conditions. Specifically, we modify the transition probability at each sampling timestep by adding to the original differential equation with a drift function , which approximately steers the generation toward the ideal fine sample. To address unavoidable approximation errors, we introduce an adaptive weight scheduler that combines a noise-level-aware initialization with a correction based on cross-timestep consistency, balancing guidance adherence and synthesis quality. Extensive experiments across diverse image and video generation tasks demonstrate its effectiveness and generalization.

Primary area:
generative models
Keywords:
Score Matching, Diffusion Model, Visual Genereation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PBr8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a training-free sampler for coarse-guided visual generation. It approximates an ideal h-transform using a coarse reference and adjusts guidance through a noise-dependent schedule and a cross-timestep consistency correction. Experiments cover image restoration, camera-controlled video generation, and additional editing applications.

Critical strengths
  1. The method is simple and broadly applicable, requiring neither additional training nor an explicit degradation operator. The formulation supports both diffusion and flow-matching models.

  2. The experiments demonstrate competitive image restoration performance and improvements in video quality. Table 2 reports better FVD and DINO distance than both the coarse input and the evaluated generation baselines.

Critical weaknesses
  1. The video evaluation protocol needs clarification. Section 6.2 defines generation from a first frame and prescribed camera poses. However, Appendix A.4.1 describes calibrating depth scale through SIFT matches between the reference frame and subsequent frames. If these are ground-truth target frames, preprocessing uses information unavailable under the stated task setting. Sharing the resulting coarse videos with a baseline would not resolve this mismatch between the evaluation protocol and the intended application.

  2. The central adaptive contribution is insufficiently validated. The practical formulation blends the model prediction with the coarse reference, making the weighting strategy central to Table 3 shows improvements over the fixed noise-dependent schedule, but these are relatively modest, and the adaptive correction is disabled in the video experiments. The consistency residual is not directly validated as a proxy for endpoint error. Furthermore, the additional model evaluations required by the correction are not accompanied by a clear runtime or equal-compute comparison.

Questions for authors
  1. Which frames are used for SIFT-based scale calibration in Appendix A.4.1? If ground-truth target frames are used, how does performance change when preprocessing uses only the first frame and prescribed camera poses?

  2. What are the model-evaluation count and runtime overhead of the adaptive correction? Can the authors provide an equal-compute comparison against the fixed noise-dependent schedule to establish whether the gains justify this overhead?

Reviewer_Z7rVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Weighted h-Transform Sampling, a training-free framework for coarse-guided image and video generation. The method approximates an ideal Doob's h-transform using the available coarse observation and introduces an adaptive weighting strategy based on noise levels and cross-timestep prediction consistency. Experiments on image restoration, camera-controlled video generation, and image editing are presented to demonstrate its effectiveness and generalization.

Critical strengths

The paper addresses a practical problem of incorporating coarse visual guidance into pretrained generative models without task-specific training or explicit knowledge of the degradation operator. The unified formulation across image and video generation is potentially useful, and the reported results demonstrate competitive performance on several restoration and video generation benchmarks.

Critical weaknesses
  1. Limited novelty relative to existing diffusion bridge and training-free guidance methods. The proposed framework is closely related to several existing studies, particularly Denoising Diffusion Bridge Models (ICLR 2024), Image Restoration Through Generalized Ornstein-Uhlenbeck Bridge (ICML 2024), and Training-Free Adaptation of Diffusion Models via Doob's h-Transform (ICML 2026). These works have already established the use of Doob's h-transform for endpoint-conditioned diffusion processes, image restoration, and training-free guidance, respectively. Although their task settings and implementations differ, the theoretical foundation of the proposed method is therefore not new. More importantly, Eqs. (12)–(15) show that the proposed guided score can be rewritten as a weighted interpolation between the pretrained score and a Gaussian conditional score centered on the coarse observation. The main additional contribution consequently appears to lie in the adaptive weighting strategy rather than in a fundamentally new sampling formulation. However, the noise-dependent initialization is relatively straightforward, and the cross-timestep consistency correction is not sufficiently justified or isolated experimentally. The authors should explicitly distinguish their contributions from these prior works and demonstrate that the proposed adaptive correction provides substantial advantages over conventional score interpolation and existing guidance scheduling strategies.

  2. Missing recent 2026 baselines. Although the paper includes HiFlow (2026), the comparisons omit several relevant recent methods, including Flower (ICLR 2026) and HDPS (ACM MM 2026) for image inverse problems, and WorldForge (CVPR 2026) for training-free camera-controlled video generation. These methods have publicly available implementations. While their assumptions differ, comparisons under compatible settings are important for assessing the claimed competitiveness against recent approaches.

  3. Insufficient justification of the adaptive weighting mechanism. The noise-dependent approximation error follows directly from the Gaussian score formulation. However, the paper does not establish why minimizing cross-timestep prediction inconsistency should reduce the actual endpoint approximation error. Stronger theoretical justification and comparisons with alternative adaptive weighting strategies are needed.

  4. Concerns about experimental validity and generalization. The video evaluation appears to use subsequent frames for depth-scale calibration, although the task is defined using only the first frame and camera parameters. The authors should clarify whether ground-truth future frames are involved. Moreover, the proposed consistency correction is disabled in video experiments, weakening the claimed generality of the complete framework.

Questions for authors

See weakness

Reviewer_AU71OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Adaptive Weighted h-Transform Sampling, a training-free method for coarse-guided visual generation using pretrained diffusion and flow-matching models. The method approximates an ideal h-transform using the available coarse reference and introduces a noise-dependent weighting schedule to control guidance strength during sampling. Experiments cover image restoration, camera-controlled video generation, and image editing.

Critical strengths
  1. The proposed approach is general and training-free.
  2. Experiments are extensive.
Critical weaknesses
  1. The overall performance improvements are limited and inconsistent.
  2. The writing is hard to follow.
  3. The formulation seems to be a weighted interpolation between the pretrained score and a reference-based score. The technical difference and performance advantages to existing approaches like SDEdit are not obvious enough.
  4. The motivation of tractable transform operator is not well-justified and weak. For example, SDEdit does not require tractable transform.
Questions for authors

See above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.