acceptodds
Under review as a conference paper at ICLR 2027

SDE Trajectory Steering with Spatial-Aware VLM Self-Verification: Test-Time Scaling for Image Editing

Abstract

Test-time inference strategies for diffusion-based editing currently face distinct bottlenecks. Standard ODE sampling is rigid and cannot correct early errors; Best-of-N scaling is computationally prohibitive; and recent tree-search alternatives introduce external reward semantic discrepancies, devolving into undirected stochastic sampling via directionless prune-and-random-restart mechanisms. To address these compounding limitations, we propose SdeSteer, a novel training-free test-time scaling framework. Our approach replaces blind noise injection with an adaptive prune-and-steer mechanism. By progressively pruning misaligned sampling paths and dynamically modulating the classifier-free guidance (CFG) scale based on an intermediate verification heuristic, SdeSteer actively steers SDE-induced noise to synthesize novel, optimized sampling trajectories. Furthermore, we achieve a zero-overhead architecture by dual-purposing the native Vision-Language Model (VLM) backbone as both the text encoder and an internal verifier. To ensure reliable evaluation without auxiliary dependencies, we introduce a spatially-aware verification strategy that effectively mitigates the self-enhancement bias typical of same-model assessment. Extensive experiments demonstrate that our actively guided test-time compute translates directly into superior generative alignment and visual quality.

Primary area:
learning theory
Keywords:
Test-time Scaling, Diffusion-based Editing, Prune-and-Steer, Self-Verification with Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_h45wOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SDESteer, a training-free test-time scaling method for instruction-based image editing. It branches multiple SDE trajectories from a shared noise, and the VLM built into the editing model scores and prunes the branches at intermediate steps. The surviving branches are steered by adjusting the CFG scale based on that score, which gives the trajectory a direction, unlike existing tree search that restarts with random noise. The core design is to use the editing model's own VLM as the verifier instead of an external reward model, and the authors claim this removes the judgment mismatch with external models and the cost of an additional model.

Critical strengths

S1. The method is simple and practical. It reuses the VLM that already exists in the editing model as the verifier, and no additional model is needed. The shared-prior branching and score-based CFG control are also easy to implement. If the results hold, this method can be used directly for test-time scaling in image editing.

S2. The reported effect is large and the evidence points in the same direction. The success rate is higher than Best-of-N on the same backbone, and this gap appears consistently across the VLM judge, human evaluation, ablation, and all eight editing types. If the gap survives the comparisons requested in W1, it would be a strong result.

Critical weaknesses

W1. There is no comparison or analysis that supports the novelty and superiority of the main claims. All of the paper’s contributions are comparative claims of being better than existing methods, but on the same backbone the only comparisons are a single ODE run and Best-of-N, whose final selection criterion is not defined. The remaining baselines are separate editing models run with their own official implementations and base models. None of the tree search methods the authors criticize are in the comparison. Also, this is a test-time scaling method, but compute costs such as wall-clock time, number of forward passes, number of VLM calls, and API cost are not reported. More model calls will of course help. The question is whether this method uses them better than Best-of-N or tree search at the same budget, and the paper does not show this.

It has been shown in several studies that the initial noise strongly affects the result [1, 2, 3], but this paper starts all branches from a single noise and gives up initial noise diversity. If diversity among branches is low, the search can become inefficient. There is no analysis of whether independent initialization is actually worse or whether initialization variance ever helps. The novelty of the steering is also unclear. Gradient guidance methods such as DPS [4] and FreeDoM [5] actively steer by changing the drift directly with the reward gradient, and FK steering [6] and DAS [7] guide the search with reward-weighted resampling. The paper groups existing TTS into either passive filters or random restarts. Gradient guidance belongs to neither category, and reward-weighted resampling is not blind guessing. There is no explanation of how adjusting the CFG scale with verifier scores differs from them, and without this it is hard to judge the novelty of the core contribution.

W2. The design rationale of the method and the evaluation design do not support the conclusions. First, it is hard to tell how reliable the spatial attention mask M is. There is no description of how it is extracted or how accurate it is. As in the Figure 2 example, it may work well for localized edits, but the behavior may differ when a larger object has to change or when the global style or background has to change. I am concerned that the method targets too narrow a type of editing. In fact, of the nine types in EdiVal-Bench, the authors selectively report only eight, excluding background change, the only global edit.

Second, there is no rationale or analysis for why Eq. 6, 8, and 9 are designed this way, and they feel heuristic. The ablation is also insufficient. Table 3 consists only of three variants that each remove one component, with a single seed and no variance. Given my concern about the reliability of the VLM score, the score differences look marginal. Design choices such as the shared prior versus independent initialization, the form of the equations and hyperparameter selection, and the choice of verifier VLM are hard to accept based on the current results alone.

Third, the evaluation is measured only with a single VLM (gemini-3.1-pro), and its rubric has the same IF/BP/IQ structure as the method’s internal verifier. I cannot rule out that outputs preferred by the internal verifier are carried directly into the external judge’s score. The authors did not use the official EdiVal-Agent metrics. The human evaluation is small, with eight tasks, and the number of images, the selection procedure, and significance testing are not reported.


[1] Guo et al. InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization. CVPR 2024.

[2] Zhou et al. Golden Noise for Diffusion Models: A Learning Framework. ICCV 2025.

[3] Ma et al. Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps. CVPR 2025.

[4] Chung et al. Diffusion Posterior Sampling for General Noisy Inverse Problems. ICLR 2023.

[5] Yu et al. FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model. ICCV 2023.

[6] Singhal et al. A General Framework for Inference-time Scaling and Steering of Diffusion Models. ICML 2025.

[7] Kim et al. Test-time Alignment of Diffusion Models without Reward Over-optimization. ICLR 2025.

Questions for authors
  1. What is the final selection criterion of Best-of-N? What are the wall-clock time and the number of forward and VLM calls for Baseline, Best-of-N, and SDESteer? Can you compare against Best-of-N at the same budget, and against one of the tree search methods you cite (DTS, FK steering, DAS, etc.) on the same backbone?
  2. Is there a comparison between the shared prior and independent initialization? How does adjusting the CFG scale with verifier scores fundamentally differ from gradient guidance or reward-weighted resampling?
  3. How is the mask M extracted, and how accurate is it?
  4. What happens when an external VLM is used as the verifier? If an external VLM is unreliable as a verifier, why is gemini reliable as a judge?
  5. Please report how the final output of SDESteer relates to the capability of the verifier VLM, the design rationale of Eq. 6, 8, and 9 with comparisons to alternatives, and the number of images and the selection procedure of the human evaluation.
  6. The paper uses only Qwen-Image-Edit-2511 as the backbone. Results on other editing models would make the method’s generality clearer.
Reviewer_xLP1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SDESteer, a training-free test-time scaling framework for image editing. The method combines stochastic trajectory branching, spatial-aware VLM self-verification, and adaptive classifier-free guidance (CFG) to improve instruction following and background preservation. Experiments on EdiVal-Bench report improvements over Best-of-N sampling and several image editing baselines.

Critical strengths
  1. Practical important. The paper addresses an important problem in inference-time scaling for image editing, aiming to improve generation quality without additional model training.

  2. Promising empirical results. The method achieves substantial improvements in reported success rates and includes component ablations and human evaluations.

Critical weaknesses
  1. Limited methodological novelty.

The proposed framework largely combines existing test-time search, VLM-based verification, and adaptive CFG techniques. The direction-aware steering mechanism mainly adjusts the CFG scale based on scalar feedback, without explicitly estimating a targeted correction direction. Its distinction from existing inference-time steering approaches[1][2] remains insufficiently justified.

  1. Insufficient experimental controls and unclear sources of improvement.

The Best-of-N comparison lacks sufficient clarification regarding matched computational budgets and evaluation strategies. Table 3 also indicates that steering contributes relatively modest improvements compared with native self-verification. Stronger baselines and ablations are needed to explain the substantial overall performance gains. Furthermore, the paper does not clearly explain how the proposed SDE sampling formulation is implemented with the flow-matching backbone.

  1. Limited benchmark coverage and generalization.

The evaluation is restricted to a single benchmark, EdiVal-Bench, making it difficult to assess the generalizability of the reported improvements. Additional experiments on established image editing benchmarks, such as GEdit-Bench, ImgEdit-Bench, and I2EBench, would provide stronger evidence of robustness across different editing tasks and evaluation settings.

[1] Shufan Li et al., Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflection. ICCV 2025.

[2] Raghav Singhal et al., A General Framework for Inference-time Scaling and Steering of Diffusion Models. ICML 2025.

Questions for authors
  1. How does the proposed approach fundamentally differ from existing inference-time search and steering methods, such as [1][2]? Does the proposed steering outperform fixed CFG or alternative dynamic CFG schedules under identical computational budgets?

  2. Are Best-of-N and SDESteer evaluated using the same verification strategy and comparable inference costs?

  3. Can the authors provide additional results on GEdit-Bench, ImgEdit-Bench, or I2EBench to demonstrate generalization beyond EdiVal-Bench?

[1] Shufan Li et al., Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflection. ICCV 2025.

[2] Raghav Singhal et al., A General Framework for Inference-time Scaling and Steering of Diffusion Models. ICML 2025.

Reviewer_bt1SOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a training-free test-time scaling framework that improves instruction-guided image editing without retraining the base model. The adaptive prune-and-steer mechanism removes low-scoring branches and adjusts the CFG scale of surviving branches based on verification scores. The self-verification architecture reuses the native VLM as both the multimodal encoder and the internal verifier, avoiding the need for an additional reward model.

Critical strengths
  1. The training-free design is practically useful, as it improves editing performance without the cost of retraining the base model.
  2. The stage-aware verification is well motivated: spatial comparisons are used when early previews are difficult to interpret, while semantic evaluation is introduced later.
  3. The reported gains are substantial and supported by both external-model and human evaluations.
Critical weaknesses
  1. The novelty appears primarily compositional. dMLLM-TTS has explored self-verification and hierarchical trajectory search, while ELECT uses early spatial consistency to select editing candidates. The paper provides limited insight beyond combining these ideas.
  2. Experiments are limited to Qwen-Image-Edit-2511, leaving generalization to other editing backbones untested, particularly given the reliance on the native VLM’s verification capabilities.
  3. The paper lacks time complexity, end-to-end latency measurements, or compute-budget comparisons. Additional decoding and VLM verification costs are not quantified.
  4. The steering ablation does not establish whether the gains come from the adaptation or simply stronger CFG. Comparisons with tuned fixed guidance and simple time-dependent schedules are needed to isolate the proposed mechanism.
Questions for authors
  1. Could the authors explain the evaluation protocol in more details?
  2. Do the reported gains generalize beyond EdiVal-Bench?It would be better to test the propose method on more benchmarks.
  3. Could the authors explain how the spatial mask is extracted from the native VLM and how sensitive the method is to errors in the VLM-derived spatial mask?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.