acceptodds
Under review as a conference paper at ICLR 2027

FASTER: Fast Adjoint Stochastic Transport for Endpoint Refinement in Reward-Guided Image Editing

14 views

Abstract

Reward-guided image editing at test time seeks to improve a specified reward while preserving source content and visual plausibility. Many existing approaches optimize candidates through pretrained generation processes, making repeated adjustment depend on costly large-model execution and, in some cases, backbone backpropagation. We develop a theoretical framework that jointly accounts for reward, source preservation, and pretrained-prior preferences, allowing the desired output distribution to be specified separately from the dynamics used to realize it. Based on this framework, we introduce FASTER, which trains a small network for each source and objective to perform inexpensive editing, while pretrained and reward models provide feedback on candidate outputs. By reusing each candidate and its feedback across multiple small-network updates, FASTER reduces repeated sampling and supervision queries without placing the pretrained generative backbone inside the inner optimization loop. On SD3, FASTER leads all four target metrics and several validation metrics among the evaluated methods. Compared with the evaluated baseline that optimizes controls along pretrained generation trajectories, FASTER achieves editing-time speedups of up to \(6.91\times\) on Stable Diffusion 3 and \(24.14\times\) on Stable Diffusion 1.5.

Primary area:
generative models
Keywords:
Test-Time Image Editing, Reward-Guided Generation, Endpoint Redistribution, Stochastic Optimal Control, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_fMTFOpenReview ↗
Rating
3(+1)
Confidence
4(+1)
Informativeness
3
Presentation
3(+1)
Soundness
3
Read review
Paper summary

This paper proposes FASTER, a test-time reward-guided image editing method that decouples pretrained generative knowledge from the editing dynamics. Instead of repeatedly executing a large diffusion or flow model, FASTER constructs a source-conditioned endpoint distribution incorporating reward improvement, source preservation, and pretrained-prior preferences. A lightweight stochastic controller is then trained independently for each source image and editing objective. By adopting Brownian reference dynamics and reciprocal adjoint matching, the method reuses endpoint supervision through Brownian-bridge sampling, reducing the computational cost of repeated optimization. The authors additionally introduce a semantic Editing Term derived from differences between source- and target-conditioned velocity predictions. Experiments on SD3 and SD1.5 cover four reward-guided editing tasks. FASTER achieves the best target metrics across all four SD3 tasks and reports substantial speedups relative to ITOC, reaching up to 6.91× on SD3 and 24.14× on SD1.5.

Critical strengths
  1. Decoupling the editing process from pretrained generative dynamics is an appealing direction. If successful editing can be achieved using lightweight test-time optimization without repeated diffusion-model execution, the implications could extend beyond computational acceleration.

  2. The use of Brownian reference dynamics simplifies adjoint supervision, while Brownian-bridge regression allows expensive endpoint feedback to be reused. The overall formulation is mathematically coherent and computationally motivated.

  3. FASTER achieves the best optimized reward metrics across all four SD3 tasks and improves several independent validation metrics. The speedups relative to ITOC are substantial and demonstrate the practical feasibility of lightweight per-instance optimization.

Critical weaknesses
  1. A successful image edit should introduce the requested semantic change while preserving the unrelated content and perceptual quality of the source image. However, the main experimental results primarily demonstrate improvements in the reward functions being optimized.

  2. A central motivation is to retain pretrained generative knowledge while separating editing from pretrained generation trajectories. However, Table 2 shows that the pretrained-prior coefficient is zero in five of the eight experimental configurations, including three of the four SD3 tasks. Furthermore, when ατ² = 1 and the prior coefficient is zero, Eq. (11) simplifies exactly to a scaled reward gradient. Consequently, several principal experiments train the controller using reward-gradient supervision without explicitly querying the diffusion backbone. This observation is potentially important: it suggests that certain editing objectives may be optimized without pretrained generative dynamics. However, the paper does not adequately investigate why this works or whether the resulting images retain the semantic and perceptual properties expected from generative editing. Additionally, the practical prior estimator corresponds to a noisy-prior surrogate rather than necessarily recovering the clean pretrained distribution used in the theoretical formulation. The paper lacks systematic ablations over prior strength, noise-time ranges, and the number of prior-score queries. These experiments would clarify whether pretrained-prior guidance provides meaningful benefits or whether the principal results are better understood as reward-driven latent optimization.

  3. The general idea of optimizing a neural representation separately for an individual image is not entirely new, as related ideas appear in Deep Image Prior [1] and Text2LIVE [2]. Although FASTER introduces a different stochastic-control formulation, the practical necessity of this formulation is not sufficiently established. The controller contains a full latent-shaped trainable offset and a small three-layer convolutional network. In the SD3 configuration, the trainable offset accounts for the vast majority of its parameters. This raises the question of whether much of the observed performance could be achieved by directly optimizing the latent offset.

  4. The proposed objective explicitly includes a source-preservation penalty, yet the main results do not directly report source-output LPIPS, source-output CLIP-I, or task-specific identity and structural preservation metrics. These evaluations are particularly important because a uniform latent-space L2 penalty may suppress necessary semantic changes, leading to under-editing, while excessive reward optimization may cause unintended modifications. Additionally, the method fixes the controller's Euler–Maruyama rollout to 16 steps for both SD1.5 and SD3. There is no systematic investigation of whether similar results could be achieved using 1, 2, 4, or 8 steps. Given the simple controller architecture and the presence of a trainable latent offset, a few-step or even one-step alternative seems worth investigating.

Minor Weakness:

  1. Figure 2 is somewhat difficult to follow, particularly the distinction between controller rollout, endpoint feedback computation, cached-target reuse, and controller parameter updates. The diagram also does not clearly convey which operations involve the frozen pretrained backbone and which are performed by the lightweight trainable controller. I encourage the authors to redraw or reorganize this figure into a clearer step-by-step pipeline.

Ref:

[1] Deep image prior. (CVPR)

[2] Text2LIVE: Text-driven layered image and video editing(ECCV)

Questions for authors

In rebuttal, I want to see:

  1. In configurations without diffusion-prior guidance, can the authors demonstrate that reward improvements correspond to actual semantic editing success rather than reward exploitation? Can they provide task-specific editing success rates, source preservation, and perceptual-quality evaluations?

  2. Why is the pretrained-prior coefficient zero in most configurations? What happens when it is enabled? Can the authors investigate different prior strengths, noise-time ranges, and multiple noisy-score queries to clarify whether the pretrained prior meaningfully improves the editing results?

  3. How does FASTER compare with optimizing only its trainable latent offset, using a deterministic controller, or performing direct regularized latent optimization? Can the authors isolate the individual benefits of Brownian-bridge regression and learned stochastic dynamics under matched computational budgets?

  4. Why are 16 controller rollout steps necessary? Have the authors evaluated N ∈ {1, 2, 4, 8, 16}? How does the overall quality-efficiency trade-off change when K, M, and N are jointly varied under a fixed wall-clock or reward-query budget?

  5. Could the authors provide more extensive side-by-side qualitative comparisons across diverse editing instructions Additionally, could the authors include comparisons with recent inversion-free editing methods, such as FlowEdit [1], FlowChef [2], and ChordEdit [3].

Ref:

[1] FlowEdit: Inversion-free text-based editing using pre-trained flow models.(ICCV)

[2] FlowChef: Steering of rectified flow models for controlled generations.(ICCV)

[3] ChordEdit: One-step low-energy transport for image editing. (CVPR)

8 comments

We thank Reviewer fMTF for the thorough review and valuable suggestions for strengthening our evaluation and clarifying the methodological details.

Weakness 1: A successful image edit should introduce the requested semantic change while preserving the unrelated content and perceptual quality of the source image. However, the main experimental results primarily demonstrate improvements in the reward functions being optimized.

AW1: Our evaluation is not limited to improvements in optimized rewards: it also assesses perceptual quality and provides qualitative evidence of source-content preservation. : Table 1 also reports multiple non-target validation metrics assessing perceptual quality, semantic alignment, and style similarity. Figure 4 and Appendix F (Figures 8–11) further provide qualitative comparisons of editing effects and retained source content.

Weakness 2(1): A central motivation is to retain pretrained generative knowledge while separating editing from pretrained generation trajectories. However, Table 2 shows that the pretrained-prior coefficient is zero in five of the eight experimental configurations, including three of the four SD3 tasks. Furthermore, when ατ2=1\alpha\tau^2 = 1 and the prior coefficient is zero, Eq. (11) simplifies exactly to a scaled reward gradient. Consequently, several principal experiments train the controller using reward-gradient supervision without explicitly querying the diffusion backbone.

AW2(1): Thanks for your insights. We would like to clarify that reward-gradient supervision does not imply the absence of source preservation. As shown in Eq. (5), even when ατ2=1\alpha\tau^2=1 and the pretrained-prior coefficient is zero, the target distribution remains

\propto \exp\left( -\frac{|\mathbf{y}-\mathbf{s}|^2}{2\tau^2} +\beta r(\mathbf{y}) \right). \tag{5}$$ The source-preservation term is already incorporated into the Brownian reference dynamics (Eq. (6)), explaining its cancellation in Eq. (11). Moreover, the source image itself provides a strong prior, making explicit pretrained-prior guidance optional for certain editing objectives. We further investigate its effects through the ablation experiments in **AW2(4)**. > **Weakness 2(2)**: This observation is potentially important: it suggests that certain editing objectives may be optimized without pretrained generative dynamics. However, the paper does not adequately investigate why this works or whether the resulting images retain the semantic and perceptual properties expected from generative editing. **AW2(2):** Thanks for your question. FASTER retains pretrained generative knowledge without relying on pretrained generation trajectories. Specifically, it uses the **pretrained latent space and frozen decoder for image editing and evaluation**, while the pretrained diffusion backbone can provide prior scores (Eq. 11) and semantic feedback (Sec. 4.3) at candidate endpoints, rather than executing the editing process. As explained in Sections 4.1–4.2 and Appendices B–C, the source image itself provides a strong structural prior, allowing certain tasks to rely primarily on reward gradients. Our evaluation goes beyond optimized rewards, with additional metrics in Table 1 and visual results in Figures 4, 8, and 10–11. The need for explicit pretrained-prior guidance is task-dependent, further investigated through our new ablation on $c_p$ in Appendix I.4. > **Weakness 2(3)**: Additionally, the practical prior estimator corresponds to a noisy-prior surrogate rather than necessarily recovering the clean pretrained distribution used in the theoretical formulation. **AW2(3)**: Thank you for your comment. Using a noisy-prior surrogate is a common practice in **optimization with pretrained diffusion priors**; for example, EAM [1], DreamFusion and Feng & Bouman (2024) [3] also adopts this practice. In addition, for velocity-field models such as SD3, we cannot directly obtain the prior term at the data timestep. Therefore, using a small-noise prior proxy avoids the time cost of training a prior model and instead allows us to immediately leverage an already well-trained diffusion prior. [1]: Shin, Jeongwoo, et al. "Efficient Adjoint Matching for Fine-tuning Diffusion Models." *arXiv preprint arXiv:2605.11480* (2026). [2] Poole et al. *DreamFusion: Text-to-3D using 2D Diffusion.* ICLR, 2023. [3] Feng and Bouman. *Variational Bayesian Imaging with an Efficient Surrogate Score-based Prior.* TMLR, 2024.

Weakness 2(4): The paper lacks systematic ablations over prior strength, noise-time ranges, and the number of prior-score queries. These experiments would clarify whether pretrained-prior guidance provides meaningful benefits or whether the principal results are better understood as reward-driven latent optimization.

Q2(3) Can the authors investigate different prior strengths, noise-time ranges, and multiple noisy-score queries to clarify whether the pretrained prior meaningfully improves the editing results?

AW2(4)&A2(3): Thank you for your comment. Here, we have prepared three tables of quantitative experiments for your reference. Empirically, the prior term weakens editing by pulling images toward an “average image”, so its use should depend on the specific editing performance. The noise-time range generally involves a trade-off; we recommend [0.1,0.4][0.1, 0.4], which offers a reasonable prior proxy while maintaining a reliable prior. The number of prior-score queries has little effect on performance but increases runtime, so we recommend using a single query.

Effect of pretrained-prior strength cpc_p. Here, we set noise-time range: [0.10,0.40][0.10,0.40] and 1 prior-score query when prior guidance is enabled.

cpc_pGramDiff ↓\downarrowCLIP-I ↑\uparrowDINO-S/16 ↑\uparrowTime (s) ↓\downarrow
0 (no prior guidance)5.45290.49280.15165.26
0.0015.45960.49250.15176.59
0.0035.46340.49240.15206.60
0.015.48500.49270.15276.60
0.035.54580.49280.15466.59
0.15.63090.49610.16236.58
0.35.90430.49910.17586.59
1.06.73100.48540.18846.58

Effect of the noise-time range used for prior estimation. Here we set cp=0.1c_p=0.1 and 1 prior-score query.

Noise-time rangeGramDiff ↓\downarrowCLIP-I ↑\uparrowDINO-S/16 ↑\uparrowTime (s) ↓\downarrow
[0.05,0.20][0.05, 0.20]5.80660.49680.17886.58
[0.10,0.40][0.10, 0.40]5.63090.49610.16236.58
[0.40,0.70][0.40, 0.70]5.49000.49320.15146.58

Effect of the number of prior-score queries. Here we fix cp=0.1c_p=0.1 and noise-time range: [0.10,0.40][0.10,0.40].

Prior-score queriesGramDiff ↓\downarrowCLIP-I ↑\uparrowDINO-S/16 ↑\uparrowTime (s) ↓\downarrow
15.63090.49610.16236.58
25.62190.49520.16277.88
45.62090.49510.162910.35

Weakness 3(1): The general idea of optimizing a neural representation separately for an individual image is not entirely new, as related ideas appear in Deep Image Prior [1] and Text2LIVE [2]. Although FASTER introduces a different stochastic-control formulation, the practical necessity of this formulation is not sufficiently established.

AW3(1): Thank you for pointing this out. FASTER distinguishes itself from DIP [1] and Text2LIVE [2] through an endpoint-based SOC formulation that decouples editing from pretrained generation. The necessity of our SOC formulation lies in enabling multiple lightweight controller updates to reuse endpoint feedback, thereby reducing optimization cost. This practical benefit is supported by improvements over GA (Table 1) and the effect of feedback reuse at matched numbers of feedback evaluations (Figure 15).

[1] Ulyanov et al. Deep Image Prior. CVPR, 2018.

[2] Bar-Tal et al. Text2LIVE: Text-Driven Layered Image and Video Editing. ECCV, 2022.

Weakness 3(2) The controller contains a full latent-shaped trainable offset and a small three-layer convolutional network. In the SD3 configuration, the trainable offset accounts for the vast majority of its parameters. This raises the question of whether much of the observed performance could be achieved by directly optimizing the latent offset.

AW3(2): Thank you for pointing this out. Optimizing the latent offset alone is effective, but performs worse than the combined controller m+fψm+f_\psi(CNN). Although mm accounts for most parameters, it represents a state- and time-independent offset, whereas the CNN fψf_\psi provides state- and time-dependent corrections. We compare the two settings under identical training.

ControllerGramDiff ↓CLIP-I ↑DINO-S/16 ↑DINO-B/16 ↑Time (s) ↓
m+fψm+f_\psi (default)5.45290.49280.15160.17095.25
mm (offset-only)7.73550.48890.12000.13895.14

Weakness 4(1): The proposed objective explicitly includes a source-preservation penalty, yet the main results do not directly report source-output LPIPS, source-output CLIP-I, or task-specific identity and structural preservation metrics. These evaluations are particularly important because a uniform latent-space L2 penalty may suppress necessary semantic changes, leading to under-editing, while excessive reward optimization may cause unintended modifications.

AW4(1): Thank you for raising this point. We explicitly quantify source–output preservation across all four SD3 tasks.

TaskLPIPS ↓CLIP-I ↑DINO structure error ↓
Human preference0.08190.96810.00655
Style transfer0.52690.73600.01431
Counterfactual generation0.45030.68770.02575
Facial editing (Editing Term)0.06620.82710.00191

Weakness 4(2): Additionally, the method fixes the controller's Euler–Maruyama rollout to 16 steps for both SD1.5 and SD3. There is no systematic investigation of whether similar results could be achieved using 1, 2, 4, or 8 steps. Given the simple controller architecture and the presence of a trainable latent offset, a few-step or even one-step alternative seems worth investigating.

Q4(2) Have the authors evaluated N∈{1,2,4,8,16}N \in \{1, 2, 4, 8, 16\}?

AW(2)&A4(2): Thank you for raising this point. Sixteen steps are not necessary.

NNGramDiff ↓CLIP-Isty\text{CLIP-I}_{\mathrm{sty}} ↑DINOsty\text{DINO}_{\mathrm{sty}} ↑Time (s) ↓
15.67090.49200.14935.21
25.56900.49260.15025.20
45.50140.49300.15095.26
85.46980.49280.15115.25
165.45290.49280.15165.25

Our lightweight controller makes additional rollout steps inexpensive. Even N=1N=1 achieves comparable CLIP-I and DINO scores, while N=16N=16 improves GramDiff with little additional runtime. We therefore use N=16N=16 as a shared default, although fewer steps remain effective.

Weakness 5 (Minor): Figure 2 is somewhat difficult to follow, particularly the distinction between controller rollout, endpoint feedback computation, cached-target reuse, and controller parameter updates. The diagram also does not clearly convey which operations involve the frozen pretrained backbone and which are performed by the lightweight trainable controller. I encourage the authors to redraw or reorganize this figure into a clearer step-by-step pipeline.

AW5: The revised images have been updated in the paper.

Q1(1): In configurations without diffusion-prior guidance, can the authors demonstrate that reward improvements correspond to actual semantic editing success rather than reward exploitation?

A1(1): Thank you for your question. In the prior-free SD3 preference and style settings, editing quality is also evaluated using metrics not optimized by FASTER, including HPSv2 and reference CLIP-I/DINO (Table 1). Together with the qualitative comparisons in Fig. 4 and Appendix F, Figs. 8–11, these evaluations provide evidence of task-relevant editing beyond improvements in the optimized reward.

Q1(2): Can they provide task-specific editing success rates, source preservation, and perceptual-quality evaluations?

A1(2): Thank you for your question. The other tasks have no predefined editing success rate in our benchmark. For counterfactual editing, target-class logits provide a continuous indicator of editing success rate, and FASTER achieves the highest logit on SD3 (Table 1). Source preservation and perceptual quality are illustrated in Fig. 4 and Appendix F (Figs. 8–11).

Q2(1): Why is the pretrained-prior coefficient zero in most configurations?

A2(1): Thank you for this question. The pretrained prior is a task-dependent regularizer. FASTER already constrains deviations through its source-preservation term; additional prior evaluations introduce a quality–efficiency trade-off, supporting cp=0c_p=0 in configurations where they provide limited benefit.

Q2(2): What happens when it is enabled?

A2(2): Thank you for this question. Enabling the pretrained prior can smooth image textures and suppress fine stylistic details, particularly at higher prior strengths. Appendix I.4 (p. 30) provides side-by-side visual comparisons illustrating this effect.

Reviewer_yn4GOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper adapts stochastic optimal control theory to an image editing task, where it borrows adjoint matching objective to learn the residual drift network in an efficient manner. It starts with the new objective function incorporating the source latent closeness. This gives a new cost function which the authors use adjoint matching objective to fit a lightweight residual drift. The authors derive the reward model from the log density ratio from the pretrained model, which gives a tractable form when differentiated. The experiment results look fair and include most of the well-adopted baselines, and the proposed method show promising results.

Critical strengths

S1. Novel approach. The paper proposes adopting SOC to image editing task. While the core objective function, adjoint matching is a known result, their adoption to image editing task shows interesting engineering effort. For instance, the parameterization of the residual drift and the reward modeling both enable tractable and efficient test-time training. Also I find the image editing quite well suited for test-time training, where the reward would change at every user's query, hence fine-tuning-based approaches would struggle.

S2. Empirical results. The proposed method show promising results in their experiments. While the experiments are not exhaustive, it still covers most of the well-adopted baselines. However, it could be more strengthened by adding more applications, such as different models or reward (application).

Critical weaknesses

W1. The method heavily relies on the findings of the previous work, Adjoint Matching. Still, I find their engineering ideas and application quite interesting and useful to the field.

W2. While I find image editing application to be well suited for the test-time training method, it also feels quite limited at the same time. In most cases, we fine-tune the model to amortize the downstream inference costs for commonly used rewards such as aesthetic, compositionality, etc. Is there any other application where test-time training or FASTER method in particular, can be more beneficial over fine-tuning based methods?

Questions for authors

Q1. Can the proposed method also work in other modalities such as videos or audios? I find there is no theoretical bottleneck in extending the approach.

Q2. Does the most of the efficient gains come from? Is this from residual parameterization or model network design? It might be useful to compare with [1] in the runtime. How does the method behave with respect to the design or capacity of the model? Can the proposed architecture design still work effectively for images that require large content change?

[1] Rout et al., RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Control

3 comments

We thank Reviewer yn4G for recognizing our method’s practical value and for insightful questions.

Weakness 1: The method heavily relies on the findings of the previous work, Adjoint Matching. Still, I find their engineering ideas and application quite interesting and useful to the field.

AW1: Thank you for recognizing the value of our approach. FASTER’s novelty lies in its editing formulation and computational architecture. We separate a source-conditioned endpoint objective from pretrained generation dynamics and learn an independent lightweight controller to execute the edit. This removes pretrained-generator execution from controller rollouts and parameter updates; pretrained prior and semantic guidance, when used, enter through endpoint feedback. Adjoint Matching provides a solver for this control problem; it does not itself provide the editing formulation or the decoupled architecture introduced by FASTER.

Weakness 2: While I find image editing application to be well suited for the test-time training method, it also feels quite limited at the same time. In most cases, we fine-tune the model to amortize the downstream inference costs for commonly used rewards such as aesthetic, compositionality, etc. Is there any other application where test-time training or FASTER method in particular, can be more beneficial over fine-tuning based methods?

Q1: Can the proposed method also work in other modalities such as videos or audios? I find there is no theoretical bottleneck in extending the approach.

AW2&A1: We thank the reviewer for the suggestion. Beyond image editing, FASTER's decoupled framework may also be useful for tasks requiring lightweight, instance-specific test-time adaptation, such as audio editing[1], localized video editing with motion preservation[2], and robotic trajectory refinement[3]. These tasks involve objectives and constraints that vary across individual instances, making FASTER a promising approach for efficient adaptation.

[1] Niu et al., SteerMusic: Enhanced Musical Consistency for Zero-shot Text-Guided and Personalized Music Editing. AAAI, 2026.

[2] Zhu, Lianghan, et al. "Training-free video editing via optical flow-enhanced score distillation." Pattern Recognition (2026): 115077.

[3] Bodmer et al., Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control. CoRL, 2026.

Q2(1): Does the most of the efficient gains come from? Is this from residual parameterization or model network design?

A2(1): Thank you for this question. Neither residual parameterization nor the specific network design is the main source of our speedup. The main gain comes from decoupling editing from pretrained generation: a lightweight controller executes the edit using reusable endpoint feedback, avoiding repeated backbone execution along the editing trajectory.

Q2(2): It might be useful to compare with [1] in the runtime. [1] Rout et al., RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Control

A2(2): Thank you for this question. The end-to-end runtime comparison is shown below.

Output resolutionOursRB-Modulation [1]Speedup
512 × 5122.49 s12.92 s5.20×
768 × 7685.32 s20.82 s3.92×

Our FASTER achieves 5.20× and 3.92× speedups, respectively.

Q2(3): How does the method behave with respect to the design or capacity of the model?

A2(3): Model design refers to using the latent offset mm, the three-layer CNN fψf_\psi, or their combination. Capacity is varied through the CNN hidden width hh, with total trainable parameter counts reported below.

Controller settingParametersGramDiff ↓CLIP-I ↑DINO-S/16 ↑LPIPS ↓Time (s) ↓
mm (offset-only)147,4567.73550.48890.12000.34905.14
fψf_\psi, h=16h=16 (CNN-only)9,4089.06880.48810.15820.45495.25
m+fψm+f_\psi, h=8h=8151,5925.69740.49290.14750.50605.25
m+fψm+f_\psi, h=16h=16 (default)156,8645.45290.49280.15160.52695.25
m+fψm+f_\psi, h=32h=32170,8645.23700.49690.15650.54245.29

Combining mm and fψf_\psi improves GramDiff over either component alone. Although h=32h=32 offers modest gains in GramDiff and CLIP-I, our default h=16h=16 achieves comparable performance with fewer parameters.

Q2(4): Can the proposed architecture design still work effectively for images that require large content change?

A2(4): Yes. Our architecture design remains effective even for images that require substantial content changes. The additional results provided in Appendix I.1 and Figure 16 (p. 27) demonstrate significant content modifications. achieved using the same lightweight controller architecture.

Thank you for preparing the experiments and clarifications. As mentioned, the paper's method mostly relies on the previous work. However, the proposed architecture change enables efficient sampling for specific downstream use cases. The paper could have been more strengthened by providing actual examples. Still I find the paper brings non-trivial contributions to the community.

Thank you for recognizing FASTER’s non-trivial contributions and practical value. We appreciate the suggestion to demonstrate broader downstream applications. Given the limited time available during the rebuttal period, we will explore these applications in future work.

Reviewer_vGhJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FASTER proposes a new test-time reward-guided image editing formulation that uses a specialized lightweight editor instead of repeatedly executing a pretrained diffusion/flow model. Multiple techniques are used while training the editor to improve editing efficiency and quality: Adjoint Matching is used to train the lightweight editor, cached endpoint feedback is reused across updates to reduce computation, and a semantic reward supports text-based edits without backpropagation through the pretrained model. Across multiple metrics FASTER beats prior works on latency and quality.

Critical strengths

1.) Decoupling the image editor from the generative backbone makes sense and is a good idea

2.) Extensive appendix provides significant qualitative and quantitative evidence, strengthening improvement claims of FASTER

Critical weaknesses

1.) Multiple techniques are utilized for training the lightweight editor, however ablations for each technique and its contribution towards latency and quality is missing.

2.) Technical novelty of the work is fairly minor. Specifically, the adjoint matching and endpoint feedback reuse essentially follow "Adjoint Sampling" (Havens et al., 2025), while the semantic editing term is similar to "Contrastive Prompts Improve Disentanglement in Text-to-Image Diffusion Models" (Wu & De la Torre, 2024). These methods are adapted into a new setting in a fairly straightforward way.

Questions for authors

None

3 comments

We thank Reviewer vGhJ for the careful review and constructive feedback on our method’s novelty and the contributions of its individual components.

Weakness 1: Multiple techniques are utilized for training the lightweight editor, however ablations for each technique and its contribution towards latency and quality is missing.

AW1: Thank you for raising this point. The submitted paper already provides extensive analyses of the key design choices: Sec. 5.2 and Fig. 5 compare the Editing Term with CLIPScore optimization, with further evidence in Appendix G; Appendix H and Fig. 15 examine endpoint-feedback reuse across four tasks using both target and validation metrics.

We additionally ablate the CNN branch by comparing the full controller with the offset-only variant on SD3 style transfer. The results are shown below.

Controller settingGramDiff ↓CLIP-I ↑DINO-S/16 ↑DINO-B/16 ↑LPIPS ↓Time (s) ↓
offset-only + CNN(default)5.45290.49280.15160.17090.52695.25
offset-only7.73550.48890.12000.13890.34905.14

Weakness 2: Technical novelty of the work is fairly minor. Specifically, the adjoint matching and endpoint feedback reuse essentially follow "Adjoint Sampling" (Havens et al., 2025), while the semantic editing term is similar to "Contrastive Prompts Improve Disentanglement in Text-to-Image Diffusion Models" (Wu & De la Torre, 2024). These methods are adapted into a new setting in a fairly straightforward way.

AW2: Thank you for your comment. These techniques serve different roles in FASTER’s decoupled editing formulation.

Regarding Adjoint Sampling [1]: While [1] trains diffusion samplers for energy-based distributions, FASTER adapts its endpoint-feedback learning to test-time image editing, training a lightweight controller that decouples editing from pretrained generation dynamics.

Regarding Contrastive Guidance [2]: Although the score-difference formulation is similar, FASTER turns this signal into reusable endpoint supervision for an independent editing controller, enabling multiple lightweight updates per expensive feedback evaluation. This controller-training and feedback-reuse mechanism is absent from Contrastive Guidance [2].

[1] Havens et al., Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching, ICML 2025.

[2] Wu and De la Torre, Contrastive Prompts Improve Disentanglement in Text-to-Image Diffusion Models, 2024.

Thank you for the clarifications. Looking at the mentioned experiments improve my confidence in the method's results. However, I still maintain that the technical contributions are minor and lean more towards combinations of multiple existing methods. Nonetheless, due to strong and consistent evaluations and clear writing I will maintain my current score/rating of 3.

We would like to thank you for your thoughtful review and for your positive comments on our manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.