- Rating
- 3(+1)
- Confidence
- 4(+1)
- Informativeness
- 3
- Presentation
- 3(+1)
- Soundness
- 3
Read review
Paper summary
This paper proposes FASTER, a test-time reward-guided image editing method that decouples pretrained generative knowledge from the editing dynamics. Instead of repeatedly executing a large diffusion or flow model, FASTER constructs a source-conditioned endpoint distribution incorporating reward improvement, source preservation, and pretrained-prior preferences. A lightweight stochastic controller is then trained independently for each source image and editing objective. By adopting Brownian reference dynamics and reciprocal adjoint matching, the method reuses endpoint supervision through Brownian-bridge sampling, reducing the computational cost of repeated optimization. The authors additionally introduce a semantic Editing Term derived from differences between source- and target-conditioned velocity predictions. Experiments on SD3 and SD1.5 cover four reward-guided editing tasks. FASTER achieves the best target metrics across all four SD3 tasks and reports substantial speedups relative to ITOC, reaching up to 6.91× on SD3 and 24.14× on SD1.5.
Critical strengths
-
Decoupling the editing process from pretrained generative dynamics is an appealing direction. If successful editing can be achieved using lightweight test-time optimization without repeated diffusion-model execution, the implications could extend beyond computational acceleration.
-
The use of Brownian reference dynamics simplifies adjoint supervision, while Brownian-bridge regression allows expensive endpoint feedback to be reused. The overall formulation is mathematically coherent and computationally motivated.
-
FASTER achieves the best optimized reward metrics across all four SD3 tasks and improves several independent validation metrics. The speedups relative to ITOC are substantial and demonstrate the practical feasibility of lightweight per-instance optimization.
Critical weaknesses
-
A successful image edit should introduce the requested semantic change while preserving the unrelated content and perceptual quality of the source image. However, the main experimental results primarily demonstrate improvements in the reward functions being optimized.
-
A central motivation is to retain pretrained generative knowledge while separating editing from pretrained generation trajectories. However, Table 2 shows that the pretrained-prior coefficient is zero in five of the eight experimental configurations, including three of the four SD3 tasks. Furthermore, when ατ² = 1 and the prior coefficient is zero, Eq. (11) simplifies exactly to a scaled reward gradient. Consequently, several principal experiments train the controller using reward-gradient supervision without explicitly querying the diffusion backbone. This observation is potentially important: it suggests that certain editing objectives may be optimized without pretrained generative dynamics. However, the paper does not adequately investigate why this works or whether the resulting images retain the semantic and perceptual properties expected from generative editing. Additionally, the practical prior estimator corresponds to a noisy-prior surrogate rather than necessarily recovering the clean pretrained distribution used in the theoretical formulation. The paper lacks systematic ablations over prior strength, noise-time ranges, and the number of prior-score queries. These experiments would clarify whether pretrained-prior guidance provides meaningful benefits or whether the principal results are better understood as reward-driven latent optimization.
-
The general idea of optimizing a neural representation separately for an individual image is not entirely new, as related ideas appear in Deep Image Prior [1] and Text2LIVE [2]. Although FASTER introduces a different stochastic-control formulation, the practical necessity of this formulation is not sufficiently established. The controller contains a full latent-shaped trainable offset and a small three-layer convolutional network. In the SD3 configuration, the trainable offset accounts for the vast majority of its parameters. This raises the question of whether much of the observed performance could be achieved by directly optimizing the latent offset.
-
The proposed objective explicitly includes a source-preservation penalty, yet the main results do not directly report source-output LPIPS, source-output CLIP-I, or task-specific identity and structural preservation metrics. These evaluations are particularly important because a uniform latent-space L2 penalty may suppress necessary semantic changes, leading to under-editing, while excessive reward optimization may cause unintended modifications. Additionally, the method fixes the controller's Euler–Maruyama rollout to 16 steps for both SD1.5 and SD3. There is no systematic investigation of whether similar results could be achieved using 1, 2, 4, or 8 steps. Given the simple controller architecture and the presence of a trainable latent offset, a few-step or even one-step alternative seems worth investigating.
Minor Weakness:
- Figure 2 is somewhat difficult to follow, particularly the distinction between controller rollout, endpoint feedback computation, cached-target reuse, and controller parameter updates. The diagram also does not clearly convey which operations involve the frozen pretrained backbone and which are performed by the lightweight trainable controller. I encourage the authors to redraw or reorganize this figure into a clearer step-by-step pipeline.
Ref:
[1] Deep image prior. (CVPR)
[2] Text2LIVE: Text-driven layered image and video editing(ECCV)
Questions for authors
In rebuttal, I want to see:
-
In configurations without diffusion-prior guidance, can the authors demonstrate that reward improvements correspond to actual semantic editing success rather than reward exploitation? Can they provide task-specific editing success rates, source preservation, and perceptual-quality evaluations?
-
Why is the pretrained-prior coefficient zero in most configurations? What happens when it is enabled? Can the authors investigate different prior strengths, noise-time ranges, and multiple noisy-score queries to clarify whether the pretrained prior meaningfully improves the editing results?
-
How does FASTER compare with optimizing only its trainable latent offset, using a deterministic controller, or performing direct regularized latent optimization? Can the authors isolate the individual benefits of Brownian-bridge regression and learned stochastic dynamics under matched computational budgets?
-
Why are 16 controller rollout steps necessary? Have the authors evaluated N ∈ {1, 2, 4, 8, 16}? How does the overall quality-efficiency trade-off change when K, M, and N are jointly varied under a fixed wall-clock or reward-query budget?
-
Could the authors provide more extensive side-by-side qualitative comparisons across diverse editing instructions Additionally, could the authors include comparisons with recent inversion-free editing methods, such as FlowEdit [1], FlowChef [2], and ChordEdit [3].
Ref:
[1] FlowEdit: Inversion-free text-based editing using pre-trained flow models.(ICCV)
[2] FlowChef: Steering of rectified flow models for controlled generations.(ICCV)
[3] ChordEdit: One-step low-energy transport for image editing. (CVPR)
8 comments
Part 2
Weakness 2(4): The paper lacks systematic ablations over prior strength, noise-time ranges, and the number of prior-score queries. These experiments would clarify whether pretrained-prior guidance provides meaningful benefits or whether the principal results are better understood as reward-driven latent optimization.
Q2(3) Can the authors investigate different prior strengths, noise-time ranges, and multiple noisy-score queries to clarify whether the pretrained prior meaningfully improves the editing results?
AW2(4)&A2(3): Thank you for your comment. Here, we have prepared three tables of quantitative experiments for your reference. Empirically, the prior term weakens editing by pulling images toward an “average image”, so its use should depend on the specific editing performance. The noise-time range generally involves a trade-off; we recommend , which offers a reasonable prior proxy while maintaining a reliable prior. The number of prior-score queries has little effect on performance but increases runtime, so we recommend using a single query.
Effect of pretrained-prior strength . Here, we set noise-time range: and 1 prior-score query when prior guidance is enabled.
| GramDiff | CLIP-I | DINO-S/16 | Time (s) | |
|---|---|---|---|---|
| 0 (no prior guidance) | 5.4529 | 0.4928 | 0.1516 | 5.26 |
| 0.001 | 5.4596 | 0.4925 | 0.1517 | 6.59 |
| 0.003 | 5.4634 | 0.4924 | 0.1520 | 6.60 |
| 0.01 | 5.4850 | 0.4927 | 0.1527 | 6.60 |
| 0.03 | 5.5458 | 0.4928 | 0.1546 | 6.59 |
| 0.1 | 5.6309 | 0.4961 | 0.1623 | 6.58 |
| 0.3 | 5.9043 | 0.4991 | 0.1758 | 6.59 |
| 1.0 | 6.7310 | 0.4854 | 0.1884 | 6.58 |
Effect of the noise-time range used for prior estimation. Here we set and 1 prior-score query.
| Noise-time range | GramDiff | CLIP-I | DINO-S/16 | Time (s) |
|---|---|---|---|---|
| 5.8066 | 0.4968 | 0.1788 | 6.58 | |
| 5.6309 | 0.4961 | 0.1623 | 6.58 | |
| 5.4900 | 0.4932 | 0.1514 | 6.58 |
Effect of the number of prior-score queries. Here we fix and noise-time range: .
| Prior-score queries | GramDiff | CLIP-I | DINO-S/16 | Time (s) |
|---|---|---|---|---|
| 1 | 5.6309 | 0.4961 | 0.1623 | 6.58 |
| 2 | 5.6219 | 0.4952 | 0.1627 | 7.88 |
| 4 | 5.6209 | 0.4951 | 0.1629 | 10.35 |
Weakness 3(1): The general idea of optimizing a neural representation separately for an individual image is not entirely new, as related ideas appear in Deep Image Prior [1] and Text2LIVE [2]. Although FASTER introduces a different stochastic-control formulation, the practical necessity of this formulation is not sufficiently established.
AW3(1): Thank you for pointing this out. FASTER distinguishes itself from DIP [1] and Text2LIVE [2] through an endpoint-based SOC formulation that decouples editing from pretrained generation. The necessity of our SOC formulation lies in enabling multiple lightweight controller updates to reuse endpoint feedback, thereby reducing optimization cost. This practical benefit is supported by improvements over GA (Table 1) and the effect of feedback reuse at matched numbers of feedback evaluations (Figure 15).
[1] Ulyanov et al. Deep Image Prior. CVPR, 2018.
[2] Bar-Tal et al. Text2LIVE: Text-Driven Layered Image and Video Editing. ECCV, 2022.
Weakness 3(2) The controller contains a full latent-shaped trainable offset and a small three-layer convolutional network. In the SD3 configuration, the trainable offset accounts for the vast majority of its parameters. This raises the question of whether much of the observed performance could be achieved by directly optimizing the latent offset.
AW3(2): Thank you for pointing this out. Optimizing the latent offset alone is effective, but performs worse than the combined controller (CNN). Although accounts for most parameters, it represents a state- and time-independent offset, whereas the CNN provides state- and time-dependent corrections. We compare the two settings under identical training.
| Controller | GramDiff ↓ | CLIP-I ↑ | DINO-S/16 ↑ | DINO-B/16 ↑ | Time (s) ↓ |
|---|---|---|---|---|---|
| (default) | 5.4529 | 0.4928 | 0.1516 | 0.1709 | 5.25 |
| (offset-only) | 7.7355 | 0.4889 | 0.1200 | 0.1389 | 5.14 |
Part 3
Weakness 4(1): The proposed objective explicitly includes a source-preservation penalty, yet the main results do not directly report source-output LPIPS, source-output CLIP-I, or task-specific identity and structural preservation metrics. These evaluations are particularly important because a uniform latent-space L2 penalty may suppress necessary semantic changes, leading to under-editing, while excessive reward optimization may cause unintended modifications.
AW4(1): Thank you for raising this point. We explicitly quantify source–output preservation across all four SD3 tasks.
| Task | LPIPS ↓ | CLIP-I ↑ | DINO structure error ↓ |
|---|---|---|---|
| Human preference | 0.0819 | 0.9681 | 0.00655 |
| Style transfer | 0.5269 | 0.7360 | 0.01431 |
| Counterfactual generation | 0.4503 | 0.6877 | 0.02575 |
| Facial editing (Editing Term) | 0.0662 | 0.8271 | 0.00191 |
Weakness 4(2): Additionally, the method fixes the controller's Euler–Maruyama rollout to 16 steps for both SD1.5 and SD3. There is no systematic investigation of whether similar results could be achieved using 1, 2, 4, or 8 steps. Given the simple controller architecture and the presence of a trainable latent offset, a few-step or even one-step alternative seems worth investigating.
Q4(2) Have the authors evaluated ?
AW(2)&A4(2): Thank you for raising this point. Sixteen steps are not necessary.
| GramDiff ↓ | ↑ | ↑ | Time (s) ↓ | |
|---|---|---|---|---|
| 1 | 5.6709 | 0.4920 | 0.1493 | 5.21 |
| 2 | 5.5690 | 0.4926 | 0.1502 | 5.20 |
| 4 | 5.5014 | 0.4930 | 0.1509 | 5.26 |
| 8 | 5.4698 | 0.4928 | 0.1511 | 5.25 |
| 16 | 5.4529 | 0.4928 | 0.1516 | 5.25 |
Our lightweight controller makes additional rollout steps inexpensive. Even achieves comparable CLIP-I and DINO scores, while improves GramDiff with little additional runtime. We therefore use as a shared default, although fewer steps remain effective.
Weakness 5 (Minor): Figure 2 is somewhat difficult to follow, particularly the distinction between controller rollout, endpoint feedback computation, cached-target reuse, and controller parameter updates. The diagram also does not clearly convey which operations involve the frozen pretrained backbone and which are performed by the lightweight trainable controller. I encourage the authors to redraw or reorganize this figure into a clearer step-by-step pipeline.
AW5: The revised images have been updated in the paper.
Q1(1): In configurations without diffusion-prior guidance, can the authors demonstrate that reward improvements correspond to actual semantic editing success rather than reward exploitation?
A1(1): Thank you for your question. In the prior-free SD3 preference and style settings, editing quality is also evaluated using metrics not optimized by FASTER, including HPSv2 and reference CLIP-I/DINO (Table 1). Together with the qualitative comparisons in Fig. 4 and Appendix F, Figs. 8–11, these evaluations provide evidence of task-relevant editing beyond improvements in the optimized reward.
Q1(2): Can they provide task-specific editing success rates, source preservation, and perceptual-quality evaluations?
A1(2): Thank you for your question. The other tasks have no predefined editing success rate in our benchmark. For counterfactual editing, target-class logits provide a continuous indicator of editing success rate, and FASTER achieves the highest logit on SD3 (Table 1). Source preservation and perceptual quality are illustrated in Fig. 4 and Appendix F (Figs. 8–11).
Q2(1): Why is the pretrained-prior coefficient zero in most configurations?
A2(1): Thank you for this question. The pretrained prior is a task-dependent regularizer. FASTER already constrains deviations through its source-preservation term; additional prior evaluations introduce a quality–efficiency trade-off, supporting in configurations where they provide limited benefit.
Q2(2): What happens when it is enabled?
A2(2): Thank you for this question. Enabling the pretrained prior can smooth image textures and suppress fine stylistic details, particularly at higher prior strengths. Appendix I.4 (p. 30) provides side-by-side visual comparisons illustrating this effect.
Part 1
We thank Reviewer fMTF for the thorough review and valuable suggestions for strengthening our evaluation and clarifying the methodological details.
AW1: Our evaluation is not limited to improvements in optimized rewards: it also assesses perceptual quality and provides qualitative evidence of source-content preservation. : Table 1 also reports multiple non-target validation metrics assessing perceptual quality, semantic alignment, and style similarity. Figure 4 and Appendix F (Figures 8–11) further provide qualitative comparisons of editing effects and retained source content.
AW2(1): Thanks for your insights. We would like to clarify that reward-gradient supervision does not imply the absence of source preservation. As shown in Eq. (5), even when ατ2=1 and the pretrained-prior coefficient is zero, the target distribution remains
\propto \exp\left( -\frac{|\mathbf{y}-\mathbf{s}|^2}{2\tau^2} +\beta r(\mathbf{y}) \right). \tag{5}$$ The source-preservation term is already incorporated into the Brownian reference dynamics (Eq. (6)), explaining its cancellation in Eq. (11). Moreover, the source image itself provides a strong prior, making explicit pretrained-prior guidance optional for certain editing objectives. We further investigate its effects through the ablation experiments in **AW2(4)**. > **Weakness 2(2)**: This observation is potentially important: it suggests that certain editing objectives may be optimized without pretrained generative dynamics. However, the paper does not adequately investigate why this works or whether the resulting images retain the semantic and perceptual properties expected from generative editing. **AW2(2):** Thanks for your question. FASTER retains pretrained generative knowledge without relying on pretrained generation trajectories. Specifically, it uses the **pretrained latent space and frozen decoder for image editing and evaluation**, while the pretrained diffusion backbone can provide prior scores (Eq. 11) and semantic feedback (Sec. 4.3) at candidate endpoints, rather than executing the editing process. As explained in Sections 4.1–4.2 and Appendices B–C, the source image itself provides a strong structural prior, allowing certain tasks to rely primarily on reward gradients. Our evaluation goes beyond optimized rewards, with additional metrics in Table 1 and visual results in Figures 4, 8, and 10–11. The need for explicit pretrained-prior guidance is task-dependent, further investigated through our new ablation on $c_p$ in Appendix I.4. > **Weakness 2(3)**: Additionally, the practical prior estimator corresponds to a noisy-prior surrogate rather than necessarily recovering the clean pretrained distribution used in the theoretical formulation. **AW2(3)**: Thank you for your comment. Using a noisy-prior surrogate is a common practice in **optimization with pretrained diffusion priors**; for example, EAM [1], DreamFusion and Feng & Bouman (2024) [3] also adopts this practice. In addition, for velocity-field models such as SD3, we cannot directly obtain the prior term at the data timestep. Therefore, using a small-noise prior proxy avoids the time cost of training a prior model and instead allows us to immediately leverage an already well-trained diffusion prior. [1]: Shin, Jeongwoo, et al. "Efficient Adjoint Matching for Fine-tuning Diffusion Models." *arXiv preprint arXiv:2605.11480* (2026). [2] Poole et al. *DreamFusion: Text-to-3D using 2D Diffusion.* ICLR, 2023. [3] Feng and Bouman. *Variational Bayesian Imaging with an Efficient Surrogate Score-based Prior.* TMLR, 2024.