- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ActDiff, a test-time alignment framework that searches over prompt variants and classifier-free guidance (CFG) scales at each denoising step. The authors employ Monte Carlo Tree Search (MCTS) with a trajectory-feedback prior and provide an information-theoretic analysis of the proposed action space. Experiments across multiple diffusion backbones demonstrate improvements in target reward scores over existing search methods.
Critical strengths
-
The proposed structured action space provides an intuitive and practical alternative to noise-based search, particularly for few-step distilled models and deterministic samplers where intermediate stochastic branching is limited.
-
The method achieves promising improvements across several diffusion backbones and reward functions. Its compatibility with existing particle-based search algorithms further demonstrates the flexibility of the proposed framework.
Critical weaknesses
-
Proposed method requires substantial inference-time computation, as MCTS involves repeated denoising rollouts and reward-model evaluations for generated candidates. These repeated reward evaluations introduce additional computational overhead beyond the reported diffusion-model NFEs, potentially limiting practical applicability. Although the appendix provides a limited wall-clock profiling study, the main benchmark lacks end-to-end latency comparisons. Evaluating performance under matched wall-clock budgets, including reward evaluation costs and prompt generating, would better demonstrate the practical efficiency of the method.
-
Although the ablation studies demonstrate the benefits of jointly searching over prompt variants and CFG scales, it would be interesting to further investigate the effectiveness of step-wise action selection. For example, comparing ActDiff with a static search strategy that selects a single prompt–CFG pair and keeps it fixed throughout denoising could provide additional insights into the benefits of dynamically adjusting these controls.
Questions for authors
-
It would be interesting to investigate whether adjusting the CFG scale in existing methods, such as Best-of-N (BoN) and FK-Steering, also leads to performance improvements. If so, comparing these CFG-adjusted baselines with ActDiff under comparable computational budgets would help clarify the benefits of the proposed method.
-
Additionally, it would be interesting to evaluate whether reward overoptimization occurs by reporting complementary metrics, such as CLIPScore and Aesthetic Score. Such evaluations could help determine whether improvements in the optimized reward come at the expense of other aspects of image quality or prompt alignment.
We thank Reviewer RUWD for the critical feedback and detailed comments. We address your questions point by point below.
W1: Wall-clock matched comparison
We thank the reviewer for this suggestion. We now report end-to-end wall-clock results on SiD under matched NFEs, including prompt rewriting, text encoding, denoising, decoding, and reward scoring; all methods were run on a single NVIDIA RTX A6000 GPU. Table 3 breaks down the time per prompt at 128 NFE. ActDiff's per-prompt time is comparable to that of DAS and lower than those of Best-of-N and FK-Steering. ActDiff uses fewer NFEs than the budget allows and calls the reward model far fewer times than Best-of-N and FK-Steering, which reduces both decoding and reward-scoring time. Prompt rewriting accounts for only a small fraction of the total time. Therefore, the additional processing adds no significant compute overhead compared with the strongest competitors.
Table 3. Time breakdown at 128 NFE on SiD (ImageReward as the search reward; seconds per prompt). Total also includes minor bookkeeping overhead not listed separately.
W2: Static search strategy
In response to both Reviewer hnox and Reviewer RUWD, Table 2 below reports results for a static strategy, which fixes a single prompt–guidance pair for the entire trajectory, and for random action selection from the same bank. Fix pair action is least favorable since a good trace typically needs changing prompt-guidance pair inside the trajectory
Table 2. Same-bank controls with a single initial noise (SiD, 4 steps, ImageReward as the search reward, n = 100).
Q1: CFG-adjusted baselines
We have evaluated CFG-adjusted Best-of-N and FK-Steering on a fixed 100-prompt subset of our main benchmark, using SiD with ImageReward guidance and a budget of about 128 NFE. Under comparable compute, adding the same CFG grid to Best-of-N lowers ImageReward, because BoN cannot adjust CFG within a trajectory and therefore spends more of its budget on over-saturated samples. For FK-Steering, sharing the CFG bank across particles yields a modest ImageReward gain, but the result remains below ActDiff.
Table 5. Baselines given under augmented CFG bank . The CFG bank gives a modest improvement; however, the initial seed remains the dominant factor for particle-based algorithms.
Q2: Complementary metrics (CLIPScore and Aesthetic)
We compared ActDiff with the two most competitive baselines under ImageReward-guided search, following the settings of our main experiment. The results are mixed, but CLIPScore and LAION Aesthetic remain close across methods, and the differences in these two metrics are small. We acknowledge that ActDiff trails slightly in LAION Aesthetic. We attribute this to a reward trade-off within the model's capacity rather than to severe reward hacking.
Table 6. CLIPScore (ViT-B/32) and LAION Aesthetic of the returned images (ImageReward as the search reward; scored against the original prompt).