- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors observe on ImageNet that, as the branching factor increases, Top-K intermediate candidate selection in verifier-guided diffusion inference-time search leads to higher precision but lower recall and worse FID in the final output set. To mitigate this issue, the authors propose a set-based candidate selection objective: while selecting candidates with high verifier scores, it encourages the first- and second-order statistics of the selected subset in Inception feature space to match those of the entire current candidate population. The experiments show that the proposed method can improve the FID of the final output set relative to Top-K.
Critical strengths
none
Critical weaknesses
- The motivation for studying this search mechanism is insufficient.
The authors take fixed-width pruning at each round as given and investigate the effects of different candidate retention rules under this setting. However, this does not establish that repeated global pruning with a constant width of 50 is a reasonable or effective computational allocation strategy. The existing experiments do not explore alternative search-budget allocation strategies or compare against other search strategies. Meanwhile, Table 2 shows that ordinary DDIM25 sampling achieves an FID of 9.544 and recall of 0.369, compared with 12.162 and 0.238 for Moment-QUBO. The latter increases ADM precision from 0.912 to 0.935, approximately 2.3 percentage points, while increasing the observed inference time per 50 images from 23.96 to 64.78 seconds. Thus, this increase in precision comes at the cost of nearly three times the observed inference time, a higher FID, and substantially lower recall. It therefore remains unclear whether this quality–cost trade-off is practically favorable relative to ordinary sampling. Mitigating degradation relative to Top-K within the prescribed search framework is therefore insufficient to establish the practical utility of the framework itself.
Moreover, practical generation scenarios often involve generating a single target image under a given condition. The search method used in the paper requires maintaining trajectories. Thus, for DiT at , a full search population requires approximately image-level denoising network evaluations, compared with 50 evaluations for generating a single image using ordinary DDIM sampling—a roughly 112-fold increase in cumulative denoising computation. The use cases that justify this additional computation remain unclear.
- The proposed method is primarily driven by the evaluation metric, which limits the interpretation of the results.
It constructs an FID-motivated proxy by matching the first- and second-order moments of Inception features, while FID is also the central metric used to demonstrate its effectiveness. This makes it difficult to distinguish whether the gains reflect broader improvements in generation quality or primarily improved distribution matching in Inception space. The authors’ experiments also support this concern: for example, in Table 27, DPP performs better on CLIP-CMMD. Evaluations directly related to other downstream task requirements are currently lacking, leaving it unclear whether the gains correspond to broader improvements in generation quality.
- The manuscript is not self-contained.
The manuscript retains a substantial amount of internal experimental and automated workflow language, including terms describing conditional termination or acceptance, such as “gate passes” and “image gates,” as well as various manifests, source hashes, and generated files. The final “Submission checks” section, in particular, shows traces of automated checking. Moreover, the paper repeatedly refers to materials that are not visible in the manuscript, and the authors have not submitted the relevant materials.
Questions for authors
-
What justifies the use of a fixed-width, repeated global-pruning strategy, given that ordinary DDIM sampling achieves better FID and recall at substantially lower cost in Table 2? What intended application or output requirement makes this trade-off worthwhile?
-
Given that DPP achieves better CLIP-CMMD at closely matched precision in Table 27, more evidence is needed to establish that the proposed method provides broader improvements in generation quality beyond improved distribution matching in Inception space.
-
The authors should compare the proposed repeated fixed-width global pruning strategy with alternative search or budget-allocation strategies under matched computational budgets. Such comparisons are necessary to justify the choice of this particular search design.
3 comments
W3: Internal workflow language and unavailable material
We have removed internal workflow terminology and the submission-check narrative, and made the method and evaluation protocols explicit in the manuscript.
Section 3 gives a complete search round: expand the retained parents, obtain predicted-clean images and features, form the current-candidate reference, jointly select K states, and retain them as the next parents. The objective and greedy-plus-swap solver are specified directly, with derivations in the appendix.
Section 4 and Table 1 identify the early policy producing each final candidate pool and the selector applied to it. The full-class, mixed-feature crossover and the smaller same-feature diagnostic are distinguished. Appendix K.3 gives the SDXL time grid, re-noising, DDIM updates, clean previews, feature preprocessing, prompt split and NFE accounting. Section 8 reports end-to-end timing separately from denoiser NFE.
We have prepared the revised PDF and an anonymous supplementary archive to accompany the response. The archive contains selection code, generation and evaluation entry points, configurations, prompt splits, tests and recorded numerical results. README.md covers installation and minimal use, REPRODUCTION.md maps experiments to commands, and EVALUATION.md gives metric environments and conventions.
External model weights and evaluation assets must be obtained separately; full image archives and large activation caches are not bundled. Package checks cover CPU tests and recorded-result calculations, rather than a clean-machine regeneration of every experiment. These requirements are stated in the documentation.
Q3: Alternative search and budget-allocation strategies
Table 2 compares longer trajectories, independent generation followed by selection, and repeated branching at 5,600 NFE per 50 DiT outputs. DDIM56 at CFG 1.5 achieves FID 2.222 and recall 0.599; IID112 plus Moment achieves 4.321 and 0.484; dynamic Moment achieves 12.162 and 0.238. Their precisions are 0.802, 0.866 and 0.935. Top-K and DPP are included for independent and dynamic selection. The resulting trade-off favors simpler allocations on FID and recall at these settings.
Table 3 compares DDIM168, IID32 DDIM42 plus selection and dynamic SDXL search at 2,688 NFE per eight images. IID Moment has greater LPIPS, while dynamic Moment has higher ImageReward; their GenEval difference is unresolved. The selected IID policies differ from dynamic Moment by approximately 0.011 in mean development CLIP score.
Controlled timing with the same latent microbatch gives 182.92 s for DDIM168, 197.98 s for IID Moment and 257.27 s for dynamic Moment. This study uses two development prompts, each measured twice per method (four measurements per method).
Appendices K.2 and K.4 additionally vary population width and the rounds using Moment. Appendix K.5 tests a small shared-descendant allocation comparison: feedback changes final group count by +0.0625 relative to fixed branching, with an interval spanning zero, and provides no demonstrated improvement over static allocation.
Thank you for the rebuttal, but I still have some concerns.
-
The case for repeated fixed-width pruning remains unclear. The rebuttal acknowledges that there are settings where “simpler sampling gives better coverage and latency.” At the same denoiser NFE, both longer independent trajectories and independent generation followed by selection achieve lower FID and higher recall than dynamic Moment in the DiT experiments (Table 2). It remains unclear which application would benefit enough from the higher ADM precision to justify the loss of coverage and the added latency observed against DDIM25 (Appendix Table 43).
-
The multi-image results weaken the proposed use case. A full DiT search at B=2 uses about as much denoising compute as DDIM25 if only one image is delivered, or $2.24\times$$ per image on average if all 50 are delivered. The value of delivering 50 images depends on how many satisfy the condition and offer meaningfully different choices. In the SDXL experiment, dynamic Moment produces at least two GenEval-passing images with pairwise LPIPS above 0.4 on 60.9% of prompts, compared with 75.0% for independent sampling + Moment at the same denoiser NFE and 73.4% for the lower-cost DDIM40 (Appendix Table 52). Nearly 40% of prompts fail to yield even one qualifying pair under dynamic Moment. Both simpler methods have higher observed delivery rates at all three exploratory thresholds. These results leave the practical benefit of repeated search unclear even in the proposed multi-image setting.
-
Evidence beyond Inception space remains limited. At closely matched ADM precision in the SiT comparison, Moment achieves lower FID and higher ADM recall, while DPP achieves lower CLIP-CMMD (Table 4). This leaves open whether the gains measured in Inception space translate into broader improvements in semantic coverage or overall generation quality.
Therefore, I maintain the original score.
Thank you for the detailed comments. We have clarified the output requirement and added comparisons that separate the value of search from the value of the retention rule.
W1 / Q1: Motivation for fixed-width repeated pruning
The application we consider is delivering several condition-following candidates under the same prompt. The SDXL study provides direct task evidence: dynamic Moment obtains 63.50% GenEval macro success versus 54.78% for DDIM168 at equal denoiser NFE, a difference of 8.72 percentage points (conditional 95% CI [5.18, 12.38]). This measures the complete search strategy relative to ordinary sampling.
The benefit attributable to changing retention is tested separately against dynamic Top-K under the same search settings. Moment increases LPIPS from 0.189 to 0.398 with the same 649/1,024 passing images; their GenEval difference is unresolved. This supports using a population-aware rule to reduce concentration when search is used for condition-following generation.
For DiT, DDIM25 at CFG 3 obtains FID 9.544 and recall 0.369 in 23.96 s per 50 outputs, compared with 12.162, 0.238 and 64.78 s for dynamic Moment. ADM precision is 0.91232 versus 0.93484. Sections 6, 7.1 and 8 now present both the favorable condition-following results and the cases where simpler sampling gives better coverage and latency.
The 112-fold computation comparison
Our DiT search returns 50 images per condition. At B=2 it requires 5,600 image-level NFE for the complete population, averaging 112 NFE per delivered image. Ordinary DDIM25 uses 50 NFE per image under the same convention of counting both classifier-free-guidance rows. The ratio is therefore 2.24 when all 50 outputs are delivered. This is an average population cost, not the cost of running the search to obtain only one image.
If only one output is required and the full search population is generated, the approximately 112-fold comparison with a single ordinary image is applicable. Our experiments do not establish that this single-output use is worthwhile. The revised introduction explicitly states the multi-output setting, and the SDXL study uses a smaller set of eight images per prompt.
W2 / Q2: Gains beyond Inception-space distribution matching
The SDXL experiment evaluates retention outside Inception space: it uses DINOv2 features, online CLIP similarity and GenEval condition-satisfaction judgments. Relative to dynamic Top-K, Moment increases all-output LPIPS by 0.2088 (conditional 95% CI [0.1840, 0.2330]). Among 83 prompts with at least two passing outputs under both policies, LPIPS among passing images increases by 0.1901 ([0.1588, 0.2211]).
Against dynamic DPP, all-output ImageReward favors Moment by 0.0553 ([0.0152, 0.0992]), while the LPIPS difference is unresolved. LPIPS measures perceptual separation, and ImageReward is a learned preference score; neither directly measures semantic coverage.
In the independent matched-precision SiT comparison, Moment improves FID from 7.516 to 6.429 and ADM recall from 0.3922 to 0.4125, while CLIP-CMMD favors DPP: 0.33535 versus 0.34791. The Moment-minus-DPP CLIP-CMMD difference is +0.01256 ([0.00910, 0.01584]). Section 7.1 therefore limits that advantage to Inception-space metrics, and Appendix K.3 separately reports the SDXL task and perceptual outcomes.